VisionMate: AI Companion for Scene Awareness

Abstract

The aim is to design and develop an end-to-end multimodal vision-language-speech interface which will enhance the navigational independence and awareness of the Blind and Visually Impaired (BVI) people. The system uses the idea of the “Contextual Gap,” based on the live camera capture and using computer vision and OCR technology, to fill the gap of the wayfinding of BVI people in urban areas. The framework introduces an adaptive perspective calibration module that dynamically adjusts the camera's spatial perception based on user height, ensuring inclusive accessibility for individuals with diverse physical statures, including dwarfism. The framework also employs speech synthesis and voice command interfaces to provide hands-free operation with reduced dependency on manual assistance. Though the system is primarily intended to provide assistance to visually impaired individuals, the modular nature of the application can be extended to support other functionalities such as product search and shopping for the general public.