An Assistive Object Recognition System for Visually Impaired Individuals Using Mask R-CNN
TL;DR
Experimental results demonstrate that the proposed framework maintains consistent performance across diverse operating conditions, highlighting its robustness for real-world deployment, and offers significant potential to enhance the autonomy, safety, and quality of life of visually impaired individuals.
Abstract
Blind and partially sighted individuals often face significant challenges in identifying and interacting with objects in their surroundings, which limits their independence and daily functioning. To address this issue, this study proposes a deep learning-based assistive framework built on mask region-based convolutional neural network (Mask R-CNN) for real-time object detection and recognition in both indoor and outdoor environments. The proposed system integrates sensor-based obstacle awareness with vision-based object analysis to enhance environmental perception, enabling robust identification of surrounding objects at varying distances and under varying environmental conditions. In this study, object detection refers to the localization and identification of objects in the scene, whereas object recognition refers to the classification of detected objects into predefined semantic categories. The object detection performance is evaluated across multiple object-to-system distances, achieving a maximum accuracy of 99.88% at 100 cm in indoor environments and 99.64% at 120 cm in outdoor environments. Further, object recognition is assessed across five representative categories, namely person, chair, table, stairs, and washroom. The proposed framework achieves average recognition accuracies of 97.29% and 98.53% in indoor and outdoor environments, respectively, yielding an overall average of 97.91%. In addition, the optimized implementation achieves an inference speed of 10-12 frames per second (FPS), corresponding to a processing time of 0.0833-0.1000 s per frame, demonstrating suitability for real-time assistive applications. Experimental results demonstrate that the proposed framework maintains consistent performance across diverse operating conditions, highlighting its robustness for real-world deployment. Consequently, the system offers significant potential to enhance the autonomy, safety, and quality of life of visually impaired individuals.