Rating Like Blind Users: Automated App Evaluation by Detecting Critical Accessibility Issues

Abstract

Mobile applications are essential for daily life, and ensuring their accessibility is critical for visually impaired users. However, as a minority, their feedback is often underrepresented in app reviews. Current rating systems do not adequately reflect their experiences, hindering developers from focusing on core accessibility issues and blind users from identifying accessible apps. To address this, we interviewed eight blind individuals to investigate their interaction patterns and accessibility barriers. The findings reveal that blind users focus on understanding apps’ core functionalities and exhibit selective interaction behaviors. The interviews also highlighted four prevalent accessibility issues they encounter. Drawing on these insights and cognitive map models, we propose MAASdroid, an automated Mobile Application Accessibility Scoring tool. Informed by the fact that blind users rely on non-visual cues to build a mental representation of the GUI, MAASdroid is grounded in cognitive map theory to model their app exploration. It constructs the app map using Fastbot, abstracts node features based on GUI properties, extracts arteries, and computes accessibility scores by analyzing four types of issues along arteries. Experimental results demonstrate that MAASdroid achieves strong correlation with actual ratings from blind users, offering a scoring system that closely aligns with their experience.