Designing for all? Accessibility of native android interfaces from large language models

Abstract

The emergence of large language models (LLMs) has significantly impacted the automation of software development tasks, including the generation of mobile application code. This research investigates how LLMs, such as ChatGPT, can support the creation of accessible native Android applications. Specifically, it evaluates whether the generated code aligns with established accessibility guidelines, considering variations in screen layouts, prompt styles, and interface design methods. Four empirical studies were carried out to assess accessibility in seven different types of mobile UI. The first study assessed screens built with various layout techniques, while the second focused on Jetpack Compose and compared outputs from multiple LLMs (e.g., Sabiá, GPT). The third experiment explored the influence of English-language prompts on accessibility outcomes, and the fourth involved an evaluation of Github Copilot. In all studies, 702 accessibility-related issues were identified. Jetpack Compose consistently led to more accessible interfaces compared to other layout options, and prompts in English resulted in fewer errors. Surprisingly, prompts that explicitly asked for accessibility often introduced more problems, suggesting that current models struggle to accurately interpret and implement accessibility directives. These findings underscore the need for improved prompt engineering and more robust LLM-generated code to improve accessibility in AI-assisted mobile development.