This work introduces LipCoder, a voice-centric programming toolkit designed to deliver editor-level functionality through auditory and speech-based interfaces, and quantitative trends and qualitative feedback point to directions for auditory-first design that may broaden accessibility in speech-driven coding environments.
Abstract
AI-assisted programming environments have accelerated software development, giving rise to new paradigms like vibe coding. However, their benefits remain largely inaccessible to visually impaired programmers, as existing screen readers and assistive tools offer limited support for these emerging workflows. We introduce LipCoder, a voice-centric programming toolkit designed to deliver editor-level functionality through auditory and speech-based interfaces. LipCoder offers features comprising speech feedback and earcon cues for comprehension and validation, as well as natural language input for navigation and modification. In an exploratory evaluation, 5 visually impaired programmers performed a series of coding tasks comparing LipCoder with a baseline of VSCode, Copilot, and VoiceOver. Quantitative trends and qualitative feedback point to directions for auditory-first design that may broaden accessibility in speech-driven coding environments.
As multimodal interaction becomes central to digital learning, technical interfaces, such as eBooks and conversational agents, present unique accessibility challenges for users relying on the auditory modality. While modern Text-to-Speech (TTS) systems excel at rendering natural language, they often fail to convey the...
Marco Cardia, Letizia Angileri, M. Buzzi et al.· 0 citations
Audiodescription is essential for making audiovisual content accessible to visually impaired individuals, converting visual information into audio descriptions. Typically done by human narrators, audiodescription can be complemented by virtual technologies with synthesized voice. This study evaluated the accessibility...
A. D. de Oliveira· Revista Brasileira Multidisc...· 0 citations
Speech editing is gradually making its way into everyday applications. This paper presents Speech Editor, a practical software system built upon SpeechTripleNet for disentangled speech representation learning. The system enables real-time end-to-end speech editing, including local modification of pitch and energy as we...
Jiayi Wang, Qifeng Han, Danyi Huang et al.· International Conference on...· 0 citations
An interpretable audio-visual synchronization framework that explicitly models speech-lip temporal alignment through feature encoding, cross-attention fusion, lag estimation, stability quantification, and visualization is proposed.
Bo-Wen Yu, Ming-Yue Huang, Yi-Shen Liu et al.· 0 citations
Software debugging is a crucial but lengthy process in software development that may require developers to decode complicated error messages and stack traces. The growing complexity of Python applications today underscores the need for more intuitive and effective debugging methods. This paper focuses on enhancing debu...
P. L., Rakshitha S, M. P et al.· Engineering & Technology· 0 citations
Environmental noise obscures the content cues needed for whisper-to-normal conversion. We propose WhisperVC-AV, which restores acoustic content features while leaving the original WhisperVC conversion module unchanged. Its context-guided restoration module combines temporal acoustic context with synchronized lip featur...
Students in MIT’s Concourse program delve deeply into the human condition, debate challenging questions, and learn to develop judgment about issues that can’t be quantified.
Jennifer Neville did not want to go into computer science—but that’s exactly where she landed. Neville discusses the starts and stops that led to her professional sweet spot and her work identifying “surprising failures” making it hard for AI to handle complexity. The post What AI gets wrong and what failure teaches us appeared first on Microsoft Research.
MIT News · Artificial Intelligence· news.mit.eduSep 30, 2026
Able to defeat top-ranked human players and more efficient than other models, the new system could help decision-makers in military maneuvers or business negotiations.
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
