XPLAIN: A Proactive Scaffold Across Speech Processing Stages---Supporting Non-Native Speakers in Real-Time AI-Mediated Turn-Taking

Abstract

Predictive processing during rapid turn-taking imposes immense cognitive demands on non-native speakers (NNS), frequently inducing communicative anxiety. To mitigate these barriers, we designed and evaluated XPLAIN, a proof-of-concept, proactive Conversational User Interface (CUI) that maps just-in-time visual scaffolds to distinct stages of speech processing: lexical clarifications for comprehension, contextual suggestions for production planning, and situational summaries for alignment. We evaluated this system through a lab-based Wizard-of-Oz study with 38 NNS, gathering survey and interview data. Our findings demonstrate that stage-specific scaffolds effectively bridged knowledge gaps and boosted confidence. Participants leveraged scaffolds as a private, "backstage" repair channel to preserve social face. However, we also identified attention trade-offs: while lightweight clarifications afforded low-friction grounding, more cognitively demanding suggestions introduced evaluative bottlenecks that might diminish autonomy and authenticity. Moreover, participants retrospectively described tolerating momentary speech output hesitations as an acceptable cost for a more substantive contribution, an inversion of the common CUI evaluation criterion holding that minimizing disfluency indicates better support. We further discussed design implications derived from these observations.