Closing the Productive Half of Cross-Language Conversation: An Anticipatory Multilingual Conversation Assistance System for Wearable Devices

Abstract

Cross-language conversation support technology has advanced almost exclusively on the receptive side: translation devices and subtitle glasses help a user understand an interlocutor, but leave the user unaided at the moment of formulating a reply in the interlocutor's language. This paper presents a wearable conversation assistance system that closes this productive half of the loop. Worn by only one party to the conversation, the system (1) captures the interlocutor's speech and projects real-time translated subtitles onto a near-eye display; (2) anticipatorily generates, in parallel with the interlocutor's ongoing turn and drawing on streaming partial speech-recognition hypotheses, dialogue history, and a persistent user profile, a small ranked set of candidate responses in the interlocutor's language, presented with spoken pronunciation guidance delivered by in-ear text-to-speech; and (3) provides a free-expression fallback in which the wearer speaks once in their native language and reads aloud the projected translation. The central mechanism is latency concealment through anticipation: because candidate generation overlaps the interlocutor's turn, large-language-model inference time is absorbed within the natural duration of speech and the turn-taking gap, so candidates are ready before the wearer's turn begins. The paper describes the three-tier architecture (smart glasses, paired mobile device, cloud services), the three-step conversation loop, the functional design of candidate ranking and selection — including silent gesture-based selection that keeps the interaction invisible to the interlocutor — and the user-profile and privacy model. The system is described as disclosed in U.S. Provisional Patent Application No. 64/170,246; performance figures cited are the design targets stated in that disclosure, not production measurements.