Human face-to-face conversation requires coordinating speech, gestures and expressions across both speaking and listening. We present ELLSA 2, an end-to-end full-duplex dialogue model that jointly optimizes speech interaction and streaming gesture-expression generation. An autoregressive, chunk-wise synthesizer combine...
Si-Yin Wang, Wen-Yi Yu, T. Chiba et al.· 0 citations
Large language model (LLM) computer-use agents are typically evaluated with clean written instructions, despite speech being an increasingly popular interface for interacting with such systems. Speech input introduces an additional failure point: transcription errors can alter task-critical entities, constraints, or ta...
T. Chiba, Guang-Zhi Sun, Zhe-Qi Yuan et al.· arXiv.org· 0 citations
SALMONN-duo is proposed, an adaptive dual-system voice agent inspired by dual-process theories of cognition that separates real-time interaction from deliberative computation by pairing an always-on, fast-thinking full-duplex speech LLM with a powerful asynchronous slow-thinking LLM agent.
Wen-Yi Yu, Si-Yin Wang, T. Chiba et al.· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.