Skip to content
Preprint

Multimodal Duplex Interaction Agent

Sep 2026 · 3 citations · 67 references
Engineering Computer Science

TL;DR

Gander is presented, a native multimodal duplex interaction model that builds on MiniCPM-o 4.5 and is further adapted for realtime interaction with an asynchronous agent loop and is released together with its models, code, and data to facilitate further research and development in the community.

Abstract

In this work, we present Gander, a native multimodal duplex interaction model that builds on MiniCPM-o 4.5 and is further adapted for realtime interaction with an asynchronous agent loop. In contrast to conventional turn based systems, Gander continuously processes streaming user inputs, enabling full-duplex interaction in both everyday conversations and complex workflow agent scenarios. Users can interrupt an ongoing response, while the model can proactively provide intermediate feedback or ask follow up questions. To natively support these capabilities, Gander adopts two key architectural designs: 1) a Cerebellum-Brain collaborative framework, Cerebellum is responsible for realtime interaction while the Brain handles complex reasoning and higher level agentic tasks. The two components interact continuously through tool calling and the agent orchestration runtime. 2) The Cerebellum is built upon a streaming Thinker-Talker architecture, where user inputs and model outputs are flattened into an ordered token stream at the chunk level. We evaluate Gander across conversational ability, interactive capability, understanding, and tool assisted task execution. Internal human evaluations show that Gander maintains natural and expressive spoken dialogue, while benchmark results demonstrate effective turn taking capability and encouraging results on spoken question answering and related understanding tasks. Gander also supports a range of challenging interaction settings, including background noise interference, multi-party interactions, and backchannel communication. While our current evaluation focuses on tool assisted settings, broader long horizon agent tasks and more diverse deployment conditions remain promising directions for further study. We release Gander together with its models, code, and data to facilitate further research and development in the community.

View source

Similar papers

Preprint Sep 2026

Realtime-Venus: A full-duplex interaction system with asynchronous delegation

Realtime-Venus, a proactive full-duplex interaction system with two separately trained 9B models: Realtime-Venus-Omni for audio-visual interaction and Realtime-Venus-Audio for spoken interaction, which leads the compared models on MMAU, Llama Questions, and Speech CMMLU.

Venus Team, Tsinghua University · 3 citations
Preprint Sep 2026

Qwen-Audio-Agent Technical Report

We present Qwen-Audio-Agent, a harness that combines full-duplex voice interaction with asynchronous task execution through a foreground-background architecture. A Frontend Agent manages dialogue and selects between direct tool use and delegation, while a Backend Agent carries out delegated tasks in a separate context....

Chong Deng, Yunjie Ji, Yu-Xiang Kong et al. · 1 citation · ⚡1
Preprint Sep 2026

StepAudio 3 Realtime Technical Report

Realtime spoken interaction demands deep reasoning, prompt responses, and fluid turn-taking. We present StepAudio 3 Realtime, an audio-language foundation model organized around a continuous listen-converse-think-act loop. Deep Perception captures rich acoustic cues to interpret user intent, while Seamless Duplex model...

Bin Lin, Bo Zhao, Bo-Yang Zhang et al. · 4 citations
#artificial intelligence Preprint Sep 2026

A Voice-Interactive Multi-Agent System for Smart Operating Rooms: Architecture Design and Key Technologies

This paper presents SurgicalRoomAgent, a voice-interactive multi-agent system for smart operating rooms based on large language models (LLMs), which achieves natural language understanding, device control, intraoperative recording, and surgical report generation through a layered architecture comprising a voice interac...

Tian Zhou · 0 citations

Delegating or Doing? Understanding User Behavior in Hybrid Human-Agent Interfaces

The findings suggest that the primary benefit of human--agent interfaces may be reducing interaction effort rather than improving speed, and that delegation reflects who the user is more than what the task demands.

Gavin Raine Dizon, Tyrone Justin Sta Maria, J. A. Deja et al. · 0 citations
#artificial intelligence Preprint Sep 2026

Qwen-Audio-3.1-Realtime: Towards Reliable Agentic Voice Interaction

Real-time voice assistants must reason over evolving requests, execute actions, and follow conversational rules. Qwen-Audio-3.1-Realtime brings these requirements together through Think, Act, and Speak and Coordinate. Think combines Core-Cocktail supervised fine-tuning with Multimodality and Multi-Teacher On-Policy Dis...

Lu-Jia Bao, Qian Chen, Luyao Cheng et al. · 1 citation

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.