Targeted controls show that tool-invocation scores alone do not establish event grounding and that reminder coverage depends on response history and frequency, and motivate developing and evaluating streaming assistants for timely action, initiative under general user needs, and coverage of entire response sequences.
Yin-Song Yan, Ling Wang, Yuxuan Wang et al.· 0 citations
Screen-recorded web editing requests contain weak deictic expressions such as ``this''and ``there,''whose referents depend on speech, cursor trajectories, page state, and edit history. Such requests require intent recovery beyond the explicit specifications assumed by many existing web-editing benchmarks. We introduce...
Multimodal large language models have become the dominant paradigm for visual understanding, but incur substantial costs by encoding inputs into dense, fixed-size patch tokens. However, visual information is unevenly distributed: some regions require fine-grained detail, while others admit compact representations. Down...
Yuan Feng, Qi-Ze Yang, Rui-Zhe Chen et al.· 1 citation
OmniVChat-Studio, a multi-agent data engine for synthesizing single- and multi-turn audio-visual dialogues and OmniVChat-RL, a reinforcement learning reward design that jointly targets reply correctness, efficiency, and style in OmniVChat.
Haolin He, Yun-Fei Chu, Qi Chen et al.· 0 citations
This work proposes VoxPolyMem, an interaction-aware multimodal memory framework combining incremental speaker identification with a memory hierarchy comprising interaction memory, fact memory, and participant profiles, and introduces Evidence-Gain GRPO (EG-GRPO), which uses round-wise credit assignment to encourage com...
Wen-Xu Jia, Xi-Ze Cheng, Zi-Han Zhang et al.· 0 citations
OmniEchoBench, a unified benchmark for spatial audio-visual perception and audio-vision-language navigation, and a spatially aware omni-modal model, which introduces an FOA spatial encoder alongside a pretrained semantic audio pathway.
MuLA-Bench exposes conditional failure patterns that a single long-context score does not capture, and evaluates ten audio-language models and conducts pooled diagnostics on a fixed eight-model cohort.
Ze-Yu Yang, Xin-Yu Zhang, Zi-Bo Bi et al.· arXiv.org· 2 citations
Long-context understanding remains a fundamental challenge for large language models, as excessively long inputs often lead models to forget salient information. This issue is even more pronounced in the speech domain, where audio, as a low-compression modality, requires substantially more embeddings than text to prese...
We define OmniVChat (Omni Video Chat) as the task of native audio-visual dialogue between a user and an omni model. In OmniVChat, omni models directly and simultaneously receive audio and video from a user and return text. The user's query is embedded in the audio and video, without a separate text question, external c...
Haolin He, Yunfei Chu, Qi Chen et al.· 0 citations
This work proposes X$^3-OPD, a cross-modal on-policy distillation framework that transfers reasoning capabilities from a powerful text teacher to an audio-language student, and extends cross-modal distillation beyond textually recoverable content to reasoning grounded in non-linguistic events, prosody, and conversation...
Dong-Jie Fu, Di Cao, Zi-Han Zhang et al.· Proceedings of the 2026 Conf...· 4 citations
DiaScriber is proposed, an end-to-end multi-speaker diarization and transcription model built on a speech large language model that achieves superior performance over comparison methods across extensive multi-speaker scenario test sets and demonstrates outstanding generalization ability in unseen multi-speaker scenario...
Bing-Shen Mu, Xian Shi, Xiong Wang et al.· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.