General turn-taking behavior in real-time dialogue systems requires deciding whether to keep listening or start responding while listening, and whether to continue or stop while speaking. Existing turn detectors use heterogeneous, task-specific label spaces and are often trained on limited annotations or evaluated on i...
Zhan-Xun Liu, Yi-Fan Duan, Heng-Tao Wu et al.· 0 citations
OmniVChat-Studio, a multi-agent data engine for synthesizing single- and multi-turn audio-visual dialogues and OmniVChat-RL, a reinforcement learning reward design that jointly targets reply correctness, efficiency, and style in OmniVChat.
Haolin He, Yun-Fei Chu, Qi Chen et al.· 0 citations
TinyAudio is introduced, a compact flow-matching-based TTA model for low-resource deployment that achieves competitive generation quality on AudioCaps and TTA-Bench and demonstrates a practical quality-footprint trade-off for low-resource TTA deployment.
Jun-Xi Liu, Xi-Quan Li, Wen-Hao Guan et al.· 0 citations
This work proposes Agentic-GER, an LLM-based agent for terminology correction in long-form speech, which uses global context from the full transcript to identify suspicious terms and resolve ambiguous hypotheses.
Yan-Qiao Zhu, Wu-Peng Wang, Zhifu Gao et al.· 0 citations
Omni Demand Understanding is established as a distinct multimodal contextual inference problem: given an interaction stream, a model must detect whether a user demand is present and infer intent from multimodal and conversational context, and ODU is evaluated along five dimensions, covering both single-turn and multi-t...
Qi Chen, Yun-Fei Chu, Haolin He et al.· 1 citation· ⚡1
We define OmniVChat (Omni Video Chat) as the task of native audio-visual dialogue between a user and an omni model. In OmniVChat, omni models directly and simultaneously receive audio and video from a user and return text. The user's query is embedded in the audio and video, without a separate text question, external c...
Haolin He, Yunfei Chu, Qi Chen et al.· 0 citations
Zero-shot text-to-speech (TTS) can clone a speaker's voice from a short audio prompt, yet most TTS systems still require the audio prompt transcript during inference. This dependency prevents cross-lingual voice cloning when the audio prompt transcript is unavailable, particularly for unseen languages. Cross-Lingual F5...
Qingyun Liu, Rixi Xu, Yu-Shen Chen et al.· 0 citations
Extensive experiments demonstrate that AnyAudio-Judge not only significantly enhances zero-shot alignment detection compared to state-of-the-art baselines, but also provides precise and interpretable reward signals that substantially improve instruction alignment in downstream reinforcement learning for audio generatio...
Hai-Tao Li, Tian Tan, Yuguang Yang et al.· arXiv.org· 3 citations
Recent advances in large language models (LLMs) and multimodal LLMs (MLLMs) have created new opportunities for wearable speech interfaces, with smart glasses providing an egocentric platform for continuous audio sensing and assistance. However, speech recognition and understanding in this setting remain challenging bec...
Dehui Gao, Zhixian Zhao, Zhennan Lin et al.· 1 citation
VibeVoice-ASR-Streaming is one of the first LLM-based end-to-end approaches to streaming speaker-attributed ASR, allowing the model to produce''who said what''as speech arrives, without a separate diarization stage.
Yu-Jie Tu, Zhiliang Peng, Jianwei Yu et al.· 0 citations
AuK, an open-source foundational model that unifies speech generation and editing through a common interface of natural-language instructions and audio context, is introduced and leading performance on zero-shot and instruction-controlled speech generation and general instruction-guided editing is demonstrated.
Zi-Yang Ma, Zhi-Kang Niu, Wen-Ming Tu et al.· 3 citations· ⚡1
X-Translator is presented, a low-cost modular cascaded S2ST system that combines streaming ASR, machine translation, and prompt-conditioned TTS through a session-level runtime controller that provides an open platform for understanding the practical trade-offs of deployment-oriented S2ST.
Yuxiang Zhao, Yichi Zhang, Yanjie An et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.