Jul 2026· Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing· Vol abs/2607.21550· 4 citations· 48 references
Computer Science
TL;DR
This work proposes X$^3-OPD, a cross-modal on-policy distillation framework that transfers reasoning capabilities from a powerful text teacher to an audio-language student, and extends cross-modal distillation beyond textually recoverable content to reasoning grounded in non-linguistic events, prosody, and conversational context.
Abstract
While large audio-language models have achieved remarkable progress in auditory perception, they still lag behind text-based large language models in deep logical reasoning, primarily due to the scarcity of high-quality audio reasoning data. To bridge this gap, we propose X$^3$-OPD, a cross-modal on-policy distillation framework that transfers reasoning capabilities from a powerful text teacher to an audio-language student. During training, the student generates reasoning trajectories conditioned on its own acoustic perception, while the teacher provides token-level guidance using matched textual inputs and verified answers. We further construct a three-tier symmetric corpus covering textual reasoning rendered into speech, audio-event reasoning grounded in complex acoustic scenes, and spoken-dialogue reasoning involving paralinguistic cues. This design extends cross-modal distillation beyond textually recoverable content to reasoning grounded in non-linguistic events, prosody, and conversational context. Experiments on MMSU, MMAU, BIG Bench Audio, and MMAR demonstrate that X$^3$-OPD substantially improves audio-grounded reasoning and chain-of-thought quality while largely preserving the model's existing capabilities under domain shift.
Experimental results show that SKAD consistently improves the average accuracy of both models across all benchmarks, and ablations show that direct contrastive decoding without the acoustic evidence sketch provides limited or negative gains, confirming the importance of constructing a reliable contrastive reference bef...
Jun-Yu Wang, Pei-Yuan Jiang, Si-Yuan Zhang et al.· 0 citations
Perception-Grounded TTRL (PG-TTRL) is proposed, which aligns label-free test-time optimisation with perceptually grounded reasoning, encouraging the model to structure its reasoning more strongly on the audio input, showing the value of perceptual-grounding optimisation for test-time audio reasoning.
Jia-Heng Dong, Xiao-Feng Yu, Jean Honorio et al.· arXiv.org· 0 citations
Experiments with silenced and replacement audio suggest that RT-OPD strengthens the student's reliance on acoustic evidence, and the 3B model achieves 72.72% accuracy on MMAU, the highest among the compared 3B models and competitive with several 7B and 8B models.
Omni-modal large language models deployed in real-world environments encounter external noise that can interfere with their perception and understanding of multimodal inputs. We study their robustness in audio-visual understanding, focusing on question answering under environmental noise and competing speech. The chall...
Xing-Ming Shui, Da-Peng Chen, Bo-Wei Liu et al.· arXiv.org· 0 citations
AURAL is introduced, which models a distribution over multiple plausible reasoning continuations in latent space and jointly predicts chunks of future states to reduce sequential forward passes and reasoning latency and shows that harder questions elicit more latent reasoning steps.
Yu-Xiang Wang, Kun-Yu Feng, Yuan-Cheng Wang et al.· 4 citations
JELAR, a Joint-Embedding Predictive Architecture (JEPA)-based latent reasoning framework that conditions latent reasoning supervision on acoustic representations learned from raw waveforms, demonstrates the effectiveness of JEPA-conditioned latent reasoning as an alternative to explicit textual CoT supervision.
Donghang Wu, Haoyang Zhang, Yizhou Peng et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.