Skip to content

Voice of Reason: Reinforcement Learning for Spoken Math

Sep 2026 · arXiv.org · Vol abs/2609.18677 · 0 citations · 36 references
Computer Science

TL;DR

This work first adapts the model to the domain using supervised fine-tuning on synthesized spoken question-answering data and shows that RL improves the accuracy on GSM8K beyond levels previously achieved for speech models only with supplementary reasoning traces, and establishes a new state-of-the-art for mathematical spoken abilities with speech-native models.

Abstract

Speech language models enable richer spoken interactions between humans and machines than cascaded systems, allowing access to paralinguistic information and lower latency. However, their accuracy on mathematical reasoning benchmarks has lagged behind those of text models. Reinforcement learning (RL) with verifiable rewards has been instrumental in extending text models'capabilities for solving complex problems and limiting hallucinations. In this work, we explore applying RL to the GLM-4-Voice speech model (Zeng et al., 2024) to bridge the gap between textual and spoken mathematical problem solving. We first adapt the model to the domain using supervised fine-tuning on synthesized spoken question-answering data. We then show that, even without extra reasoning tokens, RL improves the accuracy on GSM8K beyond levels previously achieved for speech models only with supplementary reasoning traces. When combined with existing streaming reasoning techniques, we show further gains to 74.8% free-form accuracy. This establishes a new state-of-the-art for mathematical spoken abilities with speech-native models.

View source

Similar papers

Preprint Sep 2026

Listen, Critique, and Refine: RL-Based Self-Refinement for Instruction-Following Speech Synthesis

This work training a LALM with reinforcement learning to reason over its own speech output generates a draft speech as a form of audio-token reasoning, critiques its own generation by reflecting on the acoustic realization in text, and produces a refined version conditioned on both the first-pass speech and the critiqu...

Chee-En Yu, Yi-Cheng Lin, Sung-Feng Huang et al. · 1 citation
#machine learning Preprint Oct 2026

AURAL: Adaptive Latent Reasoning with Joint Chunk for Speech Language Models

AURAL is introduced, which models a distribution over multiple plausible reasoning continuations in latent space and jointly predicts chunks of future states to reduce sequential forward passes and reasoning latency and shows that harder questions elicit more latent reasoning steps.

Yu-Xiang Wang, Kun-Yu Feng, Yuan-Cheng Wang et al. · 4 citations
#natural language process... Preprint Sep 2026

Spoken Language Models that Think Aloud

While Chain-of-Thought (CoT) reasoning has improved the capability of language models, directly applying it to Spoken Language Models (SLMs) may introduce long silent intervals under the serial"think-then-speak"paradigm, disrupting real-time spoken interaction. To address this issue, we propose an asynchronous think-al...

Junyi Ao, Kainan Peng, Mingbo Ma et al. · 0 citations
Preprint Aug 2026

SpeechGym: An Audio-Native Gym for Training Voice Agents via Reinforcement Learning

SpeechGym is presented, an audio-native agentic environment in which two omni-modal models converse in native audio, with no external ASR or TTS and no API boundary, over the unmodified tasks, tools and success check of an established text agentic benchmark, so that the interaction modality is the only variable and the...

Jia-Jun Fan, Jing-Yuan Li, Prashanth Gurunath Shivakumar et al. · 1 citation · ⚡1

Listen Then Reason: Perception-Grounded Test-Time Reinforcement Learning for Large Audio-Language Models

Perception-Grounded TTRL (PG-TTRL) is proposed, which aligns label-free test-time optimisation with perceptually grounded reasoning, encouraging the model to structure its reasoning more strongly on the audio input, showing the value of perceptual-grounding optimisation for test-time audio reasoning.

Jia-Heng Dong, Xiao-Feng Yu, Jean Honorio et al. · 0 citations
#machine learning Preprint Oct 2026

AdaLoop: Adaptive-Depth Latent Reasoning for Audio Language Models

Large audio language models answer questions about speech, sound, and music, yet their accuracy drops sharply on tasks that need fine-grained acoustic analysis. Judging which of two speakers has the higher pitch demands iterative signal-level reasoning that a content question does not. Current models spend the same com...

Seung-Woo Lee, Bo-Wen Qi · 0 citations

Related blog posts

MIT News · Artificial Intelligence Nov 17, 2026

Learning the language of machine learning

By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.