Skip to content

Thinking in Depth, Speaking Directly: Recurrent Latent Reasoning for Paralinguistically Grounded Spoken Dialogue

Sep 2026 · arXiv.org · Vol abs/2609.37818 · 2 citations · 22 references
Computer Science

TL;DR

LoopSLM builds on looped Transformers for latent reasoning, reusing a decoder block to refine hidden states with acoustic grounding at every pass, and further narrows the perception-reasoning gap by separating learning to reason from learning to respond, enabling direct inference without CoT.

Abstract

Empathetic spoken dialogue requires models to use both what is said and how it is said to decide how to respond. Explicit CoT can improve paralinguistic perception and make acoustic cues more explicit in replies, yet does not ensure their effective use in response planning. We call this mismatch the perception-reasoning gap. In addition, CoT may not fully capture acoustic cues in words, and generating it adds inference latency. To address these limitations, we introduce LoopSLM, which builds on looped Transformers for latent reasoning, reusing a decoder block to refine hidden states with acoustic grounding at every pass. Its two-stage training further narrows the perception-reasoning gap by separating learning to reason from learning to respond, enabling direct inference without CoT. On EchoMind, LoopSLM improves paralinguistic understanding, reasoning, and reply quality over Qwen2.5-Omni-7B. Against the CoT-SFT baseline, LoopSLM gains over 20 points in reasoning accuracy while generating 64.5% fewer tokens at half the latency. It also outperforms Qwen3-Omni-Thinking on most empathetic reply metrics with 34x lower latency. Despite training only on dialogue data, LoopSLM improves accuracy on general audio benchmarks.

View source

Similar papers

Preprint Oct 2026

EchoChat: Structured Cognitive Reasoning in Empathetic Spoken Dialogue

Empathetic spoken dialogue is a sophisticated cognitive process that requires not only recognizing emotions but also inferring a user's latent mental states to provide appropriate support. However, current SpeechLLMs often treat empathy as a direct input-to-response mapping, leading to"superficially warm"but emotionall...

Ding-Dong Wang, Shu-Jie Liu, Ya-Yue Deng et al. · 0 citations
#machine learning Preprint Oct 2026

AURAL: Adaptive Latent Reasoning with Joint Chunk for Speech Language Models

AURAL is introduced, which models a distribution over multiple plausible reasoning continuations in latent space and jointly predicts chunks of future states to reduce sequential forward passes and reasoning latency and shows that harder questions elicit more latent reasoning steps.

Yu-Xiang Wang, Kun-Yu Feng, Yuan-Cheng Wang et al. · 4 citations
#natural language process... Preprint Sep 2026

Spoken Language Models that Think Aloud

While Chain-of-Thought (CoT) reasoning has improved the capability of language models, directly applying it to Spoken Language Models (SLMs) may introduce long silent intervals under the serial"think-then-speak"paradigm, disrupting real-time spoken interaction. To address this issue, we propose an asynchronous think-al...

Junyi Ao, Kainan Peng, Mingbo Ma et al. · 0 citations
#artificial intelligence Preprint Sep 2026

RetroThinker: Enabling Retrospective Thinking in Speech LLMs

Speech large language models (SpeechLLMs) offer reduced latency and retain paralinguistic nuances that are typically lost in cascaded automatic speech recognition (ASR) and text-based LM architectures. However, they continue to lag behind text-only LLMs on complex reasoning tasks, while real-time spoken interaction imp...

Yi-Jen Shih, Pu-Yuan Peng, Abdel-rahman Mohamed et al. · 1 citation

Voice of Reason: Reinforcement Learning for Spoken Math

This work first adapts the model to the domain using supervised fine-tuning on synthesized spoken question-answering data and shows that RL improves the accuracy on GSM8K beyond levels previously achieved for speech models only with supplementary reasoning traces, and establishes a new state-of-the-art for mathematical...

Timothée Weisselberger, Edouard Graves, Alexandre Défossez · 0 citations
Preprint Aug 2026

When Text Misleads: Inconsistent-Aware Reasoning for Audio-Grounded Dialogue

A scalable framework that identifies text-biased surface interpretations and converts disagreement regions into conflict QA examples and develops an agentic-style reasoning framework that converts speech into an Audio Twin, a text-readable representation of localized acoustic cues that exposes acoustic evidence to the...

Yen-Ju Lu, Yu-Zhe Wang, Yao-Han Guan et al. · 1 citation

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.