Skip to content

Neither Silence nor Overlap Is Failure: Intent-Conditioned Evaluation of Turn-Taking in Full-Duplex Spoken Dialogue Models

Sep 2026 · 0 citations · 62 references
Computer Science

TL;DR

It is argued that the appropriateness of a response offset, whether delayed silence or anticipatory overlap, is conditional on the speaker's latent intent, identifiable only from that speaker's behavior.

Abstract

Benchmarks for full-duplex spoken dialogue models score turn-taking with binary fixed-window rules that reward immediate response or silence by completeness of the prior turn. We argue that the appropriateness of a response offset, whether delayed silence or anticipatory overlap, is conditional on the speaker's latent intent, identifiable only from that speaker's behavior. We introduce TACT, a benchmark of 9,728 episodes and 73.2 hours from five dyadic corpora; each episode carries dialogue history, a per-speaker memory profile, and an annotator-derived posterior over six intent classes. Scoring replaces binary windows with a strictly proper threshold-weighted continuous ranked probability score whose weights are intent-conditioned timing kernels fitted to human floor-transfer-offset distributions, proving boundedness, consistency, and binary reduction. Across eleven systems the best model reaches 0.47 against a human topline of 0.86, is nearly invariant to speaker profiles, and TACT agrees with human judgments at Spearman 0.81 versus 0.46 for binary metrics.

View source

Similar papers

#natural language process... Preprint Sep 2026

Same Words, Different Actions: Paired Turn-Taking Evaluation under Rewritten Dialogue Contexts

Real-time spoken dialogue systems must distinguish interruptions that require yielding the floor from backchannels that permit continued speaking. Existing benchmarks typically score events independently and may therefore assign high scores to systems with fixed action preferences rather than context-sensitive decision...

Shuo-Feng Zhao, Hong-Wei Cai, Wen-Ke Fan et al. · 2 citations
#artificial intelligence Preprint Oct 2026

Reference-Guided Turn-Taking Style Transfer for Full-Duplex Spoken Language Models

A single late-layer residual-stream direction is identified: a single late-layer residual-stream direction, the initiative vector, extracted from the model's own conversations without training or annotation, which transfers a speaker's turn-taking style to the model.

Chen Yuxuan, Xiao Yang, Jack Sun et al. · 0 citations
#artificial intelligence Preprint Oct 2026

XTurnix: Large-Scale Self-Supervised Turn Control through Two-State Binary Decisions

General turn-taking behavior in real-time dialogue systems requires deciding whether to keep listening or start responding while listening, and whether to continue or stop while speaking. Existing turn detectors use heterogeneous, task-specific label spaces and are often trained on limited annotations or evaluated on i...

Zhan-Xun Liu, Yi-Fan Duan, Heng-Tao Wu et al. · 0 citations

A Harness for Synthesizing Diverse Naturalistic Full-Duplex Conversations

Full-duplex dialogue systems, which listen while speaking, must distinguish a completed turn from a pause within a turn and an interruption that requests a turn from a brief acknowledgment or speech addressed to a third party. Yet existing conversational corpora provide limited control over these events and limited lab...

Matthew Z. Sun, Vinay Kothapally, Meng Yu et al. · 0 citations
#natural language process... Preprint Aug 2026

TurnBench: A Multi-Domain Benchmark for Turn-Taking Dynamics in Spoken Dialogue

TurnBench, a multi-domain benchmark that pairs a 30-hour, hand-labeled corpus of dyadic human conversation with a standardized evaluation protocol for end-of-turn and interruption detection, finds end-of-turn recall stable across types, while interruption false positives are strongly type-dependent and concentrated in...

Freeman Jiang, Ramon Sanabria, Soham Deshmukh et al. · 9 citations · ⚡4

Related blog posts

MIT News · Artificial Intelligence Nov 17, 2026

Learning the language of machine learning

By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.