Turn-transition entropy is introduced, a self-supervised target computed from the sequence of speaker transitions, which quantifies the predictability of interaction patterns and can be learned as a single-task objective.
Abstract
We propose a multi-task learning approach for multi-party dialogue intent recognition that leverages an auxiliary task that models turn-taking dynamics. Specifically, we introduce turn-transition entropy, a self-supervised target computed from the sequence of speaker transitions, which quantifies the predictability of interaction patterns. Experiments on multiple pre-trained models demonstrate that incorporating this auxiliary task improves intent recognition performance, outperforming existing approaches which ignore multi-party interaction dynamics. We find that our proposed continuous target can be learned as a single-task objective, suggesting that it is an actual signal carrying useful information.
Speech act information is used as an auxiliary learning signal alongside textual semantics in pragmatic representations of conversations to reduce lexical noise and improve generalizability in conversational derailment forecasting.
A. Yuan, C. de Kock, Christopher Leckie· 0 citations
This study forms continuous recognition of the Group Emotion at a one-second resolution, introduces the Mixed state, which captures the emotional divergence among participants in the group, and proposes a multimodal temporal framework that integrates audio and video information using a sliding-window context.
Soma Iwata, K. Inoue, Mu-Yun Wu et al.· Companion Publication of the...· 0 citations
TurnBench, a multi-domain benchmark that pairs a 30-hour, hand-labeled corpus of dyadic human conversation with a standardized evaluation protocol for end-of-turn and interruption detection, finds end-of-turn recall stable across types, while interruption false positives are strongly type-dependent and concentrated in...
Freeman Jiang, Ramon Sanabria, Soham Deshmukh et al.· 9 citations· ⚡4
A multimodal Turkish conversational dataset of unscripted dyadic interactions, comprising synchronized front-facing video, per-speaker audio channels that allow overlapping speech to be attributed to individual speakers, and time-aligned transcriptions is introduced.
Ahmet Tugrul Bayrak, Fatma Nur Korkmaz, Bekir Berker Türker et al.· 0 citations
SpeakerMem-R1 is proposed: its dual-track memory stores speaker-labeled verbatim messages and derived states organized into person-level and group-level views, then combines evidence from both tracks by entity, event, and time at query time, and ablations show that the verbatim and structured tracks, as well as person-...
Hao-Bo Zheng, Tan Tang, Yan Chen et al.· 0 citations
This work proposes a new approach that quantifies the effectiveness of each interaction by the reduction in the assistant's uncertainty, measured via entropy over recommendations, to fine-tune the LLM, enabling strategic interaction generation.
Cedar Site Bai, Zhen-Yu Liao, Duan Li et al.· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduAug 27, 2026
A new machine-learning framework aims to improve the success rate of computational protein design while moving away from results that reproduce sequences found in nature.
