Multiparty Bench is introduced, the first benchmark specifically designed to objectively evaluate conversational speech systems as active participants within multi-party contexts and assesses agent behavior along two primary dimensions: turn-taking awareness and response appropriateness.
Yi-Jen Shih, Shih-Yun-Shan Kuan, Guan-Ting Lin et al.· 2 citations· ⚡1
Tri-PvP, an 8,000-sample tri-modal conflict benchmark crossing vision, audio, and text, is introduced, revealing a systematic asymmetry in evidence-form bias: models exhibit a stronger bias toward perceptual signal in vision but propositional in audio.
Yen-Ting Piao, Shu-Yun Chen, Chin-Hui Chu et al.· 0 citations
TurnBench, a multi-domain benchmark that pairs a 30-hour, hand-labeled corpus of dyadic human conversation with a standardized evaluation protocol for end-of-turn and interruption detection, finds end-of-turn recall stable across types, while interruption false positives are strongly type-dependent and concentrated in...
Freeman Jiang, Ramon Sanabria, Soham Deshmukh et al.· 9 citations· ⚡4
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.