Enabling Streaming User Transcription in Full-Duplex Speech-to-Speech Models
This work proposes an efficient method to add streaming ASR capabilities to an existing duplex S2S model by introducing a lightweight ASR head in parallel to the agent text head, enabling real-time user transcription while preserving full-duplex conversational capabilities including turn-taking and barge-in handling.
Ke Hu, Nourchene Ferchichi, Edresson Casanova et al.
· 1 citation