Breathing Signal Prediction from Speech: Toward Reproducible and Comparable Models

Abstract

Breathing plays a fundamental role in speech production and perception, providing important cues about the speaker’s physical and psychological state. Recently, the use of breathing as a cognitive, affective, and physical marker has gained momentum, especially in the fields of speech synthesis, conversational robots and agents. However, the challenges associated with breathing annotation and the lack of annotated datasets for these purposes hinders further progress. Automatic breathing signal prediction could bridge this gap. Unfortunately, state-of-the-art (SOTA) research in this area remains difficult to replicate due to the lack of openly available code or datasets. This work makes four key contributions: (1) we implement previously proposed SOTA methods for speech-based breathing prediction, explicitly stating any assumptions made; (2) we introduce extensions to improve their performance; (3) we propose two novel WavLM-based models; and (4) we evaluate the impact of speaker-specific conditioning via feature-wise linear modulation (FiLM), using speaker embeddings (eGeMAPS and WeSpeaker) to modulate intermediate representations. To ensure fair and reproducible comparisons, we evaluate all models using a subset of the publicly available UCL-SBM dataset that was used in Interspeech 2020 Computational Paralinguistics Challenge (ComParE2020), adhering to the challenge protocol. Our proposed WavLM-based model advances SOTA for this subset, achieving a PCC of 0.806.