ResGDS-HLF: Residual Gumbel Dimension Selection with Hierarchical Layer Fusion for Stuttering Event Classification

Abstract

Current systems for stuttering detection are limited by task-agnostic feature compression and flat layer-fusion strategies. Traditional methods rely on Principal Component Analysis (PCA) to reduce self-supervised representations, which often suppresses subtle disfluency patterns in favor of speakeridentity cues. We propose ResGDS-HLF, a framework that replaces static compression with Residual Gumbel Dimension Selection to recover discriminative dimensions directly optimized for stuttering classification. By partitioning the transformer backbone into a functional hierarchy, i.e. acoustic, phonetic, and lexical, the model preserves specialized information that is otherwise diluted by weighted averaging. Evaluated on the SEP-28K corpus, ResGDS-HLF achieves a Macro-F1 of 0.61, representing an absolute gain of 0.14 over single-layer PCA baselines. Zero-shot transfer to FluencyBank confirms that taskdriven Gumbel selection captures robust, generalizable disfluency signatures across diverse acoustic environments.