Skip to content

RT-SEMamba: Real-Time Speech Enhancement Mamba via Progressive Knowledge Distillation

Aug 2026 · 1 citation · 48 references
Computer Science

TL;DR

This work presents RT-SEMamba, a fully causal speech enhancement (SE) model built upon causal time-frequency Mamba blocks, and introduces a progressive knowledge distillation (KD) strategy that compresses an 8-layer teacher into a shallow 1-layer student by jointly distilling complex spectral outputs and intermediate representations.

Abstract

We present RT-SEMamba, a fully causal speech enhancement (SE) model built upon causal time-frequency Mamba blocks. Unlike Transformer-based architectures that rely on a growing key-value cache, Mamba propagates a fixed-size recurrent state per layer, enabling memory- and bandwidth-efficient long-form inference. We further introduce a progressive knowledge distillation (KD) strategy that compresses an 8-layer teacher into a shallow 1-layer student by jointly distilling complex spectral outputs and intermediate representations. On Voicebank-DEMAND, the 8-layer RT-SEMamba achieves 3.32 PESQ with a 25 ms algorithmic latency constraint, and the distilled 1-layer student improves over a naive 1-layer baseline from 3.06 to 3.18 PESQ while preserving the same steady-state RTF, delivering a 2.64x speedup over the teacher. These results demonstrate that state-space models with progressive KD provide a competitive quality-latency trade-off for real-time SE.

View source

Similar papers

#machine learning Preprint Sep 2026

Pre-Whitening and BCJR Posterior Distillation for Bi-LSTM Detection in Faster-than-Nyquist Signaling

This work asks whether processing nested ISI windows in separate recurrent branches improves the bit error rate (BER) of a Bi-LSTM, and pre-whiten the input, restoring the conditional independence that colored matched-filter noise violates, and distill the BCJR soft posterior into the network.

Nurettin Şafak, Osman Tokluoğlu, E. Çavuş · 0 citations
#natural language process... Open access Sep 2026

SEAR: Segment-Evidence-Aware Routing for Weak-to-Strong Multilingual Speech MCQ

This paper describes the system for Task~2 of the second Multilingual Conversational Speech Language Model (MLC-SLM) Challenge, which adapt Qwen3-Omni-30B-A3B-Instruct with a segment-evidence-aware data and post-training pipeline and obtains 90.92% accuracy on the final official evaluation set.

H. Le, L. Nguyen, Minh Tri Dao · 1 citation
#small language model Preprint Sep 2026

Compressing Streaming Neural Audio Encoders via Latent-Space Distillation

This work studies how to compress a tokenizer by distillation, taking as the supervision target neither the discrete token nor the output distribution but the pre-quantizer latent the model actually consumes - the last representation the two token interfaces share.

Prasanth Yadla, Mohammad Samragh, Dongseong Hwang et al. · 0 citations
#artificial intelligence Preprint Sep 2026

X-AuT: Progressive Audio-Encoder Compression for Speech LLMs with Cross-Scale Distillation

X-AuT, a progressive framework that selects layer combinations through short behavioral probes and restores the pruned model through representation alignment, cross-scale distillation, scheduled student-policy supervision, and LoRA finetuning, is introduced.

Hao-Jun Zhang, Yi Zou, Min Chen et al. · 0 citations

Learning as Deepfakes Evolve: RF-Prompt for Continual Audio Deepfake Detection

The proposed Real-Anchored Mechanism-Incremental (RAMI) protocol reflects the practical setting in which available real speech provides a recurring mixed-domain reference while new deepfake mechanisms arrive incrementally and RF-Prompt, an asymmetric continual prompt-learning method that preserves reusable real-speech...

Yuan-Kun Xie, Xiao-Xuan Guo, Xiao-Peng Wang et al. · 0 citations

Related blog posts

MIT News · Artificial Intelligence Nov 17, 2026

Learning the language of machine learning

By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.