Jun 2026· arXiv.org· Vol abs/2606.30700· 0 citations· 28 references
Computer ScienceEngineering
TL;DR
BEST-RQ-2 is presented, an evolution of BEST-RQ that retains frozen randomprojection-based discrete targets while introducing a two-step contextualize-then-predict pretraining scheme, and consistently outperforms one-stage baselines in overall transfer while keeping inference compute unchanged.
Abstract
Self-supervised learning enables audio representations that transfer across domains and tasks. We present BEST-RQ-2, an evolution of BEST-RQ that retains frozen randomprojection-based discrete targets while introducing a two-step contextualize-then-predict pretraining scheme. A ViT context encoder processes only the unmasked spectrogram regions, and a lightweight predictor infers targets for the masked regions; the predictor is discarded after pretraining. Replacing the original Conformer encoder with a ViT shifts performance across domains, slightly reducing speech performance while improving music and environmental sounds, with comparable average scores. The main improvement comes from decomposing masked prediction into separate contextualization and prediction stages. On the X-ARES and XARES-LLM benchmarks, BEST-RQ-2 consistently outperforms one-stage baselines in overall transfer while keeping inference compute unchanged. Code and model checkpoints are publicly available.
Text-to-speech systems are improving fast, but measuring how natural they sound still requires expensive human listening tests. Existing automatic methods struggle to generalise well across different datasets. We present QMOS, a MOS prediction framework that extracts hierarchical speech quality features from a frozen Qwen2-Audio large audio language model and combines them with layer-weighted WavLM SSL representations through a learned cross-attention fusion. This design captures both high-level semantic naturalness and lowlevel acoustic distortions in a unified model. On two standard benchmarks, SOMOS and BVCC, QMOS achieves competitive system-level SRCC of 0.923 and 0.921 on SOMOS and BVCC, respectively, while using no system-ID conditioning or listener embeddings. Cross-domain evaluation yields a competitive SRCC of 0.702, suggesting the learned representations generalise across acoustic domains.
Sri Ravi Sastry Kolluru, Charan Devarakonda, S. Radhe et al.· International Conference on...· 0 citations
NAPE (Next-Audio-Patch-Embedding prediction), a self-supervised framework in which a causal Transformer predicts each next patch embedding of a log-mel spectrogram from the previous ones, using causal masking and stop-gradient as its sole training signal is introduced.
Umberto Cappellazzo, Xubo Liu, Stavros Petridis et al.· 0 citations
A multi-layer feature fusion (MLF) adapter that aggregates information from all encoder layers before projecting them into the language model is proposed and shows that MICL does not emerge naturally in ALLMs, but can be effectively acquired through targeted contextual biasing training.
Xiaoyu Yang, Xuenan Xu, Wenyi Yu et al.· 0 citations
It is suggested that many reported downstream gains reflect instance and seed dependent elicitation match, rather than universally improving the attainable performance ceiling, in self-supervised fine-tuning.
Wangjin Zhou, Yizhou Zhang, Yichi Wang et al.· 0 citations
The results show that a small, carefully chosen activation set can switch on test-time scaling inside a frozen ASR model without corrupting the model itself, converting fixed per-utterance compute into input-dependent compute where it is most needed.
Ho-Lam Chung, Yiming Chen, Dau-Cheng Lyu et al.· 0 citations
An AS-Split Conformer–Mamba framework that decouples local and global modeling into two explicit stages and achieves consistent improvements over strong baselines is proposed.
Lulu Qin, Xuan Fu, Mingchen Sun et al.· Electronics· 0 citations