A rubric-based audio-grounded evaluation framework that verifies event realization, acoustic attributes, and speech content through fine-grained semantic criteria, and human validation demonstrates that the benchmark achieves stronger alignment with human semantic judgments than conventional global similarity metrics.
Abstract
Text-to-audio (TTA) generation has recently achieved remarkable progress in synthesizing realistic audio from natural language descriptions. However, determining whether generated audio faithfully satisfies complex textual instructions remains challenging. Existing benchmarks mainly rely on global similarity metrics, providing limited insight into fine-grained semantic failures. To address this limitation, we introduce \textbf{AudioScape-TTA}, a structured and complexity-aware benchmark for fine-grained TTA evaluation. AudioScape-TTA represents realistic soundscapes through modality-aware semantic structures and characterizes generation complexity using event density and structural complexity. Based on these annotations, we propose a rubric-based audio-grounded evaluation framework that verifies event realization, acoustic attributes, and speech content through fine-grained semantic criteria. The benchmark contains 2,258 audio-text pairs with 25,707 binary QA rubrics, enabling scalable and interpretable analysis of TTA systems. Experiments on 13 representative open-source TTA models reveal persistent limitations in fine-grained attribute control, speech-content preservation, and compositional soundscape generation. Human validation further demonstrates that our rubric-based evaluation achieves stronger alignment with human semantic judgments than conventional global similarity metrics.
SonicCaps is introduced, a large-scale audio captioning dataset comprising ~15M captions paired with ~700k audio clips, generated using a multi-modal large language model (Qwen3-Omni) conditioned on both audio and text and training CLAP models on SonicCaps with a multi-caption sampling strategy consistently improves au...
Zineb Lahrichi, Marc Ferras, G. Richard et al.· 0 citations
This work introduces the Multi-control Mixed Audio Generation benchmark, a systematic evaluation protocol that measures acoustic fidelity, speech quality, semantic alignment, and temporal accuracy, and establishes MMAG as a comprehensive benchmark for future research.
Zihao Zheng, Xuenan Xu, Jiahao Mei et al.· 0 citations
Mizar, a 159.3M-parameter ALM, is introduced, a recipe that brings together architecture, data, and three-stage training to build Mizar, a 159.3M-parameter ALM that surpasses the previous best-performing ALM below 200M parameters on all three benchmarks.
VocalCoachBench is introduced, a benchmark for evaluating audio-language models on expert vocal coaching feedback for singing, moving audio-language evaluation beyond description toward analytic feedback.
Hayeon Bang, Hounsu Kim, Wonil Kim et al.· 1 citation
MuNo-SP (Music Notation unifying Score and Performance), a text-based representation that jointly encodes score content and performance information, is introduced and an automatic training-data generation pipeline is developed that uses aligned scores and performances to produce long-form auditory analyses and musicall...
Milan Liessens Dujardin, Song Yu, Kevin Miao· 0 citations
STAG is introduced, to the authors' knowledge the first post-hoc framework for token-level spectro-temporal grounding of captions generated by audio-based MLLMs, and provides behavioral support for the faithfulness and selectivity of the explanations.