Skip to content
Preprint

AudioScape-TTA: A Structured Soundscape Benchmark for Fine-Grained Text-to-Audio Evaluation

Aug 2026 · 0 citations · 29 references
Computer Science

TL;DR

A rubric-based audio-grounded evaluation framework that verifies event realization, acoustic attributes, and speech content through fine-grained semantic criteria, and human validation demonstrates that the benchmark achieves stronger alignment with human semantic judgments than conventional global similarity metrics.

Abstract

Text-to-audio (TTA) generation has recently achieved remarkable progress in synthesizing realistic audio from natural language descriptions. However, determining whether generated audio faithfully satisfies complex textual instructions remains challenging. Existing benchmarks mainly rely on global similarity metrics, providing limited insight into fine-grained semantic failures. To address this limitation, we introduce \textbf{AudioScape-TTA}, a structured and complexity-aware benchmark for fine-grained TTA evaluation. AudioScape-TTA represents realistic soundscapes through modality-aware semantic structures and characterizes generation complexity using event density and structural complexity. Based on these annotations, we propose a rubric-based audio-grounded evaluation framework that verifies event realization, acoustic attributes, and speech content through fine-grained semantic criteria. The benchmark contains 2,258 audio-text pairs with 25,707 binary QA rubrics, enabling scalable and interpretable analysis of TTA systems. Experiments on 13 representative open-source TTA models reveal persistent limitations in fine-grained attribute control, speech-content preservation, and compositional soundscape generation. Human validation further demonstrates that our rubric-based evaluation achieves stronger alignment with human semantic judgments than conventional global similarity metrics.

View source

Similar papers

#natural language process... Preprint Sep 2026

SonicCaps: Large-Scale Diverse and Fine-Grained Captioning for Improved Audio-Retrieval

SonicCaps is introduced, a large-scale audio captioning dataset comprising ~15M captions paired with ~700k audio clips, generated using a multi-modal large language model (Qwen3-Omni) conditioned on both audio and text and training CLAP models on SonicCaps with a multi-caption sampling strategy consistently improves au...

Zineb Lahrichi, Marc Ferras, G. Richard et al. · 0 citations
Preprint Aug 2026

MMAG: A Multi-Control Mixed Audio Generation Benchmark

This work introduces the Multi-control Mixed Audio Generation benchmark, a systematic evaluation protocol that measures acoustic fidelity, speech quality, semantic alignment, and temporal accuracy, and establishes MMAG as a comprehensive benchmark for future research.

Zihao Zheng, Xuenan Xu, Jiahao Mei et al. · 0 citations

Mizar: A 159M-Parameter Audio-Language Model for Audio Understanding

Mizar, a 159.3M-parameter ALM, is introduced, a recipe that brings together architecture, data, and three-stage training to build Mizar, a 159.3M-parameter ALM that surpasses the previous best-performing ALM below 200M parameters on all three benchmarks.

Kai-Yang Li, Shaobo Han, Yue Tian et al. · 0 citations
Preprint Aug 2026

VocalCoachBench: Benchmarking Audio-Language Models on Expert Feedback for Singing

VocalCoachBench is introduced, a benchmark for evaluating audio-language models on expert vocal coaching feedback for singing, moving audio-language evaluation beyond description toward analytic feedback.

Hayeon Bang, Hounsu Kim, Wonil Kim et al. · 1 citation
Preprint Sep 2026

Unifying Score and Performance for Fine-Grained Music Understanding in Audio-Language Models

MuNo-SP (Music Notation unifying Score and Performance), a text-based representation that jointly encodes score content and performance information, is introduced and an automatic training-data generation pipeline is developed that uses aligned scores and performances to produce long-form auditory analyses and musicall...

Milan Liessens Dujardin, Song Yu, Kevin Miao · 0 citations

What Did the MLLM Hear? Token-Level Spectro-Temporal Grounding for Audio MLLM Explainability

STAG is introduced, to the authors' knowledge the first post-hoc framework for token-level spectro-temporal grounding of captions generated by audio-based MLLMs, and provides behavioral support for the faithfulness and selectivity of the explanations.

Lucia Cascone, Valeria Fraenza, Michele Nappi et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.