Ablations show that feedback-based refinement and audio-variant search substantially improve attack discovery.
Abstract
Large audio-language models (LALMs) make it possible to interact with language models through speech, music, and environmental sound, but they also introduce a safety surface that is difficult to expose with text-only red-teaming. We study automated audio-grounded red-teaming, where a text query must remain safe in isolation while the joint text-audio input induces harmful target behavior. We propose ARENA, a closed-loop framework that trains a controller on an independent 2,000case text-audio dataset. MD-Judge supplies training rewards and adaptive search feedback, while a separate, non-adaptive Llama Guard 3 evaluator alone labels final outcomes. On 520 held-out AdvBench objectives, ARENA achieves FDR/PSR of 87.9/100.0%, 71.5/96.3%, 68.1/100.0%, and 75.4/98.5% on Audio Flamingo 3, Qwen2-Audio, MiMo-Audio, and GPTAudio, respectively. Ablations show that feedback-based refinement and audio-variant search substantially improve attack discovery.
Large audio language models (LALMs) can describe what is heard, but their ability to localize when queried content occurs remains less systematically evaluated. We present TAG-Bench, a benchmark for temporal audio grounding in which a model returns every time interval that matches a natural-language query. TAG-Bench co...
Yu-Hang Dai, Xin Shu, Zeng-Xi Li et al.· 2 citations
AudioICL-Bench is introduced, a diagnostic benchmark whose per-episode rules are resampled so that no correct answer is recoverable from prior knowledge, and its nine tasks are organized along two axes that separate what must be learned from demonstrations from what must be perceived in the signal, enabling failures to...
Jia-Hung Chen, Yi-Cheng Lin, Kai-Wei Chang et al.· 0 citations
SpeechAgent-R, an audio agent that coordinates its intrinsic multimodal understanding with external skills and tools, is developed and HIU-Bench is introduced to jointly evaluate task performance, interaction quality and generalization to diverse task settings.
Yuwen Wang, Tian-Hao Zhang, Ming Cai et al.· 0 citations
Large audio-language models can produce fluent and musically plausible responses, yet it often remains unclear whether those responses are grounded in the audio input. We introduce temporal music grounding, a task in which a model returns one or more time spans corresponding to a queried musical note, event, or pattern...
This work introduces Audio-Zero, the first label-free self-evolution framework in the field of LALMs that improves fine-grained auditory perception and reasoning and reveals that increasingly fine-grained auditory descriptions emerge naturally from game pressure.
Siqian Tong, Xuan Li, Chao-Zhuo Li et al.· arXiv.org· 1 citation
While large audio-language models have achieved remarkable progress in auditory perception, they still lag behind text-based large language models in deep logical reasoning, primarily due to the scarcity of high-quality audio reasoning data. To bridge this gap, we propose X$^3$-OPD, a cross-modal on-policy distillation...
Dongjie Fu, Di Cao, Xize Cheng et al.· arXiv.org· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
