Skip to content
Preprint

MMA-SafetyBench: A Benchmark for Multimodal Agent Safety Evaluation

Oct 2026 · 0 citations · 20 references
Computer Science

TL;DR

A trajectory-guided procedure that identifies candidate injection points and inserts reproducible visual payloads into the observation stream is proposed, which is used to construct MMA-SafetyBench, which contains 1,083 adversarial trajectories across web automation, desktop GUI control, mobile navigation, document understanding, and video processing.

Abstract

Multimodal agents increasingly rely on visual interfaces to perceive state and select actions, creating an attack surface not captured by text-only safety evaluations. We study Visual Semantic Hijacking, in which deceptive but semantically meaningful interface elements, such as forged system alerts, cause an agent to depart from its authorized task. We propose a trajectory-guided procedure that identifies candidate injection points and inserts reproducible visual payloads into the observation stream. Using this procedure, we construct MMA-SafetyBench, which contains 1,083 adversarial trajectories across web automation, desktop GUI control, mobile navigation, document understanding, and video processing. Across nine evaluated models, Attack Success Rate (ASR) reaches 87.96%, while Semantic Compromise Rate (SCR) reaches 98.09%. A 1--3--5 scoring scheme distinguishes target-matching outputs, semantic disruption without a target match, and successful preservation of the benign task. Controls covering all nine models and five domains show that attacks persist under neutral prompts. A safety-priority prompt lowers ASR without eliminating the failure mode.

View source

Similar papers

#artificial intelligence Preprint Oct 2026

AgentSafeEvo: Adversarial Co-Evolution for LLM Agent Safety

This paper introduces AgentSafeEvo, a co-evolutionary reinforcement learning framework for agent safety built on an attacker-defender game spanning diverse tasks and attack types, and introduces learnability-guided task weighting, which uses within-task defender-reward variability to adjust each task's contribution to...

Chen-Xu Wang, Xiaoyu Wen, Han Qi et al. · 0 citations
Open access Sep 2026

Style2Risk: A Closed-Loop Testing System for Safety Evaluation of Automated Vehicles Under Style-Conditioned Adversarial Interactions

Closed-loop safety evaluation requires traffic agents that expose planner weaknesses without producing arbitrary behavior. Existing adversarial tests often couple behavior generation, risk search, and failure counting, limiting control over how interactions are created and whether repeated events constitute new evidenc...

J. Lai, Meng Wang, Zhen Zhang et al. · 0 citations
Book Open access Nov 2026

INSURE: Inference-Time Defense Framework for Securing Agents against Hijacking

Agents are increasingly susceptible to hijack attacks, where malicious instructions embedded in cross-modal inputs (e.g., adversarial images or retrieved web content) are used to deceive the agent's execution. Existing defenses typically rely on strict filtering or refusal mechanisms, which suffer from a critical limit...

Che Wang, Fu-Yao Zhang, Jiaming Zhang et al. · 0 citations
Preprint Oct 2026

Intent-Preserving Adversarial User Simulation for Multi-Turn Agent Evaluation

The results demonstrate that StressSim exposes robustness gaps missed by standard and hand-designed user simulators while maintaining conversation realism, achieved through lightweight prompt optimisation alone, without model fine-tuning.

A. Sharma, Brian Formento, A. Lomuscio · 0 citations
Preprint Sep 2026

DIBench: Benchmarking Decision Integrity of GUI-based Mobile Agents Under Deceptive Injections

DIBench provides a unified, reproducible benchmark to quantify the risk of in-task goal deviation in mobile agents and enable comparable evaluations of safety defenses, as well as providing a decision integrity benchmark for measuring this risk in mobile agents.

Li Hu, Kang-Hua Mo, Ying-Zi Jin et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.