2026· IEEE Transactions on Information Forensics and Security· Vol 21, pp. 7767-7779· 0 citations· 45 references
Abstract
Text-to-image (T2I) models can be exploited to produce unsafe images. Existing safety measures, e.g., content moderation or model alignment, can be weakened by adversaries who attempt to restore unsafe generation through model fine-tuning. This paper presents Patronus, a defensive framework that improves T2I models’ resistance to the gradient-based adversarial fine-tuning attacks evaluated in this work. Specifically, we design a co-trained safety decoder that produces a deliberately corrupted output for a latent representation associated with unsafe content while preserving normal decoding for benign content. We also strengthen the decoder and U-Net with a non-fine-tunable learning mechanism. Across I2P, SneakyPrompt, and MMA-Diffusion, Patronus obtains attack success rates of 0.01–0.03 and true positive rates of 0.98–0.99. On benign prompts, it obtains FID 23.6, LPIPS 0.78, and a false positive rate of 0.01. The fine-tuning stress tests separately report the optimization losses of the defended decoder and U-Net under the evaluated attack settings.
DiSCO is proposed, a zero-shot, strictly black-box defense that operates entirely at the prompt level as a plug-and-play module, requiring no model retraining, fine-tuning, or access to model internals, and can be readily applied to any text-to-image system without necessitating any changes to the model itself.
Tong Zhang, M. Alfarra, Carlos Hinojosa et al.· 0 citations
Adversarial Probing for Implicit VulnErabilities (AdvPIE), a multimodal agentic framework to expose implicit vulnerabilities without requiring access to the parameters of target models, is proposed.
Chang-Sha Ma, Junlin Han, Shuo Chen et al.· 0 citations
Vision-Language Models (VLMs) are known to be vulnerable to adversarial attacks, where subtle perturbations to images or texts induce erroneous outputs. However, most text-based attacks are adapted from language-model-centric methods, in which the visual input is fixed during optimization, resulting in adversarial prom...
Li Zeng, Ze-Yu Ye, Meng Xie et al.· arXiv.org· 0 citations
A self-initiated investigation of purification-based adversarial detection, comparing three families of detection signals across six detectors that share a CLIP ViT-L/14 backbone finds that raw $|\Delta \text{logit}|$ under median-3 purification, applied through the EFFORT detector, separates adversarial inputs from cl...
Junghyun Kim, Seunghyun Kim, Ji-myung Woo· arXiv.org· 0 citations
This work suggests that existing AI-generated image detectors are easily deceived by adversarial perturbations, highlighting the need for more robust detectors.
Yun-Yun Tsai, Rui Xu, Chengzhi Mao et al.· 0 citations
I2VShield is proposed, a privacy protection method based on generative adversarial attacks tailored to Diffusion Transformer (DiT)-based I2V models that achieves highly competitive protection performance across various datasets and mainstream DiT-based I2V models, particularly in disrupting spatiotemporal coherence, wh...
