Skip to content
Conference Open access

Transferable Adversarial Attacks On Vlp Models Via Attention-Guided Dual-Modal Perturbation

Sep 2026 · 2026 23rd International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP) · pp. 1-5 · 0 citations · 14 references

Abstract

Visual-Language Pre-training (VLP) models have achieved remarkable success in various practical applications, yet they remain vulnerable to adversarial attacks that can mislead their predictions. Adversarial attacks are valuable for revealing the vulnerabilities and robustness issues of VLP models. However, existing studies on adversarial attacks often fail to fully exploit the critical alignment features of VLP models, resulting in limited transferability across different models. To address this issue, we propose attention-guided dual-modal perturbation(AGDP) method for semantic alignment failure attacks. Specifically, to disrupt the alignment between the textual and visual modalities in VLP models, we introduce an attention-guided strategy to identify and perturb critical alignment features in the textual modality. Meanwhile, we maximize the representation distance between clean and adversarial images in the model’s fused representation space to further enhance the transferability of adversarial examples. Extensive experiments under various settings demonstrate that AGDP consistently outperforms existing baseline methods, validating the effectiveness of the proposed approach.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.