BCAP: Black-Box Cross-Modal Adversarial Patch Attacks Against Multimodal Vehicle Recognition Systems via UAV Remote Sensing

Abstract

The reliability of deep learning-based object detection in uncrewed aerial vehicle (UAV) remote sensing increasingly relies on fusing visible and infrared sensors. However, existing attack methods used to evaluate their robustness are often unrealistic, assuming white-box access or targeting only a single modality. This article first reveals a critical cross-modal vulnerability through region sensitivity analysis on various YOLO detectors, demonstrating that both visible and infrared models exhibit a shared spatial weakness by being most sensitive to perturbations in the target vehicle’s central area. Leveraging this key insight, we propose a framework with black-box cross-modal adversarial patch (BCAP) attacks on UAV detectors. BCAP introduces a multiobjective optimization formulation that models visible and infrared modalities as independent yet competing objectives, which enables effective cross-modal optimization without manual weight tuning. It employs a novel grid-constrained Bezier curve generation method to efficiently search for smooth, effective patch shapes within this sensitive central region, further enhanced by an adaptive population restart strategy. Extensive experiments on the DroneVehicle dataset show BCAP achieves a 50%–61% cross-modal attack success rate (ASR), significantly outperforming the 37%–42% range of existing methods. This work provides a more potent and realistic baseline for evaluating the end-to-end reliability of multimodal UAV remote sensing systems.