EGM-Det is proposed, an entropy-guided multimodal adaptive fusion framework for RGB-IR object detection that outperforms prior approaches by more than 10 percentage points on VEDAI and introduces cross-modal distillation to regularize the learned fusion gates and reduce fusion degradation.
Abstract
Joint use of RGB and infrared (IR) imagery can improve UAV-view object detection, but most existing methods fuse multimodal features with static or fixed weights and therefore overlook spatially varying modality reliability. We propose EGM-Det, an entropy-guided multimodal adaptive fusion framework for RGB-IR object detection. EGM-Det employs a dual-stream architecture to preserve modality-specific representations and introduces an Entropy Offset Gate Fusion module for adaptive multi-scale fusion. The module derives shallow entropy priors from input intensity, local entropy, and cross-modal discrepancy, and uses them to guide local offset alignment and spatial-channel gated fusion. It therefore selectively aggregates reliable RGB and infrared cues instead of uniformly combining heterogeneous features. We further introduce cross-modal distillation to regularize the learned fusion gates and reduce fusion degradation. Each student branch extracts complementary knowledge from the cross-modality teacher branch matched to the main branch, while entropy-adaptive supervision emphasizes uncertain modality decisions. Experiments on DroneVehicle, LLVIP, and VEDAI demonstrate state-of-the-art performance across all three benchmarks; in particular, EGM-Det outperforms prior approaches by more than 10 percentage points on VEDAI.
Visible-infrared (RGB-IR) object detection equips uncrewed aerial vehicles (UAVs) with robust, around-the-clock perception capabilities. However, practical deployments are frequently constrained by weak spatial misalignment between heterogeneous sensors. This issue degrades the object detection accuracy of UAVs, for wh...
Bo-Yu Zhang, Chao-Yang Niu, Wan-Jie Lu et al.· IEEE Journal of Selected Top...· 0 citations
A Boundary-Guided Dual-Perspective Cross-Modal Fusion Network (BDPNet) is proposed to explicitly preserve shallow geometric structures and decouple deep semantic fusion into macroscopic and microscopic perspectives.
Hu Lin, Zhi-Wei Fu, Xiu-Mei Chen et al.· Remote Sensing· 0 citations
Small-object detection under low illumination remains a persistent challenge in aerial surveillance, nighttime patrol, and safety-critical vision tasks, where single-modality sensors—RGB or infrared alone—provide degraded or incomplete target information. This paper proposes an RGB-T object detection network that combi...
This paper proposes a Difference-guided Detail Enhancement Fusion Network (DDEF-Net) for UAV-based RGB–thermal (RGB-T) object detection, which enables effective complementary exploitation of visible and infrared information in complex scenarios. A Difference-guided Kolmogorov–Arnold Network (KAN) Calibration Fusion mod...
Yu-Jie Li, Zheng-Sheng Chen, D. Ma et al.· Remote Sensing· 0 citations
Multimodal remote sensing object detection benefits from the complementary characteristics of visible and infrared modalities. However, owing to the heterogeneous imaging mechanisms of RGB and infrared sensors, feature representations from the two modalities progressively become semantically inconsistent during hie...
Aerial vehicle detection using red–green–blue (RGB) and infrared (IR) imagery supports unmanned aerial vehicle (UAV) perception, but oriented localization remains difficult because vehicles are small, dense, arbitrarily oriented, and affected by illumination. Fusion must decide both how cross-modal differences are used...
Chao Huang, Zhi-Hong Xu· Italian National Conference...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.