Skip to content
Preprint

EGM-Det: Entropy-Guided Multimodal Adaptive Fusion for UAV RGB-IR Object Detection

Aug 2026 · 1 citation · 45 references
Computer Science

TL;DR

EGM-Det is proposed, an entropy-guided multimodal adaptive fusion framework for RGB-IR object detection that outperforms prior approaches by more than 10 percentage points on VEDAI and introduces cross-modal distillation to regularize the learned fusion gates and reduce fusion degradation.

Abstract

Joint use of RGB and infrared (IR) imagery can improve UAV-view object detection, but most existing methods fuse multimodal features with static or fixed weights and therefore overlook spatially varying modality reliability. We propose EGM-Det, an entropy-guided multimodal adaptive fusion framework for RGB-IR object detection. EGM-Det employs a dual-stream architecture to preserve modality-specific representations and introduces an Entropy Offset Gate Fusion module for adaptive multi-scale fusion. The module derives shallow entropy priors from input intensity, local entropy, and cross-modal discrepancy, and uses them to guide local offset alignment and spatial-channel gated fusion. It therefore selectively aggregates reliable RGB and infrared cues instead of uniformly combining heterogeneous features. We further introduce cross-modal distillation to regularize the learned fusion gates and reduce fusion degradation. Each student branch extracts complementary knowledge from the cross-modality teacher branch matched to the main branch, while entropy-adaptive supervision emphasizes uncertain modality decisions. Experiments on DroneVehicle, LLVIP, and VEDAI demonstrate state-of-the-art performance across all three benchmarks; in particular, EGM-Det outperforms prior approaches by more than 10 percentage points on VEDAI.

View source

Similar papers

Open access 2026

SCAF-Net: Sparse Cross-Modal Adaptive Fusion Network for RGB-IR UAV Object Detection

Visible-infrared (RGB-IR) object detection equips uncrewed aerial vehicles (UAVs) with robust, around-the-clock perception capabilities. However, practical deployments are frequently constrained by weak spatial misalignment between heterogeneous sensors. This issue degrades the object detection accuracy of UAVs, for wh...

Bo-Yu Zhang, Chao-Yang Niu, Wan-Jie Lu et al. · 0 citations
Open access Sep 2026

Boundary-Guided Dual-Perspective Cross-Modal Fusion Network for RGB-IR Object Detection

A Boundary-Guided Dual-Perspective Cross-Modal Fusion Network (BDPNet) is proposed to explicitly preserve shallow geometric structures and decouple deep semantic fusion into macroscopic and microscopic perspectives.

Hu Lin, Zhi-Wei Fu, Xiu-Mei Chen et al. · 0 citations
Conference Sep 2026

A low-light RGB-thermal infrared target detection algorithm based on adaptive cross-modal fusion and multiscale feature enhancement

Small-object detection under low illumination remains a persistent challenge in aerial surveillance, nighttime patrol, and safety-critical vision tasks, where single-modality sensors—RGB or infrared alone—provide degraded or incomplete target information. This paper proposes an RGB-T object detection network that combi...

Xin Song, Jian-Jun Luo, Ping Wang · 0 citations
Open access Sep 2026

DDEF-Net: A Difference-Guided Detail Enhancement Fusion Network for UAV-Based RGB-T Object Detection

This paper proposes a Difference-guided Detail Enhancement Fusion Network (DDEF-Net) for UAV-based RGB–thermal (RGB-T) object detection, which enables effective complementary exploitation of visible and infrared information in complex scenarios. A Difference-guided Kolmogorov–Arnold Network (KAN) Calibration Fusion mod...

Yu-Jie Li, Zheng-Sheng Chen, D. Ma et al. · 0 citations
Open access Sep 2026

MA-YOLO for robust multimodal feature fusion under weak cross-modal alignment in remote sensing object detection

Multimodal remote sensing object detection benefits from the complementary characteristics of visible and infrared modalities. However, owing to the heterogeneous imaging mechanisms of RGB and infrared sensors, feature representations from the two modalities progressively become semantically inconsistent during hie...

Jian-Qiong Huang, Zhi-Hong Lin, Waqar Khan et al. · 0 citations
Open access Oct 2026

MSDGFM-YOLO11n-OBB: Selective High-Resolution Difference-Guided RGB–Infrared Fusion for Aerial Oriented Vehicle Detection

Aerial vehicle detection using red–green–blue (RGB) and infrared (IR) imagery supports unmanned aerial vehicle (UAV) perception, but oriented localization remains difficult because vehicles are small, dense, arbitrarily oriented, and affected by illumination. Fusion must decide both how cross-modal differences are used...

Chao Huang, Zhi-Hong Xu · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.