DAAF: Dual-stream adaptive attention fusion with distribution alignment for remote sensing object detection

Abstract

Remote sensing object detection faces three challenges: extreme scale variation, arbitrary rotational orientations, and complex intermingled backgrounds. Although fusion of Convolutional Neural Networks (CNNs) and Transformers combines spatial precision with global modeling, it faces two limitations: (i) feature distribution misalignment between modalities, and (ii) ineffective exploitation of complementary discriminative strengths. To address these issues, we introduce DAAF (Dual-stream Adaptive Attention Fusion), a three-stage fusion framework that systematically aligns and integrates CNN and Transformer features. DAAF comprises three components: (1) The Unified Feature Distribution Calibration (UFDC) module applies instance normalization to align feature distributions, reducing Kullback-Leibler (KL) divergence by 94%; (2) The Heterogeneous Channel-wise Synergistic Selection (HCSS) module employs independent attention pathways for each stream, validated by zero overlap in top-10 channel weights between CNN and Transformer branches; (3) The Spatially Adaptive Discriminative-Preserving Gate (SADPG) module employs pixel-wise gating to adaptively balance fused and single-branch features. Experiments on three benchmarks (NWPU VHR-10, DOTA v1.0, DIOR) show consistent improvements averaging +2.05 percentage points (pp) in mean Average Precision (mAP) over the Add Fusion baseline, with substantial gains on structurally complex categories in NWPU (Bridge: + 17.36pp, Vehicle: + 13.18pp). These improvements are achieved with only 206K additional parameters (0.37% increase over baseline) and 7.2% latency rise compared to the baseline fusion method, demonstrating favorable efficiency for practical deployment.