DFPR: Dynamic Fine-Grained Perceptive Bidirectional Image-Text Retrieval
Abstract
Parameter-Efficient Fine-Tuning (PEFT) has become a promising paradigm for Remote Sensing Image-Text Retrieval (RSITR), enabling the adaptation of large pre-trained models with a small number of trainable parameters. However, remote sensing image-text pairs exhibit heterogeneous structures: compositional caption representations entangle entities, attributes, relations, and quantities, while overhead images contain substantial background clutter. To address these challenges, we propose Dynamic Fine-Grained Perceptive Bidirectional Image-Text Retrieval (DFPR), a PEFT framework tailored for RSITR that introduces structured semantic routing and entity-guided visual focusing during training. To alleviate this semantic entanglement, we introduce a multi-structured mixture-of-experts (MS-MoE). Its dual-gating mechanism adaptively routes text representations to specialized experts, injecting fine-grained domain priors to disentangle caption semantics into explicit components and yield a dedicated entity anchor. Conditioned on this anchor, spectral graph-guided focusing (SG2F) suppresses background interference through spectral filtering and aggregates salient visual features. To mitigate the inherent optimization imbalance between image-to-text and text-to-image retrieval, a bidirectional calibration constraint mechanism (BCCM) dynamically calibrates the directional loss weights using online feedback and exponential smoothing, promoting more stable and balanced optimization. Extensive experiments on three benchmarks demonstrate that DFPR consistently outperforms the compared PEFT baselines in retrieval accuracy.