Skip to content
Book Open access

DFPR: Dynamic Fine-Grained Perceptive Bidirectional Image-Text Retrieval

Nov 2026 · Proceedings of the 34th ACM International Conference on Multimedia · 0 citations · 35 references

Abstract

Parameter-Efficient Fine-Tuning (PEFT) has become a promising paradigm for Remote Sensing Image-Text Retrieval (RSITR), enabling the adaptation of large pre-trained models with a small number of trainable parameters. However, remote sensing image-text pairs exhibit heterogeneous structures: compositional caption representations entangle entities, attributes, relations, and quantities, while overhead images contain substantial background clutter. To address these challenges, we propose Dynamic Fine-Grained Perceptive Bidirectional Image-Text Retrieval (DFPR), a PEFT framework tailored for RSITR that introduces structured semantic routing and entity-guided visual focusing during training. To alleviate this semantic entanglement, we introduce a multi-structured mixture-of-experts (MS-MoE). Its dual-gating mechanism adaptively routes text representations to specialized experts, injecting fine-grained domain priors to disentangle caption semantics into explicit components and yield a dedicated entity anchor. Conditioned on this anchor, spectral graph-guided focusing (SG2F) suppresses background interference through spectral filtering and aggregates salient visual features. To mitigate the inherent optimization imbalance between image-to-text and text-to-image retrieval, a bidirectional calibration constraint mechanism (BCCM) dynamically calibrates the directional loss weights using online feedback and exponential smoothing, promoting more stable and balanced optimization. Extensive experiments on three benchmarks demonstrate that DFPR consistently outperforms the compared PEFT baselines in retrieval accuracy.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.