Construction of a fine-grained retrieval model for archival text-image based on the integration of scene graph generation and attention mechanism

Semantic Composition via Optimal Transport for Composed Image Retrieval

Given a reference image, composed image retrieval aims to search for the target image from the image gallery with a modification sentence. The complicated semantic relationships within the composed query to comprehend the preserved visual context and modified semantics pose a challenge for the researchers. Most current approaches concentrate on the global or fragmental composition to combine the multi-modal query features and bridge the distance between the query and the target. However, global-level matching tends to ignore fine-grained components that distinguish the matched instance from hard negative instances. As for the fragment-level matching, the over-sensitivity to the local noise due to the lack of explicit local labels and excessive computational demands may have a detrimental impact on performance. In this paper, we propose Semantic Composition via Optimal Transport (SCOT) for composed image retrieval to address the aforementioned issues. To optimize the computational resources, SCOT first introduces Hybrid-modal Affinity Summarization (HAS) to compact salient relationships in the affinity matrix from the reference image patches and modification tokens. Furthermore, to mitigate the over-confidence in local noise, SCOT adaptively assigns optimal transport weights on the salient aligned fragments to Retain Prototype from Alignments (RAS) and guides discriminant semantics learning to Update Semantics from Enhanced Embeddings (USEE), while preserving the characteristics from global structures. Specifically, RAS module investigates the preserved semantics from the reference image based on the affinity matrix, meanwhile USEE module distills significant modification semantics. The proposed SCOT has demonstrated advanced experimental performances on widely-used datasets FashionIQ, CIRR, and Fashion200K. Ablative studies manifest the effectiveness of optimal transport guidance on preserved visual prototypes and updated semantics.

Research on Cross-modal Image Retrieval and Image-Text Matching Based on Visual-Language Pre-trained Models

: Cross-modal image retrieval and image-text matching are core tasks connecting computer vision and natural language processing, aiming to eliminate the heterogeneous gap between visual and text modalities. Visual-language pre-trained models, through large-scale data learning and cross-modal alignment, have become the dominant technical paradigm for solving this task. This paper systematically reviews the development of visual-language pre-trained models in the field of cross-modal retrieval, classifies and analyzes existing methods from three dimensions: model architecture, pre-training objectives, and downstream adaptation, and focuses on discussing the architectural differences between dual encoders and fusion encoders, the design evolution of pre-training tasks, and adaptation techniques such as efficient parameter fine-tuning. Based on this, we summarize mainstream datasets and evaluation metrics, compare the performance of representative models, and deeply analyze three key challenges: fine-grained alignment, noise robustness, and inference efficiency. Finally, we look forward to future research directions such as few-shot generalization, unified multi-task framework, and interpretability, hoping to provide a reference for further research in this field.

BiFGA: A Bidirectional Image–Text-Guided Fine-Grained Alignment Method for Remote Sensing Image–Text Retrieval

Remote sensing image–text retrieval matches remote sensing images with textual descriptions. However, dense objects, complex spatial layouts, and large-scale variations make fine-grained cross-modal alignment challenging. Existing methods mainly rely on global representations or direct patch-token similarities. Global matching may overlook small objects and spatial details, whereas local matching is susceptible to background clutter, scale variations, and redundant tokens. To address these limitations, this article proposes BiFGA, a bidirectional fine-grained alignment framework that uses cross-modal similarity responses as semantic anchors. Its bidirectionally guided fine-grained matching module reconstructs text-guided visual relations and vision-guided textual relations, enabling second-order interactions between image regions and textual units. BiFGA also introduces a hybrid window spatial-channel attention module to enhance visual regions and a channel-splitting phrase-level feature extraction module to model multiscale textual semantics. Global similarity and local fine-grained matching are integrated through coarse-to-fine scoring. Experiments on RSITMD, RSICD, UCM-Caption, and Sydney-Captions demonstrate the effectiveness and stability of BiFGA. Averaged over five independent runs, BiFGA achieves mR scores of 50.07%, 35.15%, 59.42%, and 53.19%, respectively. It obtains the best average mR among the compared methods on RSITMD, RSICD, and Sydney-Captions, while remaining comparable to the strongest method on UCM-Caption.

Hierarchical Prompt for Task-Adaptive Composed Image Retrieval

Composed image retrieval (CIR) seeks to retrieve target images using multi-modal queries, specifically a reference image paired with modification text. Central to CIR is integrating textual semantic modifications with visual content. Despite its importance, existing approaches typically employ a static fusion paradigm, failing to account for the semantic heterogeneity of user queries, which encompass diverse task types (e.g., addition, replacement) and var-ied content. To address these limitations, we propose the Task-Adaptive Hier-archical Prompt (TAHP) framework. TAHP guides feature extraction through dynamically generated, task-specific prompts structured at three hierarchical levels: task-type, task-content, and general prompts. Furthermore, we design a Prompt Dynamic Generation Module to adaptively synthesize prompts condi-tioned on user queries and introduce a False Negative Correction Loss to optimize cross-modal feature fusion. Extensive experiments on FashionIQ and CIRR datasets demonstrate that TAHP achieves state-of-the-art performance against existing CIR approaches.

Object-Level Access to SVG Scenes: A Benchmark and Raster-Grounded Method for Text-Guided Vector Object Extraction

Rethinking Text-Based Image Retrieval in Specific Domain

The Semantic-Aware Fine-Tuning (SAFT) framework is proposed to address semantic compression in specific domains, which incorporates Semantic-Aware Soft-Label Supervision and Intra-modal Structural Distillation to establish a promising paradigm for domain-specific TBIR tasks.