Rethinking Text-Based Image Retrieval in Specific Domain

CoCo-IR: Contextual Composed Image Retrieval

A new model based on a Large Multimodal Model (LMM) that functions as a context-aware reasoner for CoCo-IR is proposed, which interprets the entire interaction history to generate Transformable Image Embeddings (TIE) that evolve across turns.

Hierarchical Prompt for Task-Adaptive Composed Image Retrieval

Composed image retrieval (CIR) seeks to retrieve target images using multi-modal queries, specifically a reference image paired with modification text. Central to CIR is integrating textual semantic modifications with visual content. Despite its importance, existing approaches typically employ a static fusion paradigm, failing to account for the semantic heterogeneity of user queries, which encompass diverse task types (e.g., addition, replacement) and var-ied content. To address these limitations, we propose the Task-Adaptive Hier-archical Prompt (TAHP) framework. TAHP guides feature extraction through dynamically generated, task-specific prompts structured at three hierarchical levels: task-type, task-content, and general prompts. Furthermore, we design a Prompt Dynamic Generation Module to adaptively synthesize prompts condi-tioned on user queries and introduce a False Negative Correction Loss to optimize cross-modal feature fusion. Extensive experiments on FashionIQ and CIRR datasets demonstrate that TAHP achieves state-of-the-art performance against existing CIR approaches.

MASCOT: Model-Aware Submodular Coverage for Composite-Attribute Text-to-Image Retrieval

Vision-Language Models (VLMs) are highly effective in retrieving semantically relevant images. However, in practice, relevance alone is often insufficient. Systems must also achieve Result Diversification (RD) across composite attributes such as geography and time, a task for which precise control remains challenging. Current re-ranking methods, such as Multi-Source Determinantal Point Processes (MS-DPP), address this using manifold-based repulsion over similarity representations. Although this strategy is effective for broad exploration, it exposes a key limitation in manifold-based models: when subjected to diversity-decrease tasks on discrete metadata, they suffer substantial degradation in early-rank recall. To bridge this gap, we introduce MASCOT (Model-Aware Submodular Coverage for Composite-Attribute Text-to-Image Retrieval). Instead of relying on manifold repulsion, MASCOT formulates multi-attribute diversity as a resource allocation problem, projecting attributes into a soft-binning space weighted by query-driven importance. Averaged across the three PixelProse diversity-decrease tasks, MASCOT preserves an early-rank recall (R@10) of 88.58%, while MS-DPP retains 67.63%. The margin widens under composite constraints: on PP_geo_hour, where temporal and geographic diversity must be suppressed simultaneously, MS-DPP's recall collapses from 0.9737 to 0.4931 and its top-ranked result degrades to R@1 = 0.23, while MASCOT holds R@10 = 0.9410 and R@1 = 0.7202 at a diversity metric above the unconstrained baseline. We do not claim uniform superiority: on aggregate diversity-relevance scores our own simpler ablations attain higher harmonic means on all three decrease tasks, and MASCOT's advantage is specific to recall beyond rank 1 under composite constraints.

Object image retrieval based on self-supervised learning and zero-shot learning

Dual Retrieval Queries Fine-Tuning for Composed Image Retrieval.

Composed Image Retrieval (CIR) is a popular multi-modal retrieval task that aims to retrieve a target image based on a query composed of a reference image and modification text. The challenge lies in how to effectively retrieve a target image that preserves the visual content of the reference image while incorporating the changes described by the modification text. Existing CIR methods primarily employ a fusion-based strategy or a textual-inversion strategy during training. Although these methods have achieved promising results, they are limited in fully leveraging multi-modal information. This results in modality redundancy, where the retrieval process is dominated by one modality while ignoring the other. To address this issue, we propose an asymmetric fusion mechanism to generate dual retrieval queries of different granularity, enabling the model to fully use multi-modal information. Specifically, we propose a novel method termed Dual Retrieval Queries Fine-Tuning for Composed Image Retrieval (DRQ-CIR), which consists of two key components: 1) a Bilateral Multi-Modal Fusion (BMMF) module based on pre-trained VLMs, which combines the reference image and modification text to generate an enriched retrieval query; and 2) a Dual Retrieval Queries Fine-Tuning (DRQ-FT) module, which employs latent prompts to generate an enhanced retrieval query. Dual retrieval queries are used for contrastive learning with the target image to fine-tune the model and improve retrieval performance. Additionally, we introduce a bi-directional training paradigm to ensure retrieval consistency and further exploit the triplets. Extensive experiments validate the effectiveness of our proposed method on four established benchmark datasets. (Code will be available at: https://github.com/Crystal-twy998/DRQ-CIR).

Unifying Granularity and Reliability: A Robust and Efficient Framework for Text-based Person Retrieval

Text-based person retrieval (TPR) has become a crucial task in cross-modal retrieval due to its broad application in fields such as public safety and criminal investigation. Existing TPR methods typically rely on fully fine-tuning large-scale pretrained vision-language models like CLIP, which incurs high computational costs and tends to exhibit poor generalization in unseen domains due to overfitting. Fortunately, Parameter-Efficient Transfer Learning (PETL) has emerged as a lightweight alternative. However, applying PETL to TPR remains challenging, as its limited adaptation capacity struggles to capture intricate identity cues and becomes highly susceptible to gradient interference from unreliable image-text pairs. To address these challenges, we present a PETL-based framework named UniGR that unifies granularity and reliability for robust and efficient TPR. Specifically, we design a multi-granularity relational adapter (MRA) to capture both coarse-grained global and fine-grained local relational features among tokens, equipping the generic backbone with the task-specific, precise understanding needed for TPR. To combat the noise sensitivity of PETL, a reliability-aware reweighting strategy (RRS) is introduced to adaptively down-weight unreliable samples during training. Furthermore, we propose a parameter-free cross-modal cyclic verification (CMCV) module to mitigate ambiguities in cross-modal matching computations and refine retrieval ranking further. Experiments on benchmarks corroborate the superiority of UniGR among parameter-efficient methods. Remarkably, with only 4.5% of trainable parameters, UniGR outperforms most fully fine-tuned methods while maintaining strong generalization.