Jul 2026· International Conference on the Theory of Information Retrieval· pp. 171-175· 0 citations· 18 references
Computer Science
TL;DR
This work proposes a retrieval-time token-aware query reformulation method that emphasizes informative query tokens without retraining or changes to the retriever architecture and constructs a focused query embedding by pooling salient tokens while preserving global semantic context.
Abstract
Text-to-image retrieval (T2IR) aims to retrieve semantically relevant images given natural language queries. However, existing systems typically adopt static retrieval pipelines that process all queries uniformly, despite differences in the informativeness of individual query tokens. To address this limitation, we propose a retrieval-time token-aware query reformulation method that emphasizes informative query tokens without retraining or changes to the retriever architecture. Our method uses token-level routing to determine when query adaptation should be applied and constructs a focused query embedding by pooling salient tokens while preserving global semantic context. The reformulated query representation is used for first-stage retrieval, and the retrieved candidates are subsequently reranked. Experiments on MS COCO and Flickr30k show modest overall gains over the corresponding base pipeline, while also providing interpretable token-level evidence and region-level visualizations. Overall, our results show that retrieval-time token-aware query reformulation improves existing T2IR pipelines while making the role of individual query tokens more explicit.
Existing image–text retrieval methods often compute cross-modal similarity using global single-vector representations. Although efficient for coarse semantic alignment, such compressed representations are limited when textual queries involve fine-grained semantics, including objects, attributes, relations, and their compositional structures. This paper focuses on fine-grained text-to-image retrieval and proposes Q-GrAM, a retrieval-oriented adaptation of the BLIP-2 Q-Former. Instead of treating Q-Former queries as a homogeneous set, Q-GrAM partitions a fixed query budget into semantically differentiated groups. A text-guided router assigns token-level semantic demands to query groups, while query conditional initialization modulates each group according to group-level textual summaries. The resulting grouped visual query features are matched with text tokens through a group-aware late interaction scorer, and auxiliary routing balance and inter-group diversity regularization are introduced to stabilize semantic specialization. Experiments on MS-COCO 5K, Flickr30K, and Flickr30K-CFQ show that Q-GrAM achieves strong text-to-image retrieval performance against both global embedding baselines and representative fine-grained image–text matching methods, while maintaining competitive bidirectional retrieval performance. These results demonstrate the effectiveness of structured, text-conditioned Q-Former query specialization for fine-grained text-driven image search.
Guihe Gu, Huawei Li, Hong Qin· Italian National Conference...· 0 citations
The novel task of cross-modal query suggestion is introduced, which interactively guides users by suggesting textual refinements based on visual clusters identified in the retrieval results, and the creation of CroQS, a benchmark dataset comprising 50 diverse queries and 295 semantic clusters in generic domain.
Giacomo Pacini, Nicola Messina, Nicola Tonellotto et al.· 0 citations
SEER is presented, a framework that learns to select query-relevant images through visual scanning and retrieve textual content only where needed, combining the efficiency of visual compression with the precision of text-based reasoning.
Jiawei Xu, Zhilin Zhai, Jinrui Fang et al.· 0 citations
Conversational Query Reformulation (CQR) is an important component in Conversational Question Answering (ConvQA), where user queries are often incomplete, ambiguous, and dependent on previous dialogue turns. Recent CQR approaches have shown the effectiveness of large language models (LLMs) in generating standalone queries. However, rewritten queries do not always lead to optimal retrieval performance. This paper presents a retrieval-aware CQR framework that integrates fine-grained retrieval feedback and keyword augmentation to improve downstream document retrieval. Instead of treating the LLM-rewritten query as the final search query, our framework uses it to retrieve candidate documents, identify relevant paragraph-level evidence, generate a pseudo-answer, and extract semantically relevant keywords. These keywords are then filtered based on their alignment with the reformulated query and dialogue history before being incorporated into the final query. We evaluate the proposed framework on a sampled subset of the QReCC dataset using BM25 and GTR as sparse and dense retrieval backbones. The experimental results show that our proposed approach improves retrieval accuracy compared to the baseline model across all metrics such as Recall@10, MRR and NDCG. Specifically, under the BM25 sparse retrieval framework, our best-performing configuration achieves improvements in Recall@10 (81.29), MRR (56.06) and NDCG (60.92) compared to the baseline. A comparable pattern of substantial gains holds under the GTR dense retrieval framework, achieving a Recall@10 (76.51), MRR (53.26) and NDCG (57.54), indicating the effectiveness of our method across different retrieval paradigms.
Andhika Putra Bagaskara, Arie Ardiyanti Suryani· International Conference on...· 0 citations
The Semantic-Aware Fine-Tuning (SAFT) framework is proposed to address semantic compression in specific domains, which incorporates Semantic-Aware Soft-Label Supervision and Intra-modal Structural Distillation to establish a promising paradigm for domain-specific TBIR tasks.
Jingyang Tan, Shengan Yang, Yuanpeng Chen et al.· 0 citations
Composed image retrieval (CIR) retrieves a target image from a reference image and a text modification. This paper studies metadata-available CIR reranking, where a fixed CIR model first returns a candidate pool and gallery metadata is then used for second-stage concept-guided scoring. We introduce AutoConcept, a training-free reranker that converts concept evidence into an interpretable memory. AutoConcept filters noisy concepts, activates query-relevant positive constraints with an auxiliary negative penalty, and combines base retrieval scores with metadata-based concept-candidate alignment through inference-time calibration. On FashionIQ, AutoConcept yields significant early-rank improvements over WeiMoCIR and consistent plug-in gains on LinCIR candidate pools. Metadata-aware controls show that structured concept memory adds signal beyond direct query-text and extracted-attribute matching, while a query-only variant further supports the effectiveness of concept-level reranking. A supplementary real-human concept-label study indicates that the same memory interface can consume participant-provided evidence. These results position AutoConcept as an interpretable concept-memory reranker for product-style CIR galleries with available metadata.
