Conversational Query Reformulation Using Fine-Grained Retrieval and Keyword Augmentation

Abstract

Conversational Query Reformulation (CQR) is an important component in Conversational Question Answering (ConvQA), where user queries are often incomplete, ambiguous, and dependent on previous dialogue turns. Recent CQR approaches have shown the effectiveness of large language models (LLMs) in generating standalone queries. However, rewritten queries do not always lead to optimal retrieval performance. This paper presents a retrieval-aware CQR framework that integrates fine-grained retrieval feedback and keyword augmentation to improve downstream document retrieval. Instead of treating the LLM-rewritten query as the final search query, our framework uses it to retrieve candidate documents, identify relevant paragraph-level evidence, generate a pseudo-answer, and extract semantically relevant keywords. These keywords are then filtered based on their alignment with the reformulated query and dialogue history before being incorporated into the final query. We evaluate the proposed framework on a sampled subset of the QReCC dataset using BM25 and GTR as sparse and dense retrieval backbones. The experimental results show that our proposed approach improves retrieval accuracy compared to the baseline model across all metrics such as Recall@10, MRR and NDCG. Specifically, under the BM25 sparse retrieval framework, our best-performing configuration achieves improvements in Recall@10 (81.29), MRR (56.06) and NDCG (60.92) compared to the baseline. A comparable pattern of substantial gains holds under the GTR dense retrieval framework, achieving a Recall@10 (76.51), MRR (53.26) and NDCG (57.54), indicating the effectiveness of our method across different retrieval paradigms.