APR: Adaptive Personalised Reranking For Conversational Search
TL;DR
Adaptive Personalised Reranking (APR) is proposed, a framework that routes queries based on intent that uses efficient similarity-based reranking for simple queries and dynamically generates tailored instructions to guide an instruction-following reranker for constraint-heavy contexts.
Abstract
Conversational search systems help users satisfy complex information needs through natural language interactions, yet incorporating user preferences into ranking remains challenging. Existing rewrite-then-rerank pipelines capture topical relevance but struggle with fine-grained constraints such as negative preferences or formatting requirements. Instruction-following retrieval approaches are promising for enforcing such constraints, yet their use in personalised conversational search remains underexplored, since ''instructions'' within this context are implicit and situated within user history and profiles, rather than being explicitly stated. We show that instruction-following models can assist with complex queries but introduce noise and latency on simpler keyword queries. To address this issue, we propose Adaptive Personalised Reranking (APR), a framework that routes queries based on intent. APR uses efficient similarity-based reranking for simple queries and dynamically generates tailored instructions to guide an instruction-following reranker for constraint-heavy contexts. Oracle analysis on TREC iKAT 2023 and 2024 shows that instruction-following provides a ''rescue'' potential for hard queries. We also show that APR trained with synthetic data performs competitively against strong baselines such as MonoT5 while offering promising new research avenues.