Interaction-Budget-Matched Analysis of Compact Cross-Encoder Reranking in Hybrid Retrieval
Abstract
Hybrid retrieval pipelines combine lexical retrieval, semantic selection, and cross-encoder reranking, but apparent gains can be confounded when systems receive different numbers of expensive query–document interactions. This study presents an interaction-budget-matched empirical analysis across eight BEIR collections, in which the controlled quantity is the number of query–document pairs scored by the cross-encoder, not FLOPs, latency, or total system compute. Under budget-constrained leave-one-dataset-out selection, a shallow hybrid configuration reranks 80.5 candidates per query versus 100 for standalone MiniLM-L4 and improves Macro-8 nDCG@10 from 0.4607 to 0.5136, with significant gains on seven collections after Holm correction. Fewer cross-encoder interactions do not imply lower total cost: in warm-query profiling of the research implementation over 4,370 queries, mean online latency is 98.83 ms for the hybrid pipeline versus 88.94 ms for the standalone reranker, and the hybrid needs more GPU memory and embedding storage. At the same operating point, candidate pooling alone adds 0.0046 nDCG@10; removing E5, the cross-encoder, or BM25 reduces nDCG@10 by 0.0363, 0.0085, and 0.0079, respectively. Pair length strongly tracks latency but not effectiveness gain. Oracle analysis shows substantial query-level headroom, yet a learned query-time router captures only 3.3% of the oracle-over-random gap. On a common five-collection subset, higher-capacity and full-corpus dense references can outperform the compact pipeline. Careful candidate selection and multi-signal fusion can therefore substantially improve compact reranking with fewer cross-encoder interactions, but in the measured implementation this does not reduce end-to-end latency, and it implies neither equal total compute nor universal superiority over dense retrieval.