Frozen Embeddings, LLM Retrieval Lanes, Critic-Guided Replies: Conversational Music Recommendation at an Information-Limited Ceiling

Abstract

Team volart’s submission to the RecSys 2026 Music-CRS challenge is a three-stage pipeline that needs no local GPU (every learned component trains on CPU or calls a hosted API): multi-lane retrieval fused by Reciprocal Rank Fusion, a LambdaMART reranker, and a critic-guided response stage. It placed 2nd overall and 1st in the Industry track on the final leaderboard. Our central finding is that, under every signal and selector we tested, clean ranking on this benchmark is information-limited, not model-limited: each dialogue is grounded in one real listening session, and even given that session’s exact 16–32-track co-listened pool, a purpose-trained cross-encoder ranks the gold #1 only 9% of the time. Across 87 logged experiments, retrieval, reranking, fusion, encoder choice, and hyperparameters all saturate; gains instead come from the response stage, worth 40% of the score through the judge and lexical-diversity terms; the judge rates replies independently of track correctness. We contribute the information-limit diagnosis and its mechanism, a reproducible pipeline, and evidence that held-out gains routinely fail to transfer, caught by paired significance tests and blind re-tests.