From Queries to Playlists: An LLM-Driven Architecture for Semantic Music Search at Scale
A core task for music streaming platforms is retrieving and ranking tracks in response to user queries over multi-million-track catalogs. Existing approaches either rely on tag-based, entity-centric retrieval and recommendation, which struggle with implicit and subjective queries that fall outside a predefined tag vocabulary, or on recent LLM-based generative methods that circumvent this limitation but are prone to hallucinations and factual errors. We introduce a semantic playlist generation service that retrieves and ranks tracks based on meaning rather than keyword overlap, while avoiding hallucination-related failures. Each track is represented as structured text combining metadata, lyrics, and descriptive attributes, and both user queries and track representations are encoded into a shared embedding space using an LLM. A cross-encoder reranker built on the same backbone refines candidate ranking, and its signals are distilled into the embedder to reduce serving cost. In offline and production evaluations, our semantic vector-search pipeline achieves the highest playlist quality, improving Precision@10 from 64% with faceted search and 74% with direct LLM generation to 81%, while remaining compatible with low-latency, large-scale deployment. In an online A/B test on smart-speaker traffic, routing a share of playlist requests to our system yields a consistent double-digit relative uplift in Average Time Spent, indicating that meaning-aware retrieval substantially enhances user engagement and supports broader production rollout.