The design and launch of SnapLGR, an LLM-based generative retrieval system for short-video recommendation at Snapchat shows that successful production SnapLGR requires joint design across representation learning, vocabulary grounding, and efficient training and serving.
Abstract
Pretrained large language models (LLMs) are promising retrieval engines because they combine rich semantic priors, strong sequence modeling capabilities, and favorable scaling behavior. However, turning a pretrained LLM into a generative retriever in production deployment raises several challenges: the model must learn an internal item vocabulary that was absent from pretraining, and generate valid item identifiers under strict latency and cost constraints. We address these challenges through the design and launch of SnapLGR, an LLM-based generative retrieval system for short-video recommendation at Snapchat. The system is built around three main designs. First, we construct semantic identifiers (SIDs) from multimodal item embeddings and enhance them with Personalized PageRank (PPR)-based co-engagement contrastive learning, resulting in improved codebook utilization, reduced collisions, and infused collaborative signal. Second, we use continued pretraining (CPT) to ground the introduced SID tokens before supervised fine-tuning (SFT) on user interaction sequences. Third, we make SnapLGR serving practical through TensorRT-LLM CUDA-backed beam search and a decentralized worker-loop architecture. In a live A/B test, the launched system increased View Time by 0.37%, Time Spent by 0.09%, Deep Sessions by 0.18%, and Deep Sessions Unique User by 0.11% relative to the existing TIGER-style generative retrieval baseline. We then decompose this offline gap under a fixed tokenizer and quantify the gains due to model architecture, scaling, and pretraining. Overall, our deployment shows that successful production SnapLGR requires joint design across representation learning, vocabulary grounding, and efficient training and serving.
Recent advances in generative recommendation (GR) leverage large language models (LLMs) as recommender backbones, enabling LLMs to directly generate recommendations conditioned on item-interaction histories. In these systems, items are often represented through semantic IDs (SIDs) added to the LLM vocabulary as special tokens. Ideally, SIDs imbue item token representations with semantic priors, thereby improving model generalization. However, standard vocabulary expansion typically initializes these tokens as random Gaussian vectors, discarding the SIDs'underlying continuous geometry and forcing the LLM to relearn token relationships from interaction data. To demonstrate the consequences of this design, we first show that training from this initialization tends to organize SID embeddings around item popularity rather than semantics. We further show that, despite partially reducing the reliance on popularity and improving cold item performance, the computationally expensive process of continual pretraining (CPT) fails to reliably recover the original semantic geometry. To address these findings, we propose a simple, parameter-free intervention that initializes SID token embeddings directly from their corresponding centroids in the semantic embedding space. Requiring only a few lines of code and no additional training or inference overhead, this drop-in approach improves pure-SFT Recall@5 by up to 16%, reaches peak performance with up to 40% fewer SFT steps, and improves cold-item Recall@5 by up to 60%. Moreover, on datasets that benefit from additional CPT, centroid initialization reaches comparable performance while requiring half as many CPT epochs. Together, our findings show that preserving SID geometry, beyond shared-prefix structure, provides a simple and effective semantic prior for LLM-based GR.
Donald Loveland, Liam Collins, B. Kumar et al.· 0 citations
EGR is proposed, an Embedding-Native Generative Retrieval framework that uses a single shared LLM to learn item representations from item metadata and user representations from interaction histories in one embedding space, simplifying system design while improving retrieval quality and ad performance.
Xiaodong Liu, Congfei Zhang, Hsiang-wei Chao et al.· 0 citations
Large language models (LLMs) are increasingly used as backbone architectures for recommender systems because of their strong sequence modeling and representation learning capabilities. However, most LLM-based recommenders operate primarily on discrete textual tokens, whereas practical recommendation pipelines also rely on continuous numerical features and dense embedding features produced by upstream feature engineering or pretrained encoders. This mismatch limits the ability of LLM-based models to exploit fine-grained non-textual signals. We propose a soft-token fusion framework that maps numerical and embedding features into the LLM embedding space, allowing heterogeneous recommendation signals to be consumed through the standard token interface. We instantiate the framework in a shared-parameter LLM-based two-tower retrieval model and introduce an interaction-based fusion module that refines embedding and numerical soft tokens before they are inserted into the final LLM input. Experiments on three Amazon recommendation benchmarks show that soft-token fusion improves retrieval performance over LLM-based baselines, and that interaction-based fusion is more effective than direct concatenation of heterogeneous soft tokens.
Zhe Xu, Ankit Peshin, Chiyu Zhang et al.· 0 citations
Large Language Models (LLMs) have emerged as powerful assets for recommender systems. However, deploying them as generative recommenders or zero-shot rankers at web-scale remains bottlenecked by prohibitive computational overhead and grounding challenges. In this paper, we revitalize the classic, highly efficient two-tower retrieval architecture by adapting LLMs as semantic representation backbones rather than generative engines. We introduce an LLM-native two-tower framework engineered for high-throughput, large-scale retrieval. Our architecture introduces several key innovations: a shared LLM encoder for joint user-item modeling, End-Of-Sentence (EOS) token pooling for compact sequence embedding, cross-dataset transfer learning, knowledge distillation from powerful cross-encoder teachers, and latent reasoning within the user tower. Extensive evaluation across three public benchmarks demonstrates that cross-encoder architecture outperforms current state-of-the-art (SoTA) models, while the efficient two-tower student achieves SoTA-comparable retrieval performance. Furthermore, experiments on internal large-scale production systems yield substantial topline retrieval improvements along with high resilience to model staleness and superior data scaling. Our findings demonstrate that when augmented with modern representation learning, the traditional two-tower paradigm remains an exceptionally competitive and practical solution for industrial retrieval systems.
Zhe Xu, Prachi Agrawal, Kavosh Asadi et al.· 0 citations
RedGR is proposed, a generative retrieval model that unifies the modeling of multiple complex retrieval tasks and has been fully depolyed in the homefeed recommendation scenario of RedNote, serving hundreds of millions of users every day.
Mengcheng Fang, Hongyu Wang, Xichuan Niu et al.· Annual International ACM SIG...· 0 citations
Sequential recommendation aims to predict users’ next items of interest based on their historical interactions. Recently, Large Language Models (LLMs) have shown strong potential in this field due to their powerful semantic understanding. However, existing methods face two core challenges: First, LLMs may generate non-existent recommendations due to hallucinations; Second, when utilizing LLM-generated embeddings for transfer to downstream recommendation tasks in different domains, the dimensional and structural differences across spaces exacerbate space misalignment. To address these challenges, we propose UniRec, a unified space learning framework that achieves deep alignment through a hierarchical space transformation path: language space → collaborative space → real space. This framework employs a two-stage learning mechanism: The first stage guides LLMs to align semantic and collaborative spaces through collaborative instruction fine-tuning, introduces a real-space constrained generation mechanism to reduce hallucination problems, and is optimized for large-scale applications; The second stage extracts core semantic principal components through singular value decomposition and freezes the representations, combines with gating networks to adaptively fuse semantic and domain-specific collaborative signals, achieving efficient cross-domain transfer. Experiments show that UniRec achieves over 27% average improvement on in-domain datasets and 24% on out-of-domain datasets, while maintaining low memory overhead.
Jiarui Yang, Junmei Feng, Yihan Zhang et al.· ACM Transactions on Informat...· 0 citations