This work proposes LaRec, an efficient generative recommendation framework designed to unleash the potential of latent reasoning in LLMs by designing Latent Pre-training that empowers LLMs with latent reasoning capabilities by providing rich supervisory signals to the latent space reasoning via step-level alignment and process direction alignment.
Abstract
Large Language Models (LLMs) have shown great promise in recommendation due to superior reasoning abilities. However, existing methods mainly rely on explicit Chain-of-Thought (CoT), resulting in verbose reasoning texts and inefficient response times. latent reasoning aims to balance efficiency by thinking within a continuous latent space, yet it faces two major challenges: (1) Lack of Fine-grained Supervision: Latent reasoning relies solely on feedback from the final labels, providing sparse supervisory signals that struggle to effectively guide the optimization of multiple hidden reasoning steps. (2) Single Reasoning Path: The deterministic nature of latent reasoning impedes the exploration of users'diverse interests and preferences, thereby limiting the recommendation capabilities of LLMs. To address these issues, we propose \textbf{$LaRec$}, an efficient generative recommendation framework designed to unleash the potential of latent reasoning in LLMs. $LaRec$ consists of two core stages: First, we design Latent Pre-training that empowers LLMs with latent reasoning capabilities by providing rich supervisory signals to the latent space reasoning via step-level alignment and process direction alignment. Second, we introduce Personalized RL-tuning. Specifically, we construct a personalized Gaussian Mixture Distribution for each user based on their historical interests. By randomly sampling distinct reasoning starting points from this distribution during training, we guide the LLMs to traverse diverse reasoning paths within the latent space, enabling efficient exploration of user's multi-faceted interests. Experiments on multiple datasets show that $LaRec$ significantly outperforms existing baselines with comparable efficiency.
WhisperRec compresses teacher-generated CoT into learnable latent reasoning tokens, enabling a Latent-Reason-then-Answer paradigm that performs reasoning in latent space without producing verbose rationales, and achieves over 10x higher online inference throughput.
Hao Jiang, Pei Du, Pengfei Yao et al.· 0 citations
Large Language Models (LLMs) have shown strong potential for recommendation by leveraging their semantic understanding and contextual modeling capabilities. Recent studies further introduce reasoning mechanisms to improve user preference modeling. However, explicit natural-language reasoning incurs substantial inference overhead, whereas existing latent reasoning methods mainly focus on generating or verifying intermediate states, leaving their layer-wise preference roles and contributions insufficiently characterized. We propose HiLaR, a Hierarchical Latent Reasoning framework with layer-aware reinforcement optimization for LLM-based recommendation. HiLaR constructs temporal-guided hierarchical user preference representations, aligns them with multiple LLM latent reasoning states, and organizes the reasoning process from broad preferences to fine-grained current intents. To further optimize the reasoning trajectory, HiLaR combines final recommendation feedback with layer-aware process rewards derived from the marginal target-likelihood gain of each state. Experiments on four Amazon benchmark datasets show that HiLaR generally outperforms strong sequential, generative, and LLM-based recommendation baselines. Ablation and sensitivity analyses further verify the contribution of hierarchical representation learning, latent alignment, and process-level optimization. Our code is available in https://github.com/hupeiyu21/HiLaR.
Large Language Models (LLMs) demonstrate significant potential in sequential recommendation, and leveraging their Chain-of-Thought (CoT) reasoning capabilities can further unlock profound user preference understanding. However, deploying explicit CoT reasoning in real-world systems faces prohibitive challenges: (i) the conflict between the large model scale required for high-fidelity reasoning and the resource constraints of online services, and (ii) the excessive latency introduced by auto-regressive rationale generation. To address these issues, we propose I Reasoning via Multi-Teacher Distillation (IRMD), a novel framework that 'compiles' the reasoning abilities of large teacher LLMs into a lightweight student Small Language Model (SLM). IRMD first employs a Multi-Teacher CoT Synthesis with Dual-Constraint Rejection Sampling module to generate a high-quality, diverse set of reasoning paths. Subsequently, our Annealing-Scheduled Reasoning Distillation strategy progressively trains the student to internalize this logic, transitioning from mimicking explicit CoT to performing purely implicit reasoning. Extensive experiments on multiple benchmark datasets demonstrate that IRMD significantly outperforms state-of-the-art baselines in both recommendation accuracy and inference efficiency. Our code is accessible at https://github.com/Cxx-0/IRMD.
Large Language Models (LLMs) have demonstrated strong potential in recommendations due to their powerful reasoning capabilities. However, existing methods typically rely on a single reasoning path to drive the entire Top-K recommendations. This paradigm is prone to reasoning path collapse, where limiting exploration of potentially superior and diverse reasoning paths within the LLMs space. As a result, both the accuracy and diversity of the recommendation outcomes are constrained. To address this issue, we propose a novel model, Divergent Reasoning for LLM-based Recommendation, named DivReason. Inspired by the structure of intellect theory, which emphasizes a two-stage cognitive process of divergent thinking followed by convergent thinking, DivReason is designed with two core components: the Divergent Reasoning Path Generation Module and the Reasoning Path Aggregation Module. In the first module, DivReason introduces a training-free form of controlled uncertainty to promote diverse reasoning, leveraging Monte Carlo Dropout and Directional Perturbation to expand exploration in the latent reasoning space. In the Reasoning Path Aggregation Module, we adaptively select a subset of high-quality reasoning paths from the entire path pool and aggregate them into a unified reasoning representation. Meanwhile, we further adopt an alternating reinforcement learning strategy to optimize the model, explicitly balancing accuracy and diversity during training. Extensive experimental results show that DivReason effectively mitigates the issue of reasoning path collapse, while improving both the accuracy and diversity of LLM-based recommendations.
Guojia An, Jie Zou, Yuhan Yang et al.· Annual International ACM SIG...· 1 citation
The design and launch of SnapLGR, an LLM-based generative retrieval system for short-video recommendation at Snapchat shows that successful production SnapLGR requires joint design across representation learning, vocabulary grounding, and efficient training and serving.
Liam Collins, Jiwen Ren, Donald Loveland et al.· 0 citations
The emergence of Large Language Models (LLMs) has driven a paradigm shift in sequential recommendation from discriminative ranking to generative modeling. However, existing generative approaches predominantly rely on semantic IDs (SIDs)—discrete identifiers derived from hierarchical quantization that function as semantic black boxes. Solely designed for in-domain reconstruction, these opaque tokens preclude the mining of explicit user behavioral logic. In contrast, plain-text modeling unlocks the full inferential potential of LLMs, enabling superior generalization and precise, logic-driven interest discovery. To bridge this gap, we propose GUIDER (Generative User Interest Discovery & Explicit Reasoning), a framework that fundamentally decouples intent planning from item matching by reformulating sequential modeling within a rigorous closed-set semantic interest space. GUIDER first internalizes domain-specific semantics via continual pre-training and activates explicit reasoning through a hindsight-guided chain-of-thought strategy. Crucially, to ensure precise optimization of the generated interest set, we introduce Token-Adaptive Policy Optimization (TAPO). Addressing the credit assignment problem—where standard reinforcement learning indiscriminately assigns identical advantages to the entire response regardless of local correctness—TAPO employs a token-level advantage adaptation mechanism. This strategy explicitly differentiates valid interest circles from hallucinations by dynamically redistributing advantages based on fine-grained quality, while a dynamic thinking penalty suppresses redundant reasoning. Extensive offline evaluations and large-scale online A/B testing on the Kuaishou platform validate that GUIDER not only achieves superior accuracy but also yields consistent gains in average app duration and effectively facilitates out-of-cocoon interest exploration.
Jinke Wu, Yinghao Wu, Shuchang Liu et al.· Proceedings of the 32nd ACM...· 0 citations