Skip to content
#small language model Book Open access

PROMISE: Process Reward Models for Unlocking Test-Time Scaling Laws in Generative Recommendations

Sep 2026 · Proceedings of the 20th ACM Conference on Recommender Systems · 0 citations · 19 references

TL;DR

This work proposes Promise, a novel framework that integrates dense, step-by-step verification into generative models, and unlocks Test-Time Scaling Laws in recommender systems, demonstrating that by increasing inference compute, smaller models can match or surpass larger models.

Abstract

Generative Recommendation has emerged as a promising paradigm, reformulating recommendation as a sequence-to-sequence generation task over hierarchical Semantic IDs. However, current approaches face a severe challenge that we define as Semantic Drift, where errors in early, high-level tokens irreversibly divert the generation trajectory into irrelevant semantic subspaces. Inspired by Process Reward Models (PRMs) that enhance reasoning in Large Language Models, we propose Promise, a novel framework that integrates dense, step-by-step verification into generative models. Our framework utilizes a lightweight PRM to assess the quality of intermediate inference steps, and a PRM-guided beam search strategy that leverages dense feedback to dynamically prune erroneous branches. Most importantly, this method unlocks Test-Time Scaling Laws in recommender systems, demonstrating that by increasing inference compute, smaller models can match or surpass larger models. Extensive offline experiments and online A/B tests on a large-scale platform demonstrate that Promise effectively mitigates Semantic Drift, significantly improving recommendation accuracy while enabling efficient deployment.

Read PDF

Similar papers

#artificial intelligence Preprint Sep 2026

FLARE: A Full-Lifecycle Dense Supervision Paradigm for Long-Horizon Coding Agents via Generative Reward Model

While test-time scaling enhances Large Language Model (LLM) agents in long-horizon software engineering (SWE), sparse binary rewards (Pass/Fail) create a severe credit assignment crisis and waste failed exploratory trajectories. Current trajectory optimization and scaling methods are costly and structurally limited, re...

Jing-Xuan Xu, Gang Wu, Ya-Nan Wu et al. · 0 citations
#machine learning Preprint Sep 2026

Sampling via Decision-Flow: Training-Free Extraction of Improved Latent Reasoning Paths in Large Language Models

Decision-Flow Sampling (DF-Sample), a training-free, data-free inference-time framework that constructs a hierarchical reasoning tree, scores terminal nodes for quality, and back-propagates utilities to inform each intermediate branching decision, indicating substantial latent reasoning potential in pretrained base mod...

Zhen-Dong Mi, Shao-Yi Huang · 0 citations
#artificial intelligence Preprint Sep 2026

Beyond Verified Answers: Solver-Informed Self-Distillation for Bootstrapping Operations Research Language Models

SOLID is proposed, a novel framework for self-improving OR language models without verified answers or external evaluators that improves solution accuracy for both general-purpose and OR-tuned models over outcome-only group-relative training.

Rui-Chen Zhu, Ming-Long Cao, Chen-Yu Zhou et al. · 0 citations
#machine learning Preprint Sep 2026

Rethinking Personalized Generation: Test-Time Alignment via Factorized Ranking Models

This work uses empirical studies to reveal the existence of a massive, untapped performance headroom for personalized generation through test-time alignment, and proposes a parameter-efficient framework utilizing million-parameter scale multi-layer perceptron (MLP) ranking models.

Qi-Yao Ma, Jun-Shan Zhang, Zhe Zhao · 0 citations

GRPO Training Dynamics for Small Language Models

Group Relative Policy Optimization (GRPO) has emerged as a memory-efficient reinforcement fine-tuning (RFT) technique for reasoning-intensive tasks. How- ever, GRPO training dynamics on small language models (SLMs) remain poorly understood, limiting its reliable adoption and reproducibility in open and resource- constr...

Rajat Ghosh, Vaishnavi Bhargava, Henry Wong et al. · 0 citations
Conference Open access Sep 2026

A Survey of Joint Online-Offline Fine-tuning for Large Language Models

This work advocates for Joint Online-Offline Fine-Tuning as a superior paradigm that breaks the convention of restricting offline data to SFT and online data to RFT, and provides the first comprehensive survey focusing specifically on the synchronization of data provenance.

Tai-Hang Zhen, Guang Yang, Chen-Zhang Li et al. · 0 citations

Related blog posts

Microsoft Research Blog Nov 16, 2026

TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient

AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.