Skip to content
Book Open access

Generalizable Multi-Pass Training of Ads Recommendation Models with Foundation Model Guidance

Aug 2026 · Knowledge Discovery and Data Mining · pp. 4000-4011 · 0 citations · 55 references
Computer Science

TL;DR

This work proposes TeMPO, a principled framework for generalizable multi-pass training of recommendation models that leverages supervision from a teacher recommendation Foundation Model, and proposes a two-stage optimization strategy to stabilize multi-pass learning.

Abstract

High-capacity Click-Through Rate (CTR) models for ads recommendation often exhibit pronounced one-epoch overfitting: performance peaks after a single training pass (epoch) over the data, while additional epochs degrade generalization as the model memorizes noise in high-variance click labels. To address this challenge, we propose TeMPO, a principled framework for generalizable multi-pass training of recommendation models that leverages supervision from a teacher recommendation Foundation Model (FM). Our key idea is to move beyond maximizing the conditional likelihood of noisy click labels. Instead, we maximize the joint likelihood of observing both click labels and the teacher's rich representational knowledge, combining the task loss with teacher-guided alignment objectives. We propose a two-stage optimization strategy to stabilize multi-pass learning. Our theoretical analysis shows that distillation acts as complexity regularization, yielding tighter generalization bounds than standard empirical risk minimization under noisy labels. Extensive experiments on public benchmarks and Meta's production-scale ads application demonstrate that TeMPO enables additional training passes to improve, rather than degrade, performance.

Read PDF

Similar papers

Preprint Oct 2026

Improving Ads Ranking Models with Auxiliary Supervision: Techniques and Insights

Accurate prediction of post-click Conversion Rate (CVR) is essential for personalized ad delivery in the advertising industry. Traditional ads ranking models rely on conversion events (e.g., a user purchasing an item) as ground truth labels for training. However, these conversion events are often sparse or missing, whi...

Nikhil Bhale, Apurva Pathak, SaiSujithKumarReddy MarapaReddy et al. · 0 citations
#artificial intelligence Open access Sep 2026

COPE: Continual Personalization of LLMs under Sparse User Feedback via User Embeddings and Self-Evaluation

Experiments show that COPE consistently outperforms strong training-free and training-based baselines under sparse feedback, and remains complementary to Retrieval-Augmented Prompting, and further analyses confirm COPE's reliable self-evaluation, meaningful preference patterns, stable general capabilities, and robustne...

Rui-Ke Cao, Fu-Gen Yao, Liang Dong et al. · 0 citations
Book Open access Sep 2026

Uncertainty-Aware Reward Modeling: A Large-Scale Case Study in Video Recommendation

Historically, recommendation systems have focused on maximizing precision by treating user preference as a static, predictable target. However, this approach ignores both the inherent randomness of human behavior and the model’s own varying levels of confidence. To compensate, many industrial systems rely on "reserved...

Nitu Sharaff, Brian Y. C. Leung, Shawn Andrews et al. · 0 citations
Preprint Oct 2026

LoopGR: Parameter-Efficient Scaling for Generative Recommendation via Recurrent Depth

Generative recommendation formulates item recommendation as autoregressive generation rather than a discriminative scoring problem. Although empirical studies exhibit favorable scaling behavior in recommendation, unleashing the benefits of scaling remains challenging under the strict model size and inference cost const...

Hui-Zhe Zhang, Yu-Dong Li, Liang Chen et al. · 0 citations
Open access Oct 2026

MIDA: Multi-Interest Decoding with Adaptive Aggregation for Session-based Recommendation

Session-based recommendation (SBR) infers user interest from limited interaction sequences, yet current models show a consistent performance drop when longer histories are available. This degradation is associated with decoder designs that overemphasize recent items while underutilizing distant, yet semantically releva...

Lin-Jiang Guo, N. Bisht, Shi-Qing Wu et al. · 0 citations
Book Open access Nov 2026

FAF: A Feature-Aware Adaptive Pre-training Framework for Bridging Gaps in CTR Prediction

Self-supervised pre-training has emerged as an effective solution to data sparsity in Click-Through Rate (CTR) prediction. However, existing paradigms still face two critical limitations: (1) task degeneration in pre-training, where heuristic or random target construction may lead to trivial shortcuts; and (2) limited...

Kai Cheng, Hao Wang, Ke-Fan Wang et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.