Skip to content

On the Regularization Landscape for the Linear Recommendation Models

Sep 2026 · 0 citations · 39 references
Computer Science

TL;DR

It is found that all linear performance leaders effectively add only a nuclear-norm based regularizer, or a Frobenius-norm based regularizer, or a Frobenius-norm based regularizer.

Abstract

Recently, a wide range of recommendation algorithms inspired by deep learning techniques have emerged as the performance leaders on several standard recommendation benchmarks. While these algorithms were built on different DL techniques (e.g., dropouts, autoencoder), they have similar performance and even similar cost functions. This paper studies whether the models'comparable performance are sheer coincidence, or they can be unified under a single framework. We find that all linear performance leaders effectively add only a nuclear-norm based regularizer, or a Frobenius-norm based regularizer. The former ones possess a (surprising) rigid structure that limits the models'predictive power but their solutions are low rank and have closed form. The latter ones are more expressive and more efficient for recommendation but their solutions are either full-rank or require executing hard-to-tune numeric procedures such as ADMM. Along this line of finding, we further propose two low-rank, closed-form solutions, derived from carefully generalizing Frobenius-norm based regularizers. The new solutions get the best of both nuclear-norm and Frobenius-norm world.

View source

Similar papers

Book Open access Sep 2026

GradSup: Gradient Superposition for Personalised and Scalable LLM Recommendation

Experiments conducted on movie, book and electronics benchmark datasets demonstrate that GradSup outperforms iterative fine-tuning to provide scalable and personalised recommendation that is consistently sustained above the frozen LLM backbone.

Kanishka Dandeniya, C. Dasanayaka, Daswin de Silva et al. · 0 citations
#large language models Review Open access Sep 2026

PALRec: Large Language Model-Based Sequential Recommendation with Parameter-Preserving Augmentation

PALRec is proposed, a parameter-preserving augmentation framework that equips an LLM with recommendation capabilities while keeping its original parameters fixed and consistently outperforms fully fine-tuned counterparts in recommendation accuracy while preserving the LLM’s pre-trained knowledge.

Hyunsoo Na, Minseok Gang, Sang-goo Lee et al. · 0 citations
Preprint Aug 2026

Empowering Compact LLMs with Fusion of Layer-wise Exits for Recommendation

The Fusion of Layer-wise Exits for Sequential Recommendation (FLEXRec), a discriminative framework that enhances compact LLMs while retaining scalable full-corpus ranking and achieves state-of-the-art accuracy among competing methods while remaining highly efficient.

Xurong Liang, Tong Chen, Q. Nguyen et al. · 0 citations
#machine learning Preprint Sep 2026

Simpler Methods Work Better for L1 Penalized Logistic Models and Large Datasets

This work test several proposed ``state-of-the-art''solutions from the literature and find that older methods are currently far superior for general use, and finds that a new and simple baseline, using LBFGS on a sub-gradient, is highly effective with minor tweaks, despite being dismissed in the literature for theoreti...

Edward Raff, James Holt · 0 citations
Preprint Sep 2026

OneTrans-V2: Unifying Retrieval, Pre-rank, and Fine-rank with One Transformer in Industrial Recommender

OneTrans-V2 is presented, one Transformer that unifies the entire cascade of retrieval, pre-rank, and fine-rank and introduces Decision-Conditioned Generative Retrieval (DCGR), which predicts a decision prefix describing the upcoming interaction and generates items conditioned on it, allowing business objectives to ste...

Han-Nan Cao, Jun Guo, Hao-Lei Pei et al. · 0 citations

Stabilizing the Dynamic Low-Rank Training

The gradient flow of the best rank-$r$ approximation is derived and it is pointed out that the offset of DLRT comes from a curvature-coupling term which is large and thus non-negligible under aggressive compression, and a stable dynamic low-rank training method is proposed, named SDLRT.

Zhong-Han Xu, Ling Wang, Jun-Hao Chen et al. · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.