It is found that all linear performance leaders effectively add only a nuclear-norm based regularizer, or a Frobenius-norm based regularizer, or a Frobenius-norm based regularizer.
Abstract
Recently, a wide range of recommendation algorithms inspired by deep learning techniques have emerged as the performance leaders on several standard recommendation benchmarks. While these algorithms were built on different DL techniques (e.g., dropouts, autoencoder), they have similar performance and even similar cost functions. This paper studies whether the models'comparable performance are sheer coincidence, or they can be unified under a single framework. We find that all linear performance leaders effectively add only a nuclear-norm based regularizer, or a Frobenius-norm based regularizer. The former ones possess a (surprising) rigid structure that limits the models'predictive power but their solutions are low rank and have closed form. The latter ones are more expressive and more efficient for recommendation but their solutions are either full-rank or require executing hard-to-tune numeric procedures such as ADMM. Along this line of finding, we further propose two low-rank, closed-form solutions, derived from carefully generalizing Frobenius-norm based regularizers. The new solutions get the best of both nuclear-norm and Frobenius-norm world.
Experiments conducted on movie, book and electronics benchmark datasets demonstrate that GradSup outperforms iterative fine-tuning to provide scalable and personalised recommendation that is consistently sustained above the frozen LLM backbone.
Kanishka Dandeniya, C. Dasanayaka, Daswin de Silva et al.· Proceedings of the 20th ACM...· 0 citations
PALRec is proposed, a parameter-preserving augmentation framework that equips an LLM with recommendation capabilities while keeping its original parameters fixed and consistently outperforms fully fine-tuned counterparts in recommendation accuracy while preserving the LLM’s pre-trained knowledge.
Hyunsoo Na, Minseok Gang, Sang-goo Lee et al.· ACM Transactions on Informat...· 0 citations
The Fusion of Layer-wise Exits for Sequential Recommendation (FLEXRec), a discriminative framework that enhances compact LLMs while retaining scalable full-corpus ranking and achieves state-of-the-art accuracy among competing methods while remaining highly efficient.
Xurong Liang, Tong Chen, Q. Nguyen et al.· 0 citations
This work test several proposed ``state-of-the-art''solutions from the literature and find that older methods are currently far superior for general use, and finds that a new and simple baseline, using LBFGS on a sub-gradient, is highly effective with minor tweaks, despite being dismissed in the literature for theoreti...
OneTrans-V2 is presented, one Transformer that unifies the entire cascade of retrieval, pre-rank, and fine-rank and introduces Decision-Conditioned Generative Retrieval (DCGR), which predicts a decision prefix describing the upcoming interaction and generates items conditioned on it, allowing business objectives to ste...
Han-Nan Cao, Jun Guo, Hao-Lei Pei et al.· 0 citations
The gradient flow of the best rank-$r$ approximation is derived and it is pointed out that the offset of DLRT comes from a curvature-coupling term which is large and thus non-negligible under aggressive compression, and a stable dynamic low-rank training method is proposed, named SDLRT.