Text embeddings from retrieval-tuned (dual-encoder) models are increasingly used as context features in contextual bandits for recommendation, on the assumption that an embedding space optimized for inner-product similarity will speed up a linear exploration policy. This study tests that assumption with a controlled, shared-encoder design: the same BERT-base model in two forms, vanilla (mean-pooled) and retrieval-fine-tuned (MS MARCO dot-product), used as frozen bandit context. Experiments span three datasets (MIND, MovieLens-1M, Amazon CDs and Vinyl), two linear policies (LinUCB and linear Thompson sampling), and two dimensionality-reduction methods (PCA and random projection), over 20 seeds. Neither the training objective nor the reduction method determines performance on its own; cumulative regret is governed by their interaction, which is dataset-dependent. On MovieLens the reduction reverses the encoder ranking (PCA favors the retrieval-tuned encoder, random projection the vanilla one), with large effects in both directions, while on MIND the encoders are close. The reduction method can be the larger lever, moving up to 34% of the learnable margin. A ridge-regression probe on the candidate contexts screens the reduction choice offline, without running the bandit. For linear exploration policies, the encoder and the reduction should be treated as a joint choice rather than assuming retrieval-tuned embeddings are universally preferable.
A unified pipeline deployed at Walmart that addresses both signal quality and model evolution is presented, and a Warm-Start Distillation technique that transfers domain-specific expertise from the legacy model to the new backbone is introduced.
Zhen Yang, Juexin Lin, Hongwei Shang et al.· Annual International ACM SIG...· 1 citation
This work introduces Giga-Embeddings, a family of text embedding models designed to combine strong retrieval quality with efficient serving, and trains the compact model using a dimension-agnostic objective that aligns teacher and student similarity distributions.
Egor Kolodin, Egor Krasnoperov, Evgeniy Kosarev et al.· 0 citations
Recent works have highlighted the expressive limitations of embedding based retrieval models through both theoretical analyses and challenging benchmarks such as LIMIT. While multi-vector embeddings consistently outperform single-vector embeddings, the precise representational gap between them remains poorly understood. In this work, following Jayaram's work, we provide the first explicit family of query and document sets, together with their relevance matrices, for which single-vector embeddings that rank all relevant documents above irrelevant ones require exponential size, whereas polynomial-size multi-vector embeddings suffice. Our result establishes an exponential separation between the expressive power of single-vector and multi-vector embeddings for the task of ranking of documents as opposed to approximating numerical scores as in the work of Jayaram. Motivated by our theoretical construction, we introduce ANDOR, a new retrieval benchmark that naturally instantiates these hard examples. We show that state-of-the-art single-vector embedding models perform poorly on ANDOR in the zero-shot setting and exhibit only marginal improvements after fine-tuning, highlighting the inherent difficulty of the benchmark compared to prior work. In contrast, multi-vector models consistently outperform their single-vector counterparts and improve substantially with fine-tuning, closely aligning with our theoretical predictions.
Mihir Agarwal, Viraj Agrawal, Sabyasachi Basu et al.· 0 citations
DESS is introduced, a lightweight uncertainty layer that augments an existing embedding model with a predicted mean vector and an independent per-dimension spread vector that provides a modular, geometry-aware uncertainty layer for embedding-space models, provided its spread is calibrated to local embedding geometry.
Morten Grundetjern, J. Voigt, Per-Arne Andersen et al.· KI - Künstliche Intelligenz· 0 citations
Cross-domain recommendation is a well-known technique for improving recommendations in a target domain, especially under sparse data or cold-start conditions. A common strategy is to train user embeddings separately in the source and target domains and learn a transfer function between them. In contrast, we propose SetCDR, which constructs more effective user representations in the target domain by directly incorporating each user's source and target history. We additionally introduce a lightweight domain indicator that preserves data–domain relational information. These histories, composed of item–rating pairs, are represented as variable-length sets and processed using a permutation-invariant neural architecture. This differs from conventional neural networks, which do not naturally handle unordered inputs, and is well suited to recommender systems where user history sizes vary greatly among users. The use of a permutation-invariant architecture ensures consistent embeddings regardless of input order, improving robustness to real-world variability and training efficiency. We demonstrate SetCDR in two forms: a simple sum-pooling method and an extended multihead attention–based method that captures more complex dependencies within user histories. Moreover, because SetCDR operates directly on sets of user history records, it provides a natural way to examine how histories influence user representations. Finally, SetCDR adapts immediately to new interactions without additional retraining, enabling on-the-fly performance improvement. Experimental results across multiple cross-domain benchmarks confirm that SetCDR consistently outperforms strong baselines in recommendation quality.
K. Kim, James Geraci· Annual International ACM SIG...· 0 citations