Skip to content

COPE: Continual Personalization of LLMs under Sparse User Feedback via User Embeddings and Self-Evaluation

Sep 2026 · Conference on Empirical Methods in Natural Language Processing · Vol abs/2609.26853 · 0 citations · 35 references
Computer Science

TL;DR

Experiments show that COPE consistently outperforms strong training-free and training-based baselines under sparse feedback, and remains complementary to Retrieval-Augmented Prompting, and further analyses confirm COPE's reliable self-evaluation, meaningful preference patterns, stable general capabilities, and robustness under shifting preferences and alternative evaluators.

Abstract

While Large Language Models (LLMs) have achieved remarkable results across various benchmarks, their alignment with normative values often results in homogenized responses that fail to address diverse user preferences. Existing training-free methods often occupy valuable context windows through prompt engineering, while training-based methods typically remain static post-training, failing to support the continual optimization required in real-world settings. To address these challenges, we propose COPE (Continual Optimization with Personalized embedding and self-Evaluation), a novel optimization framework tailored for real-world-motivated interaction settings with sparse user feedback. Our framework assigns learnable personalized embeddings to each user and synergistically integrates preference capture, self-evaluation calibration, and personalized response optimization within a single update step. A key innovation of our method is the use of self-evaluation to generate proxy rewards, enabling continuous model updates even when explicit user feedback is unavailable. Experiments show that COPE consistently outperforms strong training-free and training-based baselines under sparse feedback, and remains complementary to Retrieval-Augmented Prompting (RAP). Further analyses confirm COPE's reliable self-evaluation, meaningful preference patterns, stable general capabilities, and robustness under shifting preferences and alternative evaluators.

Read PDF

Similar papers

#artificial intelligence Preprint Aug 2026

Hypotheses-Guided Self Distillation for Continual Personalization

HypReflect is introduced, a reliable, scalable framework for continual personalization that infers explicit, uncertainty-aware preference hypotheses from diverse user signals, reflectively refines them as new evidence accumulates, and incorporates the resulting user model through hypotheses-guided self-distillation.

Eunjeong Hwang, Kushan Mitra, Dan Zhang et al. · 1 citation
#natural language process... Preprint Sep 2026

HyperTrace: Hypothesis-Based Preference Tracing for Online LLM Personalization

Experiments on PRISM and PersonaMem-v2 show that HyperTrace improves response alignment, preference prediction, and profile consistency over strong online baselines, demonstrating the effectiveness of tracing latent user preferences for robust personalization.

Jian-Zhi Shen, Ke-Yu Mao, Ming-Hao Shao et al. · 1 citation
Preprint Aug 2026

Progressive Content Refinement with Decaying Reward Joint LinUCB

A novel contextual bandit algorithm that explicitly incorporates reward decay modeling that achieves significant performance gains over strong baselines and confirms that the integration of reward decay modeling within the bandit framework is crucial for mitigating over-exploitation and optimizing the iterative refinem...

Shion Ishikawa, Pablo Loyola, Young-joo Chung et al. · 0 citations
Book Open access Aug 2026

Generalizable Multi-Pass Training of Ads Recommendation Models with Foundation Model Guidance

This work proposes TeMPO, a principled framework for generalizable multi-pass training of recommendation models that leverages supervision from a teacher recommendation Foundation Model, and proposes a two-stage optimization strategy to stabilize multi-pass learning.

Yun-Zhe Qi, Qinghai Zhou, Bo-Yang Liu et al. · 0 citations
#artificial intelligence Preprint Sep 2026

PLUME: Parameter-Efficient Personalization of Large Language Models via Low-Rank User Modulation in Shared Subspaces

PLUME (Personalized Low-Rank Adaptation through User Modulation and Shared Subspace), a lightweight framework that achieves efficient and expressive per-user adaptation by leveraging a shared task-specific subspace, is proposed as a scalable and semantically grounded approach to LLM personalization.

Xin-Yu Li, Hao Zhou, Jian-Feng Zhu et al. · 0 citations
#large language models Review Open access Sep 2026

PALRec: Large Language Model-Based Sequential Recommendation with Parameter-Preserving Augmentation

PALRec is proposed, a parameter-preserving augmentation framework that equips an LLM with recommendation capabilities while keeping its original parameters fixed and consistently outperforms fully fine-tuned counterparts in recommendation accuracy while preserving the LLM’s pre-trained knowledge.

Hyunsoo Na, Minseok Gang, Sang-goo Lee et al. · 0 citations

Related blog posts

MIT News · Artificial Intelligence Sep 29, 2026

Who we become when we talk to machines

Professor Sherry Turkle’s new book, “Artificial Intimacy,” offers a withering critique of chatbots and the antisocial dynamics she believes they encourage.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.