Generating Personalized Images for Sparse-Interaction Users with Uncertainty-Aware Retrieval and Dense Knowledge Guidance

Abstract

Personalized image generation aims to synthesize target images tailored to individual preferences based on users’ historical interaction data. Existing methods typically inject features from historical interaction records to guide personalized generation. However, such methods encounter two critical challenges when serving sparse-interaction users: (1) Preference Misalignment: Sparse interactions tend to lack target-semantic preference information, causing the direct injection of interaction features to misalign with users’ desired target styles. (2) Lack of Reliable Supervision: Mining preferences from sparse interactions requires sufficient supervision signals, yet personalized generation with sparse data inherently lacks direct feedback or ample preference signals for generated outputs. To address these challenges, we propose Uncertainty-aware retrieval with Dense guidance for Sparse personalized Image Generation (UDSIG). To our knowledge, it is the first effective framework tailored for sparse personalized image generation. For preference alignment, we retrieve reference images from the entire dataset that exhibit low-uncertainty preference matching under each user’s sparse interaction history. For reliable supervision, we propose a dense-to-sparse scheme that incorporates a reward model derived from active users’ dense interaction data to drive personalized generation in sparse scenarios. Extensive experiments and human evaluations across three public datasets confirm the superiority of UDSIG, along with its strong generalization to dense interaction scenarios.