Do We Really Need LLMs to Augment All? A Selective Augmentation Framework with Lightweight Language Models for Multimodal CTR Prediction

Abstract

In recent years, multimodal models and large language models (LLMs) have been increasingly applied to click-through rate (CTR) prediction, owing to their ability to extract recommendation-relevant information from raw content and thereby alleviate the long-standing issue of collaborative signal sparsity. Existing paradigms typically either feed continuous embeddings from pretrained multimodal encoders into CTR models, or employ LLMs for unified knowledge augmentation. While these approaches have demonstrated promising performance, when and why their gains emerge remains unclear, and the inference cost of large LLMs introduces severe latency and scalability challenges in large-scale deployment. In this work, we empirically show that the gains of LLM-augmented CTR are highly item-dependent: uniformly applying stronger augmentation to all items is often inefficient, while the main benefits concentrate on a subset of items with more complex interaction patterns. Motivated by this observation, we propose A Selective Augmentation Framework with Lightweight Language Models for Multimodal CTR Prediction, dubbed SALM, which revisits the data augmentation paradigm and replaces indiscriminate full-data augmentation with a difficulty-aware selective strategy targeting necessary items. Extensive experiments on real-world multimodal datasets, across multiple CTR backbones and augmentation baselines, demonstrate that SALM consistently improves AUC and reduces LogLoss, while simultaneously reducing LLM-based augmentation cost1