Skip to content
Book Open access

FAF: A Feature-Aware Adaptive Pre-training Framework for Bridging Gaps in CTR Prediction

Nov 2026 · Proceedings of the 35th ACM International Conference on Information and Knowledge Management · 0 citations · 54 references

Abstract

Self-supervised pre-training has emerged as an effective solution to data sparsity in Click-Through Rate (CTR) prediction. However, existing paradigms still face two critical limitations: (1) task degeneration in pre-training, where heuristic or random target construction may lead to trivial shortcuts; and (2) limited adaptivity in fine-tuning, where static transfer strategies fail to accommodate optimization heterogeneity across diverse feature fields and ID frequencies. To address these issues, we propose the Feature-aware Adaptive pre-training and fine-tuning Framework (FAF). The core of FAF lies in two coupled designs. First, we improve pre-training target construction from both token and feature levels. At the token level, we introduce a position-decayed future item prediction task to incorporate additional future behavior supervision while reducing the influence of noisy distant targets. At the feature level, we formulate masked feature selection as a combinatorial optimization problem and derive an entropy-redundancy scoring criterion, which selects informative target fields with low contextual redundancy to construct non-trivial reconstruction tasks. Second, we design an adaptive fine-tuning strategy that combines field-aware and item frequency-aware update regulation. This strategy dynamically adjusts embedding update magnitudes across heterogeneous feature fields and item frequencies, improving the compatibility between pre-trained representations and downstream task adaptation. Extensive experiments on three real-world datasets show that FAF brings consistent overall performance gains over strong baselines across multiple backbone models, demonstrating its effectiveness in representation learning and compatible knowledge transfer.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.