FAF: A Feature-Aware Adaptive Pre-training Framework for Bridging Gaps in CTR Prediction
Abstract
Self-supervised pre-training has emerged as an effective solution to data sparsity in Click-Through Rate (CTR) prediction. However, existing paradigms still face two critical limitations: (1) task degeneration in pre-training, where heuristic or random target construction may lead to trivial shortcuts; and (2) limited adaptivity in fine-tuning, where static transfer strategies fail to accommodate optimization heterogeneity across diverse feature fields and ID frequencies. To address these issues, we propose the Feature-aware Adaptive pre-training and fine-tuning Framework (FAF). The core of FAF lies in two coupled designs. First, we improve pre-training target construction from both token and feature levels. At the token level, we introduce a position-decayed future item prediction task to incorporate additional future behavior supervision while reducing the influence of noisy distant targets. At the feature level, we formulate masked feature selection as a combinatorial optimization problem and derive an entropy-redundancy scoring criterion, which selects informative target fields with low contextual redundancy to construct non-trivial reconstruction tasks. Second, we design an adaptive fine-tuning strategy that combines field-aware and item frequency-aware update regulation. This strategy dynamically adjusts embedding update magnitudes across heterogeneous feature fields and item frequencies, improving the compatibility between pre-trained representations and downstream task adaptation. Extensive experiments on three real-world datasets show that FAF brings consistent overall performance gains over strong baselines across multiple backbone models, demonstrating its effectiveness in representation learning and compatible knowledge transfer.