DPEO: Dynamic Preference Evolution Optimization for Self-Evolving CTR Prediction
TL;DR
This paper proposes DPEO (Dynamic Preference Evolution Optimization), a co-evolutionary framework that transforms CTR modeling into a dynamic policy contention task, enabling the sub-learners to serve as alternating evolutionary benchmarks and 'self-evolve' toward the global optimum.
Abstract
Click-through rate (CTR) prediction is a pivotal component in large-scale industrial systems. Historically, CTR prediction paradigms have been confined to monolithic architectures governed by a single-policy optimization process. However, such isolated learning paths lack the intrinsic evolutionary mechanisms necessary for optimal convergence. Without policy diversity and internal competition, models tend to get trapped in local optima as performance reaches saturation, hindering further breakthroughs in modeling capacity. In this paper, we propose DPEO (Dynamic Preference Evolution Optimization), a co-evolutionary framework that transforms CTR modeling into a dynamic policy contention task. DPEO decouples the monolithic architecture into dual sub-learners to induce policy diversity, constructing an internal preference landscape without external rewards. A performance-driven Role Arbiter then dynamically designates the superior sub-learner as the Reference Policy and the other sub-learner as the Target Policy per batch, driving continuous model evolution. Through an asymmetric gradient flow, the target policy is optimized to surpass the reference policy in both probability and logit spaces. This process drives a co-evolution, enabling the sub-learners to serve as alternating evolutionary benchmarks and 'self-evolve' toward the global optimum. Extensive experiments on public benchmarks and a massive industrial dataset with over 10 billion samples demonstrate that DPEO significantly outperforms state-of-the-art models.