Precision at Scale: An End-to-End Graph-based Framework for Mitigating Network Interference in TikTok A/B Tests
TL;DR
This paper presents a production-ready framework deployed at TikTok, which integrates three core contributions to address the challenges of large-scale A/B tests on social platforms, and reduces interference rates by 68.8%, correcting a biased treatment effect estimate and enables previously undetectable cross-ecosystem measurements.
Abstract
Large-scale A/B tests on social platforms suffer from inherent network interference, violating the Stable Unit Treatment Value Assumption (SUTVA) and distorting measured results. Industrial practices for mitigating network interference face a foundational trade-off. For scalability, they often rely on clustering static graphs, which serve as imperfect proxies for true interference pathways. Conversely, theoretically sound methods remain computationally intractable at production scale. This paper presents a production-ready framework deployed at TikTok, which integrates three core contributions to address these challenges: 1) Learned Interference Graph (LIG): Estimates interference probabilities using dynamic interaction patterns for more context-aware modeling. 2) Scalable Community Partitioning (SCP): A Spark-optimized ParLeiden implementation that performs billion-node graph clustering daily and generalizes effectively across diverse interaction types, achieving a purity score of 0.898 for group chat interactions. 3) Sensitivity-Enhanced Estimation (SEE): A multivariate system leveraging Controlled-experiment Using Pre-Experiment Data (CUPED) to mitigate variance inflation from cluster-based randomization. In live production tests, our framework reduces interference rates by 68.8%, correcting a biased treatment effect estimate from +1.44% to a statistically significant +2.08%. It also enables previously undetectable cross-ecosystem measurements, revealing a +0.2% lift in creator upload volumes driven by user-side treatments.