DuetDA: Decomposed and Dynamic Data Attribution with Model-State Gating for Accelerated Scientific Endeavors

Abstract

Scientific datasets, such as materials and molecular datasets, are often large, complex, and open-ended, posing a core challenge for data efficiency and model training. While data attribution (DA) offers a principled way to score and select samples for efficient learning, we identify a fundamental misalignment between existing DA assumptions and the phase-structured training process. Most DA methods assign each sample a single and static score under a fixed objective, which is difficult to align with scientific training dynamics. Through preliminary results, we identify a clear phase shift: early training primarily benefits from memorizing common and recurring patterns from abundant in-distribution (ID) samples, while later training increasingly relies on generalization by emphasizing complex, long-tailed out-of-distribution (OOD) samples. This phase-dependent behavior implies that a sample's value is not fixed, but can change with the model's state. To address this, we propose DuetDA, a decomposed and dynamic DA framework with model-state gating. DuetDA assigns each sample two complementary values, i.e., a memorization value and a generalization value, and uses a lightweight gate conditioned on the current model state to adaptively integrate them across training. We train DuetDA via meta-learning and apply it at inference time for data attribution, avoiding expensive second-order computations while directly optimizing for downstream performance gains. Theoretically, we show that single-objective static attribution is inherently limited under phase-structured objectives, and that dynamic integration is necessary to recover phase-dependent sample attribution. Empirically, on scientific benchmark MatBench, we show that (1) for accelerated training, DuetDA achieves more than 50% compute reduction while even improving the baselines on OOD set, and (2) compared to data pruning baselines, DuetDA gains about an overall 11% performance compared to the best general data pruning methods on OOD set, showing the effectiveness for open-ended data. We open-source our code at https://github.com/cjpcool/DuetDA.