AlignDiff: Exploiting Model-Intrinsic Information for Better Preference Data Selection
AlignDiff, a preference data filtering framework driven by intrinsic model signals, first identifies samples with clear preferences using both positive and inverse signals, then prioritizes the more challenging samples based on the average negative log-likelihood gap, encouraging the model to learn richer information f...
Peng Lai, He Zhu, Zhiwen Ruan et al.
· 1 citation