Data Strategies for Large Language Models: A Unified Perspective on the Training Lifecycle
Data strategy plays a central role in training large language models (LLMs). It governs the selection, weighting, and scheduling of training data, thereby determining the capabilities a model can acquire from a fixed data budget. We treat high-quality data as having high training-dependent utility, which depends on the...
Yuan-Jian Xu, Jia-Ning Hao, Tian-Ze Sun et al.
· 0 citations