Yu-Dong Chen

From Spectra to Joint Schedules in LLM Pre-training: 3+3(+2) Scaling-Law Regimes

Power-law learning curves are often treated as fixed properties of a model and its data, although learning-rate and batch-size schedules can change the observed loss. We study this dependence in noisy online SGD with linear random features. Conditional on the representation, an exact Volterra equation separates two res...