Preprint
Aug 2026
Effective Learning Rate Governs Loss Dynamics in Language Model Pretraining
Results establish ELR as a common coordinate linking LR scheduling, norm control, and loss dynamics, and Controlled interventions further show that weight decay and Hyperball shape loss dynamics primarily through the ELR schedules they induce.
Zi-Han Liu, Rui-Heng Zheng, Shaobo Zhang et al.
· 1 citation