Si-Le Zhang

Understanding the Optimization Dynamics of Large Language Models: A Perspective from Non-Equilibrium Statistical Physics

The training process of modern deep learning models, particularly Large Language Models (LLMs), exhibits complex stochastic behaviors that are difficult to explain using traditional optimization theories. In this paper, we propose a novel perspective by modeling the optimization process of neural networks as a non-equi...