Bing-Hua Li

Gradient Descent on Two ReLU Neurons: Global Landscape and Bifurcation Dynamics

This work tracks the gradient-flow trajectory, showing that student neurons first collapse toward a bisector saddle before escaping and specializing to teacher neurons, and provides both a landscape and dynamics account of the multi-phase symmetry-breaking behavior that arises in a simple multi-index model and under st...