Qiang He

Hierarchy-Aligned Learning Rates for Vision Networks

Learning rate (LR) initialization and decay remain important factors in the optimization of deep vision networks. Although these models exhibit a clear hierarchical structure, training typically starts from a single global learning rate, with little explicit consideration of stage depth. This paper investigates a simpl...