HyperTransfer: Understanding the Equivalence between Base Optimizer and Hyperball
HyperTransfer is proposed, which constructs a Hyperball optimizer that reproduces the dynamics of a target Base Optimizer using only its initialization and learning-rate schedule, without running the target optimizer itself, to extend the framework to non-scale-invariant networks.