Wenpeng Zhang

Matrix AdaGrad: Row-wise and Column-wise Adaptive Subgradient Methods

Adaptive optimization methods such as AdaGrad and Adam are widely used in modern deep neural network training, but their adaptive scaling is primarily designed for vector-valued parameters and does not explicitly exploit matrix structure. Recent matrix-aware optimizers demonstrate the benefits of structured optimizatio...