Zhaoxian Wu

Optimization under Persistent State-Dependent Bias: Gradient-based Method and Complexity Analysis

This paper studies the convergence of stochastic gradient descent when the implemented updates are subject to a persistent and state-dependent bias, in which the desired update is scaled by response functions component-wise, and proposes a gradient-based algorithm, termed Residual Learning.