Improved Convergence of Large Stepsize Gradient Descent for Logistic Regression
We study gradient descent (GD) with a large constant stepsize for logistic regression on linearly separable data. Existing analysis shows an accelerated rate of $\widetilde{O}(1/\sqrt{\epsilon})$ to reach loss $\epsilon$ with an aggressive stepsize, although the loss may initially oscillate. Tighter control of the osci...
Xiao-Chuan Gong, Ang Li
· 0 citations