Improved Gradient Descent Optimization Using Adaptive Step-Sizing
Gradient Descent (GD) is among the classic and most common optimization algorithms in machine learning, signal processing and numerical optimization. Although conceptually appealing and simple, traditional gradient descent with a constant learning rate is prone to slow convergence, oscillation or divergence, particular...