Variable Smoothing for Weakly Convex Problems with Non-Euclidean Directions

Abstract

We propose MELMO (Moreau Envelope Smoothing with Linear Minimization Oracles), an algorithm for composite optimization problems of the form min x f (x) + g(T x), where f is smooth and g may be non-smooth. The method leverages the Moreau envelope to smooth the non-smooth component while adapting to problem geometry through linear minimization oracles. Assuming g is $\rho$-weakly convex, we establish a family of convergence bounds parameterized by the step-size and smoothing schedules, thereby making explicit the trade-off between optimizing the smoothed objective and recovering stationarity for the original composite problem. In particular, one regime yields O(k -1/4 ) rates for both the smoothed-gradient norm and a composite stationarity proxy, while another yields O(k -1/3 ) for the smoothed-gradient norm together with O(k -1/4 ) for the composite proxy. We also establish a K-horizon-dependent convergence rate that yields O(K -1/3 ) for the composite proxy. Empirically, MELMO is competitive with variable smoothing and subgradient baselines on sparse low-rank matrix factorization and image denoising.