Gradient-Free Methods for Stochastic Convex Optimization with Stochastic Functional Constraints

Abstract

We develop accelerated gradient-free methods for stochastic convex optimization with constraints defined by expectations. Our batched primal-dual sliding method uses two-point evaluations sharing a random sample and guarantees expected objective error and expected maximum constraint violation at most $\varepsilon$. It achieves $O(\varepsilon^{-1/2})$ sequential oracle rounds for smooth data and $O(d^{1/4}/\varepsilon)$ for nonsmooth data in dimension $d$, recovering the accuracy and dimension dependence of the corresponding unconstrained accelerated methods. The smooth rate is optimal in accuracy. Each round collects all samples needed for its inner primal-dual updates. Total evaluations retain quadratic dependence on inverse accuracy, with only polylogarithmic dependence on the number of constraints under vector feedback. Complementary lower bounds distinguish the dimension cost of gradient estimation from the unavoidable logarithmic cost of estimating noisy constraint levels. For smooth strongly convex objectives, restarts give logarithmic round complexity and objective evaluation cost linear in inverse accuracy, while constraint-level estimation necessarily remains quadratic. Known affine constraints require no constraint-oracle calls.