Nonsmooth Learning Algorithms Behave Differently from Smooth Ones
Many learning algorithms, including Q-learning, update their estimates through noisy recursive rules. When these updates use a constant step size and involve nonsmooth operators, their long-run behavior can be difficult to characterize: the iterates do...
It is derived that the asymptotic bias of nonsmooth contractive stochastic approximation (SA) with constant stepsize is proportional to the square root of the stepsize, which stands in sharp contrast to smooth SA.
For constant-stepsize stochastic approximation (SA), the iterates converge in distribution to a stationary law that depends on the stepsize $\alpha.$ Steady-state convergence (SSC) concerns the limit of the scaled stationary distribution as $\alpha \downarrow 0.$ Existing SSC theory requires i.i.d. or additive noise an...
We develop Gaussian approximation bounds in higher-order Wasserstein distance $W_p$, $p\geq2$, for sums of multivariate martingale differences generated by a uniformly ergodic Markov chain. Under an $L^{(2+\eta)p}$-moment condition with $\eta>0$, we establish the explicit bound $$ O\left( p^3 \|A\|_4^2 + pd^{1/4}\|A\|_...
Yi-Xuan Zhang, Qiao-Min Xie· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
