This work proposes a policy iteration algorithm with a step-size rule that preserves stability across all sampled systems at each iteration, and shows that the method yields monotonic improvement of the sample-average objective.
Abstract
In this work, we study policy optimization under domain randomization for linear quadratic control, focusing on learning a single state-feedback controller that minimizes the average cost across systems with uncertain dynamics. We propose a policy iteration algorithm with a step-size rule that preserves stability across all sampled systems at each iteration. We show that the method yields monotonic improvement of the sample-average objective and that a stabilizing step size always exists. Under standard smoothness assumptions, the iterates converge subsequentially to stationary points, and under a gradient-dominance condition, we obtain a global linear convergence rate.
This letter develops a time-parallel policy-gradient framework for discrete-time nonlinear control-affine systems and proves that, for any finite horizon T, the state solver recovers the exact trajectory from any initialization in at most T iterations.
An Duy Nguyen, Lei-Lei Cui· arXiv.org· 0 citations
We propose a continuous policy-value iteration framework for infinite-horizon stochastic control problems, in which the policy evolves through Langevin-type dynamics driven by the gradient of the in-homogeneous Hamiltonian, while the value function is updated simultaneously and continuously along the iteration, avoidin...
This article investigates data-driven policy iteration (PI) for continuous-time linear systems without requiring an initially stabilizing policy. Standard infinite-horizon PI is not self-starting because its policy-evaluation step is well posed only when the feedback gain is stabilizing. However, verifying this propert...
This work develops a policy gradient framework that combines rigorous convergence analysis with learning from discrete data and integrates this framework with PhiBE, which constructs a continuous-time surrogate generator from discrete transition data.
Martín Hernández, Xin-Yan Qing, Mo Zhou et al.· 0 citations
A novel adaptive optimal controller design scheme for continuous-time nonlinear systems through neural network-based policy iteration through Carleman linearization, which allows the optimal feedback control to be derived from a state-dependent Riccati (SDR) equation rather than the Hamilton-Jacobi-Bellman equation.
Jian-Guo Zhao, Zhi-Jiang Gao, Chun-Yu Yang et al.· Neural Networks· 0 citations
This paper addresses a robust tracking problem for linear discrete-time systems by proposing a reinforcement learning (RL) control method based on a diagonal-scaling strategy, offering a solution tailored to the demands of enhanced reliability. To overcome a common limitation in policy-iteration-based RL design, namely...
Kan-Yang Jiang, Zheng Gao, Ye Zeng et al.· Eksploatacja I Niezawodnosc-...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.