Skip to content
Preprint

Policy Iteration for Domain Randomized Linear Quadratic Systems

Sep 2026 · 0 citations · 28 references
Mathematics Computer Science Engineering

TL;DR

This work proposes a policy iteration algorithm with a step-size rule that preserves stability across all sampled systems at each iteration, and shows that the method yields monotonic improvement of the sample-average objective.

Abstract

In this work, we study policy optimization under domain randomization for linear quadratic control, focusing on learning a single state-feedback controller that minimizes the average cost across systems with uncertain dynamics. We propose a policy iteration algorithm with a step-size rule that preserves stability across all sampled systems at each iteration. We show that the method yields monotonic improvement of the sample-average objective and that a stabilizing step size always exists. Under standard smoothness assumptions, the iterates converge subsequentially to stationary points, and under a gradient-dominance condition, we obtain a global linear convergence rate.

View source

Similar papers

Sep 2026

Parallel Policy-Gradient Methods for Parameter Optimization of Nonlinear Feedback Controllers

This letter develops a time-parallel policy-gradient framework for discrete-time nonlinear control-affine systems and proves that, for any finite horizon T, the state solver recovers the exact trajectory from any initialization in at most T iterations.

An Duy Nguyen, Lei-Lei Cui · 0 citations
Preprint Oct 2026

Continuous Policy and Value Iteration for Stochastic Control Problems with Inhomogeneous Synchronous Coupling

We propose a continuous policy-value iteration framework for infinite-horizon stochastic control problems, in which the policy evolves through Langevin-type dynamics driven by the gradient of the in-homogeneous Hamiltonian, while the value function is updated simultaneously and continuously along the iteration, avoidin...

Q. Feng, Ren-Jie Shao, Gu Wang · 0 citations
Preprint Sep 2026

Data-Driven Policy Iteration Without an Initial Stabilizing Policy: A Finite-Horizon Bootstrap Method

This article investigates data-driven policy iteration (PI) for continuous-time linear systems without requiring an initially stabilizing policy. Standard infinite-horizon PI is not self-starting because its policy-evaluation step is well posed only when the feedback gain is stabilizing. However, verifying this propert...

Jia-Cheng Wu, Yang Zhu · 0 citations
Preprint Oct 2026

Continuous-Time Policy Gradient from Discrete Data: Convergence Analysis and Model-Free Algorithms

This work develops a policy gradient framework that combines rigorous convergence analysis with learning from discrete data and integrates this framework with PhiBE, which constructs a continuous-time surrogate generator from discrete transition data.

Martín Hernández, Xin-Yan Qing, Mo Zhou et al. · 0 citations
Sep 2026

Carleman approximation based adaptive optimal control design of nonlinear systems: A three-phase policy iteration approach.

A novel adaptive optimal controller design scheme for continuous-time nonlinear systems through neural network-based policy iteration through Carleman linearization, which allows the optimal feedback control to be derived from a state-dependent Riccati (SDR) equation rather than the Hamilton-Jacobi-Bellman equation.

Jian-Guo Zhao, Zhi-Jiang Gao, Chun-Yu Yang et al. · 0 citations
Open access Sep 2026

Reinforcement-Learning Robust Tracking Control of Discrete-Time Systems with Diagonal Scaling

This paper addresses a robust tracking problem for linear discrete-time systems by proposing a reinforcement learning (RL) control method based on a diagonal-scaling strategy, offering a solution tailored to the demands of enhanced reliability. To overcome a common limitation in policy-iteration-based RL design, namely...

Kan-Yang Jiang, Zheng Gao, Ye Zeng et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.