Skip to content
Preprint

Critic-Free Policy Iteration for Continuous-Time Zero-Sum Games: A Policy-Space Riccati Approach

Sep 2026 · 0 citations · 30 references
Engineering Computer Science

Abstract

This paper develops a critic-free policy iteration (PI) method for continuous-time linear zero-sum games. The central idea is to characterize the saddle-point policies directly in the joint policy space, rather than treating the quadratic value matrix as an iterative variable. A policy game Riccati equation (PGRE) is introduced whose unknowns are policy gains only. Its solutions are shown to be in one-to-one correspondence with the symmetric solutions of the game algebraic Riccati equation. Based on a stabilizing anchor, PI is performed directly in the actor space. The actor-space Jacobian is nonsingular at every stabilizing policy, and the resulting policy sequence coincides with that of simultaneous PI. For unknown dynamics, a data-driven algorithm uses a single batch of data and nullspace projection of endpoint increments to eliminate the value matrix, yielding an actor-only regression. A necessary and sufficient rank condition for unique policy recovery is established and shown to be iteration-invariant, demonstrating that critic identifiability is unnecessary. A power systems frequency-regulation example verifies convergence and policy recovery, while scalability tests demonstrate substantial reductions in computational and memory requirements.

View source

Similar papers

Preprint Sep 2026

Data-Driven Policy Iteration Without an Initial Stabilizing Policy: A Finite-Horizon Bootstrap Method

This article investigates data-driven policy iteration (PI) for continuous-time linear systems without requiring an initially stabilizing policy. Standard infinite-horizon PI is not self-starting because its policy-evaluation step is well posed only when the feedback gain is stabilizing. However, verifying this propert...

Jia-Cheng Wu, Yang Zhu · 0 citations
Sep 2026

Carleman approximation based adaptive optimal control design of nonlinear systems: A three-phase policy iteration approach.

A novel adaptive optimal controller design scheme for continuous-time nonlinear systems through neural network-based policy iteration through Carleman linearization, which allows the optimal feedback control to be derived from a state-dependent Riccati (SDR) equation rather than the Hamilton-Jacobi-Bellman equation.

Jian-Guo Zhao, Zhi-Jiang Gao, Chun-Yu Yang et al. · 0 citations
Preprint Aug 2026

On the Optimality of Markovian Policies for Chance-Constrained Covariance Steering

It is shown that, even for the covariance steering problem with a broad class of commonly used state and control safety constraints, the synthesized Markovian policy almost surely produces the same control actions as the history-dependent policy and therefore the same state trajectories, cost, and moments.

Naoya Kumagai, K. Oguri · 0 citations
Sep 2026

Parallel Policy-Gradient Methods for Parameter Optimization of Nonlinear Feedback Controllers

This letter develops a time-parallel policy-gradient framework for discrete-time nonlinear control-affine systems and proves that, for any finite horizon T, the state solver recovers the exact trajectory from any initialization in at most T iterations.

An Duy Nguyen, Lei-Lei Cui · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.