Adaptive Observer-Based Optimal Control via Dual Integral Reinforcement Learning

Abstract

Integral reinforcement learning (IRL) solves the linear quadratic regulator (LQR) problem without requiring the system dynamics matrix, but needs full state measurements. In the absence of states, a dedicated PI observer structure can be used in the learning process, which avoids the limitation of P observer. This paper presents a two-step framework for output-feedback control. The first step applies dual IRL to learn proportional-integral observer gains that achieve loop transfer recovery, preserving the robustness margins of LQR under output feedback. The second step applies primal IRL with the learned observer to obtain the optimal controller from output measurements. The dual IRL algorithm is derived from Newton-Kleinman iteration on the observer algebraic Riccati equation, with convergence guaranteed under standard stabilizability and detectability conditions. Simulation results demonstrate near-optimal performance for both minimum and non-minimum phase systems.