Inverse optimal control of general nonlinear systems based on adaptive dynamic programming

Adaptive Compensation and Adaptive Optimal Control of a Single Link Manipulator

This paper presents an online solution to the finite-horizon optimal tracking control problem for continuous-time nonlinear systems with partially unknown dynamics, based on an Adaptive Dynamic Programming (ADP) approach. The method employs a dual-approximation identifier–critic neural network (NN) architecture, with both networks tuned simultaneously during online implementation. The unknown weights of the identifier and critic activation functions are estimated using a filter-based adaptive algorithm, which provides a simple online validation of the persistence of excitation (PE) condition required for convergence of the control parameters. The controller is evaluated in simulation on an ideal single-link robotic manipulator with partially unknown dynamics and is compared against two classical adaptive nonlinear control strategies: an adaptive Lyapunov-based nonlinear (ALN) controller and an adaptive backstepping (ABS) controller. Performance is assessed in terms of adaptive parameter convergence, tracking accuracy, control input smoothness, and tuning complexity. The ADP-based controller demonstrates the most intuitive tuning process, as its parameters are directly linked to observed system behaviour, and achieves superior tracking performance with the smoothest control input among the three controllers.

Iterative Learning Control for Unknown Nonlinear Systems Based on Data-Driven Model-Free Feedback Linearization

This paper proposes a novel iterative learning control (ILC) scheme for unknown affine nonlinear systems by integrating model-free feedback linearization with two-dimensional (2D) structure of the controlled dynamics. The approach eliminates the requirement for prior model knowledge by employing model reference adaptive control (MRAC) and Q-learning to achieve feedback linearization of unknown nonlinear systems. A computationally efficient method is developed to estimate feedback linearization parameters using historical data from previous trials. Upon obtaining the linearized system representation, the control design is performed within the 2D system setting, resulting in a set of linear matrix inequality (LMI) constraints that leads to the ILC law. The efficacy of the proposed approach is validated through numerical experiments on an inverted pendulum system, demonstrating high-precision trajectory tracking across iterative executions.

Koopman-based model predictive control for nonlinear systems with bounded model uncertainty

Robust Adaptive Neural Network-Based Backstepping Tracking for Second-Order Euler-Lagrange Systems with Unknown Parameters

This paper proposes a robust adaptive tracking control scheme for a class of second-order Euler–Lagrange systems with completely unknown parameters and nonlinear dynamics. System uncertainties, including unmodeled dynamics, parametric variations, and external disturbances, are formulated as a time-varying lumped perturbation. Radial Basis Function Neural Networks (RBFNNs) approximate the unknown state-dependent nonlinear component within the perturbation bound, while adaptive laws estimate the unknown bounding constants of input-dependent terms and disturbances. By integrating backstepping with a $\sigma$-modification mechanism, a continuous adaptive control law is developed that eliminates chattering typically caused by discontinuous robust terms. Lyapunov analysis proves that all closed-loop signals are uniformly ultimately bounded, achieving asymptotic trajectory tracking with smooth control inputs. Simulations on an underactuated Unmanned Surface Vehicle (USV) under complete model uncertainty and environmental disturbances validate the effectiveness and superiority of the proposed method.

Reinforcement Learning‐Based Optimal Prescribed‐Time Tracking Control for Strict‐Feedback Systems With Unknown Affine Terms

This article proposes a novel prescribed‐time optimal (PTO) tracking control scheme for nonlinear strict‐feedback systems with unknown affine terms based on radial basis function (RBF) neural networks. First, by combining the barrier Lyapunov function method, a simple coordinate transformation is used to enable the system's tracking performance to be artificially set. Subsequently, at each step of the backstepping method, reinforcement learning algorithms with critic‐actor structures are introduced to design the optimal virtual controllers and the actual controller by finding solutions to Hamilton–Jacobi–Bellman (HJB) equations for the corresponding subsystems. Meantime, the complexity of system stability analysis caused by PT optimal control is overcome by appropriately decomposing ideal virtual controllers and the ideal actual controller. Based on the new HJB equation with PT characteristics, the easy‐to‐implement critic‐actor updating laws are designed by introducing tracking error as a driving term. It is demonstrated for the first time that, under the persistent excitation (PE) condition, the critic‐actor weights can converge exponentially to the same values. Finally, simulation and experimental results illustrate the effectiveness of the proposed control scheme.

Model-Free Q-Learning for Infinite-Horizon Stochastic Linear Quadratic Problems with Regime Switching

This paper addresses infinite-horizon continuous-time stochastic linear quadratic optimal control problems with regime switching. We propose a paradigm shift from model-based design by adopting an adaptive dynamic programming approach, specifically developing on-policy and off-policy Q-learning algorithms that learn the optimal controller solely from online state trajectory data. The theoretical core of our work consists of a complete proof of the equivalence between the on- and off-policy architectures, alongside a rigorous analysis establishing the stability of the closed-loop system and the convergence of the algorithms to the optimal solution. For computational tractability, we implement these algorithms using vectorization and Kronecker product algebra. The theoretical results are corroborated by numerical case studies that clearly demonstrate the operational effectiveness and practical feasibility of the proposed model-free control strategy.