A stochastic behavioral modeling framework, termed Gaussian behaviors, which augments a deterministic linear time-invariant behavior with a Gaussian noise component is proposed, which enables simple and tractable stochastic data-driven control methods.
Abstract
We propose a stochastic behavioral modeling framework, termed Gaussian behaviors, which augments a deterministic linear time-invariant (LTI) behavior with a Gaussian noise component. We show that this notion is a tractable subclass of stochastic behaviors and encompasses classical parametric stochastic LTI state-space system models as special cases. Analogously to deterministic LTI behaviors, the framework enables simple and tractable stochastic data-driven control methods. To this end, we obtain a method for prediction by conditioning the Gaussian behavior on the known part of the trajectory, which is identified directly from the sample covariance of trajectory data. Building on this method, we develop predictive control formulations that optimize over feedforward or disturbance affine feedback policies. The resulting formulations are shown to be convex. We further derive a finite-sample confidence bound on the prediction accounting for both aleatoric and epistemic uncertainty, and incorporate it into a robust control method, for which a tractable convex upper bound is obtained. Within this framework, subspace predictive control is recovered when only the mean prediction is used, while data-enabled predictive control is shown to account for the prediction uncertainty in an optimistic fashion. Numerical case studies illustrate the benefits of the proposed methods.
Many studies on finite-horizon stochastic optimal control, including covariance steering, parameterize control policies as state-history-affine. This parameterization enables a convex reformulation, thereby yielding a tractable solution method. However, the necessity of dependence on previous states has not been well established. \textit{Is this dependence necessary, or merely an artifact of the convex reformulation?} We show that it is an artifact that can be removed losslessly. Given an optimal solution of the state-history-affine formulation, we construct a deterministic Markovian policy which is affine in the current state. We show that, even for the covariance steering problem with a broad class of commonly used state and control safety constraints, the synthesized Markovian policy almost surely produces the same control actions as the history-dependent policy and therefore the same state trajectories, cost, and moments. Thus, every optimum of the history-dependent formulation admits a lossless Markovian transformation. Geometrically, the history-dependent formulation lifts the policy space for convexity, and its optimal solution can be projected back to the Markovian policy space. We extend the analysis to output feedback and a convex upper-bounding surrogate for value-at-risk costs.
We develop a filtering and optimal-control framework for partially observable stochastic systems in which each observation identifies a class of a finite measurable partition of the hidden state space. This structure covers regional-information mechanisms associated with threshold, quantized, censored, event-triggered, and intermittent observations, and allows observable classes with atomic, continuous, or mixed components. The formulation is constructed first at the level of measures: for each observable class, we define a class-restricted unnormalized conditional measure, and the posterior distribution is obtained by normalizing with its predictive probability. Based on this recursion, we introduce an information state consisting of the observed class and the conditional measure supported on it, thereby transforming the original problem into a fully observable Markov decision process. We formulate the discounted-cost criterion, derive the Bellman equation, and establish conditions for the existence of stationary optimal policies. To address the infinite-dimensional nature of the information space, we propose class-dependent finite-dimensional approximations capable of preserving both continuous components and atomic masses. We also derive an abstract bound linking the error of the approximate filter to the error of the value function. A reference model illustrates the construction through histogram-based approximations
Saul Díaz-Infante Velasco, Yofre H. García, J. Minjárez‐Sosa· 0 citations
This paper presents a subspace data-driven predictive control method for linear parameter-varying (LPV) systems. Starting from an affine LPV state-space model in innovation form, we derive a multi-step predictor that separates the effects of past data, future inputs, scheduling trajectories, and innovations. By projecting this representation onto the row span of lifted input-output-scheduling data, we obtain an asymptotically unbiased data-driven predictor that can be embedded directly in a receding-horizon control problem, without explicitly identifying an LPV model. To make the resulting LPV data-driven predictive control (DDPC) formulation tractable, we introduce an LPV extension of $\gamma$-DDPC based on an LQ factorization. This formulation fixes the number of online decision variables independently of the length of the dataset. A reduced-order predictor is then proposed to curb the exponential growth of scheduling-dependent regressors, which also relaxes the persistence-of-excitation condition. Simulation studies, including an unbalanced-disk example, show that the proposed controller achieves good tracking performance and, compared to existing LPV DDPC schemes, achieves better robustness to measurement noise and reduced computational cost, making multi-step LPV DDPC practically deployable, even with longer past horizons.
Federico Porcari, C. Verhoek, V. Breschi et al.· 0 citations
Learning-based Model Predictive Control (MPC) using Gaussian processes (GPs) is an effective approach for safe control in the presence of model mismatch. High-probability safety guarantees typically require uncertainty bounds that hold uniformly over the entire state--input domain, but existing bounds are available only for full GP regression. Since exact GP inference scales poorly with the number of data points, its deployment is impractical in large-data regimes. We close this gap by developing a scalable GP framework that admits the derivation of uniform uncertainty bounds. We formalize a deterministic trigonometric feature Gaussian process (DTF-GP), a finite-dimensional kernel approximation based on discretized trigonometric features that reduces GP regression to Bayesian linear regression in feature space. We derive a high-probability uniform uncertainty bound for the proposed DTF-GP and provide its closed-form solution for the squared-exponential kernel case. Finally, we integrate the DTF-GP into a learning-based MPC scheme and demonstrate that it provides high-probability safety guarantees and exploration performance comparable to a full GP while improving computational efficiency in large-data regimes.
Julius Jagdt, Johanna Menn, Sebastian Trimpe et al.· 0 citations
Sparse ergodic control provides a natural framework for long-run stochastic decision-making under resource constraints. Existing formulations, however, are typically restricted to control-affine systems with control-independent diffusion. When the diffusion coefficient depends explicitly on the control input, the associated ergodic Hamilton–Jacobi–Bellman (HJB) equation becomes non-separable through the term trax, u∇2V, so classical arguments based on control-affine separability no longer apply directly. In this work, we study sparse ergodic control of stochastic systems with control-dependent diffusion and nonlinear dynamics within a viscosity-solution and learning-based framework. To address the discontinuous ℓ0-type sparsity penalty, we introduce smooth non-convex sparsity approximations that preserve differentiability while retaining sparse threshold behavior. Within a viscosity-solution framework, we analyze the existence and uniqueness properties of the associated ergodic pair and establish localized approximation error estimates for the smooth approximation. We further characterize a quasi-threshold sparse structure of the resulting optimal feedback policies in non-affine stochastic systems with control-dependent noise. On the computational side, we develop a Physics-Informed Neural Network (PINN)-based solver with adaptive residual-driven sampling for high-dimensional sparse ergodic HJB equations, together with a distributed monotone-inspired iterative scheme for weakly coupled multi-agent systems. Numerical experiments on multi-robot swarm navigation and renewable-integrated smart-grid control demonstrate that the proposed methods produce sparse control policies while preserving stable long-run performance under stochastic disturbances.
Zhaosheng Xu, Jianbang Liu, Mei Choo Ang et al.· Electronics· 0 citations
This letter develops a data-driven control framework for nonlinear ensemble systems using reservoir computing (RC). We consider ensemble control problems, in which the objective is to regulate a large, potentially uncountable, population of systems with unknown dynamics. To address this challenge, we introduce a moment kernelization approach that yields a dual representation and enables a valid finite-dimensional approximation of ensemble dynamics. Building on this reduction, we cast ensemble control synthesis as the approximation of a causal operator that maps moment trajectories to control inputs. We show that continuous-time reservoir systems induce well-defined causal input-output operators with the fading-memory property, providing a principled foundation for learning these feedback operators from moment trajectory data. Based on this theory, we design an RC-based controller trained on input-output moment trajectories and deployed in a closed-loop configuration for tracking and stabilization of nonlinear ensemble systems.
Yuan-Hung Kuan, Lin Tang, Jr-Shin Li· IEEE Control Systems Letters· 0 citations