Complexity Dependent Error Rates for Physics-informed Statistical Learning via the Small-ball Method

Abstract

Physics-informed statistical learning (PISL) integrates empirical data with physical knowledge to enhance the statistical performance of estimators. While PISL methods are widely used in practice, a comprehensive theoretical understanding of how informed regularization affects statistical properties is still missing. Specifically, two fundamental questions have yet to be fully addressed: (A) what is the trade-off between soft penalties and hard constraints, and (B) what is the statistical gain of incorporating physical knowledge compared to purely data-driven empirical error minimisation. In this paper, we address these questions for PISL in classes of functions whose convex hull satisfies a small-ball condition, in particular convex classes, under physical knowledge expressed as linear equations by developing appropriate complexity dependent error rates based on the small-ball method. We specialise them to linear classes of finite dimension $q$ in which the space of solutions of the equation has dimension $k < q$, and show that (A) the physics-informed estimator attains the error bound of the hard-constrained empirical error minimiser whenever the penalty parameter is above an explicit threshold, and that (B) the penalty improves the error rate from $\sqrt{q/n}$ to $\sqrt{k/n}$ under a second moment condition on the noise only, having the effect of a dimensionality reduction. We also quantify how these conclusions degrade when the equation is misspecified, in terms of the excess residual of the target function. This work establishes a theoretical framework for evaluating the statistical properties of physics-informed estimators in such classes, contributing to closing the gap between statistical theory and practical PISL.