#artificial intelligence #machine learning #data science Preprint

General Quantification of Covariate and Concept Shifts

Hong-Bo Chen L. Xia

Sep 2026 · 0 citations · 50 references

Computer Science Mathematics

TL;DR

This paper proposes a key notion: $\gamma^{*}\!$-concept shifts, and derive a general error bound unifying covariate and $\gamma^{*}\!$-concept shifts, which applies to broad loss functions, label spaces, and stochastic labeling and develops estimators for these shifts with concentration guarantees.

Abstract

Generalization under distribution shift remains a core challenge in modern machine learning, yet existing learning bound theory is limited to narrow, idealized settings and is non-estimable from samples. In this paper, we bridge the gap between theory and practical applications. We first show that existing definition of concept shift breaks when the source and target supports mismatch. Leveraging entropic optimal transport, we propose a key notion: $\gamma^{*}\!$-concept shifts, and derive a general error bound unifying covariate and $\gamma^{*}\!$-concept shifts, which applies to broad loss functions, label spaces, and stochastic labeling. We further develop estimators for these shifts with concentration guarantees, and the DataShifts algorithm, which can quantify distribution shifts and estimate the error bound in most applications - a rigorous and general tool for analyzing learning error under distribution shift.

DOI arXiv Semantic Scholar

View source

Similar papers

#machine learning Preprint Sep 2026

Robustness of Diffusion Models under Distribution Shift

This work studies robust score estimation under Wasserstein perturbations of a reference distribution of Ornstein--Uhlenbeck diffusion and characterize how finite data, intrinsic dimension, and distribution shift affect the robustness of score-based diffusion models.

Wei Luo, N. K. Chada, Shi-Jie Zhang et al. · 0 citations

#machine learning Preprint Sep 2026

Learning under Target Shift: Optimal Density Ratio Estimation and Importance-Weighted Regression

A finite-sample theory for continuous density ratio estimation and importance-weighted learning under target shift is established and the resulting regression estimator attains the minimax-optimal rates of standard kernel regression.

Ren-Wei Liu, Zheng-Chu Guo · 0 citations

#machine learning Preprint Sep 2026

Sparse Priors for Efficient Distribution Learning

The results show that learning under an appropriate prior overcomes the curse of dimensionality with respect to the dependence on $n$ and introduces the class of sparse priors and defines the "Sparse Dimension" as a measure of sparsity of a prior over the space of all distributions.

Saumya Goyal, B. Póczos · 0 citations

Preprint Aug 2026

Statistical Properties of Robust Learning under Distributional Shifts

Finite-sample generalization error bounds in the shifted target environment for both DRO and RS are derived, which fill a gap in understanding the statistical properties of robust learning methods under distributional shifts and provide a principled basis for comparing DRO and RS.

Zhiyi Li, Xiaojie Mao, Yun-Bei Xu et al. · 0 citations

Preprint Sep 2026

Reconciling Universal and Uniform Learning with $Q$-Aggregation

This work studies regression under bounded responses in terms of excess mean squared error and proves several additional structural results about universal rates in learning with squared loss about minimax and universal exponential rates.

M. Høgsgaard, Patrick Rebeschini, Tobias Wegel · 0 citations

Preprint Aug 2026

Inference and Uncertainty Quantification for Streaming $r$-PCA

A matching lower bound is proved, up to logarithmic factors, across both dense-tail and sparse-tail regimes under a mild nondegeneracy condition, which enables a high-dimensional Gaussian approximation for the general-rank subspace estimation error with an explicit limiting covariance.

Hao-Shu Xu, Hong-Zhe Li · 0 citations