Advances in robust multivariate statistics : methods for component analysis, causal inference, and dependence modeling

Robust Multivariate Regression Based on Shrinkage S n Estimator

Simultaneously Addressing Collinearity and Outliers: A Robust Two-Parameter Estimation Approach

Multicollinearity and outliers remain two major challenges in linear regression modeling, often occurring simultaneously in practical applications and leading to instability, inflated variance, and unreliable inference. Although shrinkage estimators such as ridge and Liu estimators effectively address multicollinearity, but are still sensitive to outliers. Conversely, robust estimators mitigate the influence of outliers but do not adequately resolve collinearity. This study proposes a new robust two-parameter shrinkage estimator (Rprop1) designed to simultaneously handle multicollinearity and outliers within the classical linear regression framework. The estimator is derived in canonical form, and its statistical properties are established through mean squared error (MSE) analysis. A comprehensive Monte Carlo simulation study is conducted across varying sample sizes, error variances, correlation levels, numbers of explanatory variables, and outlier magnitudes. The performance of the proposed estimator is compared with existing robust and shrinkage-based estimators, including robust ridge, robust Liu, and robust Kibra Lukman estimators. Simulation results consistently demonstrate that the proposed estimator achieves superior MSE performance across moderate to severe multicollinearity and outliers’ magnitudes and it’s also supported by the real life dataset. The findings suggest that the proposed method provides a more stable and efficient alternative for regression modeling in the presence of simultaneous collinearity and outliers.

Adaptive Wrapped Robust Canonical Correlation Analysis in High-Dimensional Data

Classical canonical correlation analysis becomes numerically unstable when the number of variables is large relative to the sample size and is sensitive to contamination in observations or individual cells. This study develops an integrated robust and regularized procedure that combines bounded cellwise wrapping, shrinkage estimation of the joint correlation matrix, and robust reweighting in a low-dimensional canonical score space. The resulting observation weights enter a second regularized canonical correlation fit, so the final estimator remains well defined when the combined number of variables exceeds the sample size. The simulation study shows that relative estimation accuracy depends on the signal strength, contamination mechanism, and dimensional configuration. The proposed estimator is competitive in several moderate-signal settings and has a clear computational advantage, whereas the minimum regularized covariance determinant plug-in estimator provides lower estimation error in many high-signal configurations. An additional ultra-high-dimensional experiment demonstrates numerical feasibility with modest memory use but also reveals substantial attenuation, identifying a limitation of the present dense estimator. The results therefore support a regime-dependent interpretation rather than a claim of uniform superiority. The complete reproducible simulation workflow is provided.

A General U-Statistic Framework for High-Dimensional Multiple Change-Point Analysis

High-dimensional change-point analysis is essential in modern statistical inference. However, existing methods are often designed either for specific parameters (e.g., mean or variance) or for particular tasks (e.g., testing or estimation), making them difficult to generalize. Moreover, they typically rely on restrictive distributional assumptions, limiting their robustness to heavy-tailed data. We propose a unified framework for testing, estimating, and inferring multiple change points in high-dimensional data. Our approach leverages a two-sample U-statistic within a moving window, allowing flexible kernel function selection to accommodate structural changes in general parameters such as variance changes or robust statistics. For testing, we develop an L-infinity norm-based statistic with a high-dimensional multiplier bootstrap procedure, achieving minimax-optimal power under sparse alternatives. For estimation, we construct an initial estimator for the change-point number and locations and refine it using the U-statistic Projection Refinement Algorithm (U-PRA), attaining minimax-optimal localization rates. We further derive the asymptotic distribution of refined estimators, enabling valid confidence interval construction. Extensive numerical experiments demonstrate the better performance of our method across various settings, including heavy-tailed distributions. Applications to genomic copy number variation data highlight its practical utility. An R package implementing the proposed method, U-PRA, is publicly available at https://github.com/liubin0145/R-codes-UPRA/.

Integrating Data Cleaning and Robustification: A Study on Linear Regression

In this study, we propose a general framework that integrates both optimistic and pessimistic optimization approaches in solving the regression problem to address outlier cleaning and robustification in a unified fashion. Although data cleaning aims to down-weight the outliers, robustification renders the regression models to heavily rely on extreme data. The main objective of this framework is to construct a new optimization scheme capable of withstanding the influence of outliers without harming the robustness level, by combining these two rather contrasting concepts and operations. In addition to showing its generalization to a few well-known regression models, a set of structural properties of our framework is derived to ensure its statistical significance and to understand its computational demand. Then, we develop solution methods, including mixed integer formulations, alternating direction method of multipliers algorithms, and computation enhancement techniques, that can be applied to handle data sets of different scales. Numerical results on both synthetic and benchmark data sets from the University of California, Irvine (UCI) Machine Learning Repository verify the superiority of our new framework and demonstrate the unified strength to handle complex data sets. History: Accepted by Ram Ramesh, Area Editor for Data Science & Machine Learning. Funding: X. Qian received financial support from the U.S. National Science Foundation (NSF) [Grants SHF-2215573 and IIS-2212419]. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2024.0884 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2024.0884 ). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/ .

Robust Functional Regression via FPCA, Copula-Scale Design, and Multivariate Bernstein Smoothing

We introduce a robust nonparametric regression framework for functional covariates that combines functional principal component analysis (FPCA), marginal copula-scale normalization, bounded-score M-estimation, and multivariate Bernstein smoothing. The proposed procedure reduces the infinite-dimensional functional predictor to a low-dimensional score representation, transforms the retained scores onto the compact unit cube, and estimates a conditional M-functional through a smoothly aggregated system of local estimating equations. This construction is designed to accommodate nonlinear regression structure, heavy-tailed score distributions, and response contamination while limiting the influence of extreme observations. Under suitable regularity and undersmoothing conditions, we establish pointwise and uniform consistency, derive explicit convergence rates, and prove asymptotic normality. The limiting variance contains an explicit Bernstein concentration factor that plays a role analogous to the integrated squared kernel in classical nonparametric regression. The analysis also clarifies the interaction among the projection dimension, the Bernstein resolution, the empirical copula transformation, and the effective local sample size. The finite-sample performance of the method is examined through simulations involving heavy-tailed functional scores, Student-t errors, nonlinear regression effects, and increasing response contamination. The proposed estimator exhibits strong overall predictive performance and good robustness, with particularly favorable behavior under absolute-error criteria.