A layered stacking ensemble approach for missing data imputation

Abstract

Missing data is a challenging issue in data analysis. Researchers often address the issue by simply removing incomplete cases from the analysis. However, this approach reduces the sample size and can undermine the accuracy of statistical estimates. Alternatively, imputation techniques can be used to replace missing values with plausible estimates, resulting in a more complete dataset and more reliable estimates. In this study, we propose a new imputation method, which we call Layered Stacking. The method is designed to improve the estimation of continuous missing values. To evaluate its effectiveness, comprehensive simulation studies were carried out with four distinct data generating processes using datasets generated under the missing at random mechanism, with sample sizes of 500, 1,000, 5,000 and 10,000, and missingness rates of 10, 20, 30 and 40%. The proposed method is compared with five widely used imputation techniques: k-nearest neighbors, support vector regression, decision tree, random forest and extreme gradient boosting. Evaluation metrics include mean squared imputation error (MSIE) and mean absolute percentage imputation error (MAPIE) along with standard deviation (SD) and 95% confidence interval (CI) across replications. Two real-world datasets have also been considered to assess the performance of the method. The proposed method generally achieved lower MSIE than the benchmark models, particularly for larger datasets.