Swarm-optimized heterogeneous ensemble with quantile regression for small-sample long-tailed distribution prediction

Abstract

Predicting maintenance costs for industrial equipment faces the dual challenges of small sample sizes and long-tailed data distributions. Existing methods struggle to ensure both point prediction accuracy and reliable intervals for the tail. This paper proposes the Swarm-driven Adaptive Data Augmentation and Heterogeneous Ensemble with Distribution-Aware Quantile Fusion (SADA-HEDQF) framework. In the SADA stage, Differential Evolution automatically derives high-order features within a domain-specific operator space, and Particle Swarm Optimization optimizes the parameters of the Synthetic Minority Oversampling Technique, enhancing data from both feature and sample dimensions. In the HEDQF stage, a multi-objective Whale Optimization Algorithm constructs a decorrelated heterogeneous base learner pool, and a Quantile Long Short-Term Memory network is designed as the gated fusion meta-learner. Driven by the quantile loss, the meta-learner simultaneously outputs point estimates and multiple confidence intervals. Experiments on 673 real ship maintenance records demonstrate that SADA-HEDQF reduces the point prediction RMSE by 25.8% compared to the best single model, and the error in high-cost regions drops by 46.4%. The normalized average width of the 90% confidence interval is 0.379, and the Coverage Width-based Criterion with penalty on the tail subset reaches 294.5, both significantly outperforming benchmarks such as Quantile Regression Forest. Ablation studies and SHAP analysis quantify the layered contributions of each component and verify the physical interpretability of the derived features, respectively. The framework provides effective prediction and uncertainty quantification support for small-sample long-tailed regression tasks, such as ship maintenance cost prediction, and shows potential for transfer to similar structured prediction scenarios.