Enhancing Predictive Accuracy and Operational Performance Based on Entropy–TOPSIS Framework for Machine Learning Model Selection for Predictive Maintenance

Abstract

In the context of Industry 4.0, predictive maintenance increasingly relies on machine learning models to anticipate equipment failures and reduce unplanned downtime. This makes the selection of the most suitable ML model a multidimensional and complex decision problem, since models with comparable predictive performance may differ substantially in terms of failure detection, false positives (FP), false negatives (FN), and computational requirements. This paper proposes a reproducible multicriteria decision-making framework that integrates objective Entropy-based weighting with the TOPSIS method for ML model selection in predictive maintenance applications. Logistic Regression (LR), Random Forest (RF), XGBoost, and LightGBM are evaluated using two predictive-maintenance datasets: the MetroPT dataset, representing real industrial time-series data from an air production unit in a metro system, and the AI4I 2020 Predictive Maintenance dataset from the UCI Machine Learning Repository. The experimental protocol incorporates leakage-aware temporal evaluation for MetroPT, stratified repeated evaluation for AI4I, controlled hyperparameter optimization, statistical significance analysis, and separate assessment of predictive and computational performance. The resulting model rankings are further examined using the VIKOR method and a systematic weight-sensitivity analysis covering 56 perturbation scenarios at ±20% and ±50%. The results show that XGBoost achieves the highest TOPSIS closeness coefficient on MetroPT (Ci = 0.6097), whereas LightGBM ranks first on AI4I (Ci = 0.9837). The VIKOR analysis produces the same ranking as TOPSIS on MetroPT (Spearman ρ = 1.000) and shows a strong but not perfect agreement on AI4I (ρ = 0.800). The sensitivity analysis indicates that the MetroPT ranking remains unchanged across all ±20% perturbation scenarios, while three ranking reversals occur under the ±50% perturbations. For AI4I, the top-ranked model remains unchanged across the tested perturbation scenarios, although the complete ranking can vary under changes in criterion weighting. These findings demonstrate that ML model selection in predictive maintenance should not rely solely on conventional predictive metrics, and that the proposed framework provides a structured decision-support approach for jointly considering predictive and operational performance.