The results show that coordinated multi-LLM reasoning can support robust, calibrated, and engineer-interpretable fault classification for automotive HiL validation.
Abstract
Automotive HiL validation generates large multivariate test recordings whose analysis remains challenging due to manual review effort, rule-based limitations, and the need for explainable diagnostic decisions. Recent machine-learning and deep-learning approaches have improved fault diagnosis, but they often require large labelled datasets, generalise poorly across operating conditions, and provide limited insight into their predictions. This paper proposes an explainable multi-LLM ensemble framework for sensor-level fault classification in automotive validation. The framework uses compact evidence representations of fault-injection recordings and combines the outputs of heterogeneous large language models to improve diagnostic robustness, ranking quality, confidence reliability, and interpretability. The approach is evaluated on gasoline-engine and electric-vehicle HiL systems across three driving settings and ten single-fault classes. Among the individual models, Mistral Small~24B provides the strongest overall single-model trade-off, achieving 0.903 Top-1 accuracy, 0.887 MCC, and the lowest Brier score of 0.102. The final Top-3 ensemble combines Mistral Small~24B, Qwen2.5~32B, and Phi-4~14B using confidence-weighted voting, improving the scenario-averaged results to 0.917 Top-1 accuracy, 0.913 macro F1, and 0.902 MCC, while also providing the best calibration among the tested ensemble strategies. A Top-5 ensemble does not improve over the Top-3 configuration, indicating that model complementarity is more important than ensemble size. The results show that coordinated multi-LLM reasoning can support robust, calibrated, and engineer-interpretable fault classification for automotive HiL validation.
The results show that the proposed Condition Monitoring (CM) approach significantly reduces resource waste and prevents costly downtime, offering a practical and scalable asset management model for industrial applications.
A. Oner, Meral Bayraktar· Italian National Conference...· 0 citations
To maintain high efficiency and reduce operational downtime in industrial manufacturing, effective Predictive Maintenance (PdM) for robotic manipulators is essential. Although combining Model-Agnostic Meta-Learning (MAML) with digital twin technology offers a solid basis for quickly identifying faults, conventional methods often face challenges regarding parameter sensitivity and generalizing to new domains. To mitigate these issues, we introduce an ensemble-based metalearning framework that combines MAML with majority voting and operational grouping. This methodology improves generalization, stabilizes performance across diverse conditions, and strengthens few-shot learning capabilities. We validated the framework using a synthetic vibration dataset generated via a digital twin to simulate various robotic arm faults. Our findings demonstrate that this method achieves 93.8% accuracy and 93.1% precision in the ten-shot regime, outperforming the MAML baseline by 11.1%, across a broad range of mechanical faults, showing strength in cross-domain few-shot (CDFS) scenarios. Comparisons with established frameworks - including Reptile, Protonet, and ANIL, confirm the effectiveness of our model. By employing ensemble learning, we attain greater robustness and classification accuracy, establishing the method as a viable solution for industrial PdM. Furthermore, the integration of digital twins bridges the gap between simulation and real-world deployment, reducing data dependency and enabling effective fault classification even in dynamic environments with limited labeled data.
Mainak Mallick, Seung-Kyum Choi· 2026 6th International Confe...· 0 citations
Predictive maintenance of rotating machinery in industrial settings requires bearing fault diagnosis that is both accurate and auditable by maintenance engineers. Methods that achieve high classification accuracy typically operate as closed-box deep learning models, while methods that provide interpretability rarely report probability calibration, out-of-distribution (OOD) detection, or pairwise statistical significance. This paper addresses this gap with an Energy-Based Model (EBM) trained via Stochastic Gradient Langevin Dynamics on physics-informed features, in which the energy gradient with respect to each input feature provides intrinsic interpretability without post-hoc surrogates. A single forward pass simultaneously yields classification logits, a calibrated probability output, and a free-energy score for OOD detection. On three benchmark datasets (CWRU, MFPT, Paderborn) under file-level splitting and across five training seeds, the method attains 93.24% accuracy on Paderborn with an Expected Calibration Error of 0.017, less than half that of a Random Forest baseline. The same energy score separates inputs drawn from datasets absent at training time and preserves accuracy under additive feature noise. A leave-one-operating-condition-out evaluation bounds the operating envelope, in which three of four unseen conditions transfer with moderate loss and the lowest-speed condition does not. The results indicate that classification, probability calibration, OOD detection, noise robustness, and feature-level interpretability can be jointly delivered by a single energy-based model on bearing fault diagnosis benchmarks, without auxiliary classifiers or post-hoc explanation modules.
Electric Submersible Pumps (ESPs) are a widely used artificial-lift technology in oil production, yet unplanned failures lead to costly workovers and substantial production losses. Most machine-learning approaches for ESP predictive maintenance (PdM) are evaluated using same-well validation protocols, such as random splits or k-fold cross-validation, which do not reflect deployment conditions where models must generalize to previously unseen wells and may yield overly optimistic performance estimates. This paper applies Leave-One-Well-Out (LOWO) cross-well validation as a realistic evaluation protocol and demonstrates its implications on the public SPE E-Challenge dataset (63 retained wells; 17 failure wells used in LOWO classification). We show that Remaining Useful Life (RUL) regression performs poorly under cross-well conditions, with no evaluated model exceeding R2 = 0.24. We then evaluate five classification configurations varying featurization, calibration, and ensembling strategy, with the best results reaching AUC = 0.93 and F1 = 0.72 under LOWO. These results highlight important limitations of current PdM validation practices and show that cross-well evaluation is essential for realistic assessment of predictive maintenance models.
Mohamed A. Abdullah, M. Habib· 2026 IEEE International Conf...· 0 citations
Machine learning (ML) techniques have been widely applied to fault detection and diagnosis in Electric Submersible Pumps (ESPs), often reporting high predictive accuracy. However, high performance does not necessarily imply that learned decision boundaries reflect physically meaningful fault mechanisms. This study distinguishes epistemic interpretability, associated with model transparency, from physical interpretability, defined here as the stability of diagnostic decisions across distinct physical units, with operating-regime stability discussed as a broader requirement that cannot be directly isolated from the reference feature file used in this study. Current ML practices in ESP fault diagnosis are examined through a structured literature review and an empirical analysis of a public multi-pump vibration dataset. Representative supervised models are evaluated under sample-wise and cross-unit (pump-wise) validation. Performance is assessed using accuracy, macro-F1, confusion matrices, Principal Component Analysis-based class-space geometry, centroid distances, stability metrics, and bootstrap-based confidence intervals and empirical significance tests for cross-unit degradation. Results show that sample-wise validation can overestimate robustness, whereas cross-unit evaluation reveals performance degradation in most nonlinear and ensemble configurations, with statistical support in several high-capacity models. Class imbalance mitigation reduces majority-class bias but does not eliminate structural misclassification patterns. Representation-level analysis shows that fault condition is the stronger organizing factor in the feature space, while pump identity contributes a measurable but non-dominant fraction of feature-space variance. These findings indicate that predictive accuracy alone is insufficient to characterize diagnostic quality in ESP applications. Explicit cross-unit validation and stability-oriented evaluation are required to assess diagnostic robustness under conditions closer to deployment-relevant generalization.
Miguel A. De C. Michalski, Felipe L. Valentim, Gilberto F. M. De Souza· IEEE Access· 0 citations
Keeping industrial utility equipment reliable is a central problem in industrial predictive maintenance, especially for assets whose faults are reflected in coupled pressure, temperature, flow, power, and efficiency signals. Pure data-driven models can learn complex patterns but are difficult to audit, while conventional rule-based expert systems are interpretable but brittle when symptoms are incomplete. This paper proposes a Hybrid Probabilistic Expert System that combines a deterministic rule base, a Pattern Matching with Scoring (PMS) inference engine, and a Large Language Model (LLM) fallback for low-confidence cases. Diagnostic rules covering seven industrial asset types are organized by physical coupling structure, and PMS converts partial rule matches into normalized confidence scores. We also report a constrained Monte-Carlo coverage analysis to distinguish rule-space reachability from high-confidence diagnosability. On the controlled synthetic benchmark with a mock-LLM fallback, the hybrid system achieved a fault-only accuracy of 0.946 and a macro-F1 of 0.855, and repeated runs yielded a mean hybrid fault-only accuracy of 0.9464 with a 95% confidence interval of 0.0026. In an API-backed real-LLM experiment using DeepSeek v4-pro on a 100-sample synthetic subset, the hybrid system achieved 0.96 accuracy, compared with 0.95 for the expert system and 0.93 for the Real LLM alone. To improve external validity, we further evaluated the method on public LBNL FDD and MetroPT time-series benchmarks after converting public time-series scenarios and windows into abnormal-parameter predicates. On 104 public benchmark samples, the expert system achieved 0.808 accuracy, the Real LLM achieved 0.702, and the hybrid system achieved 0.779 with a false-alarm rate of 0.071. These results indicate that confidence-based expert-first routing can preserve interpretability while providing a controlled pathway for LLM assistance when symbolic evidence is weak or incomplete.
Yunqi Li, Xin Tang, Yinbo Dai et al.· IEEE Access· 0 citations