Leveraging LLM for Semantic Interpretation of Fused Vibro-Acoustic Descriptors in Multi-Fault Diagnosis

Abstract

In industrial applications, multiple faults occurring at the same time contribute to complex, abnormal signal characteristics compared to a single fault condition. If faults are not monitored and identified early, they will lead to unplanned maintenance, production losses, etc. It is often observed that identifying multi-faults is difficult with single-modality sensors due to the same and biased information. A multi-modality sensor approach would be more relevant and accurate. To address the limitations of single-modality diagnosis, this article proposes an alternative method to the traditional fusion process. In this work, vibration and acoustic signals from test rigs at two different institutes are acquired under various multi-fault conditions and pre-processed further to extract time-frequency features. To bring the features of the two modalities into a single comparable range, a text encoding mechanism is adopted which partitions and texturises them across multiple stages. The texturised vibration and acoustic feature vectors are then fused, and a Large Language Model (LLM) is trained with optimised hyperparameters for multi-fault classification. The proposed vibro-acoustic fusion-based methodology is tested against single-modality multi-fault datasets, and its superiority is observed. Maximum classification accuracies of 96.89% and 97.49% are achieved when the model is trained with system-level and element-level multi-fault conditions, respectively. Moreover, model performance is evaluated using data acquired at various rotating speeds and under different imbalance conditions. The proposed method is useful for multi-fault diagnosis of rotating machines under various conditions and can be helpful for incipient-fault diagnosis.