Timeseries Foundation Model for Anomaly Detection in Manufacturing Sensor Data

Abstract

This study aims to develop a foundation model for anomaly detection in industrial data. Foundation models, trained on extensive datasets, have demonstrated remarkable success in fields such as natural language processing and image analysis, and there is growing interest in applying them to time-series data. However, existing time-series foundation models primarily focus on prediction, with no models specifically tailored for anomaly detection.Anomaly detection, or identifying deviations from normal behavior, is crucial in industrial settings, especially in high-precision areas like semiconductor manufacturing. Early identification of potential issues in manufacturing processes can enhance efficiency and maintain quality standards. This study proposes a time-series foundation model specifically designed for anomaly detection in large-scale industrial datasets.Key challenges in adapting foundation models for anomaly detection include overgeneralization due to high predictive accuracy and overstationarization resulting from essential normalization layers. Ideally, anomaly detection models should yield high reconstruction errors for abnormal data. However, time-series foundation models developed for prediction often produce low reconstruction errors even for abnormal data, particularly in simpler datasets.To address these challenges, our model incorporates both reconstruction error and embedding vector-based anomaly scores, leveraging the effectiveness of embedding-based anomaly detection in simple time-series data. We also calculate an indicator for simple time-series data to balance the weights between reconstruction error and embedding vector-based scores. Finally, we utilize quality-related statistics and time-series length differences to help mitigate overstationarization.The model is validated on real industrial data and compared qualitatively to traditional quality monitoring methods, such as control charts, using simulated data.