Addressing Methodological Challenges in Time Series Anomaly Detection

Abstract

Time series anomaly detection focuses on detecting unexpected events in time series data. Traditional techniques for this task include SVMs, k-Nearest Neighbors and PCA, while more recent approaches frequently employ deep learning, e.g., using variational autoencoders and graph neural networks. In several studies, these more elaborate techniques are regularly found to outperform traditional approaches. We present an in-depth review of 18 influential papers on time series anomaly detection using deep learning and found recurring methodological issues which lead to questionable or inconclusive results; leakage, modified datasets, and unclear baselines. We illustrate the importance of these issues with a novel empirical investigation and demonstrate it is not trivial to replicate reported relative performance of classical and deep learning methods. We find that older methods are frequently shown to perform worse than our non optimized implementations. We fail to reject the assumption of similar performance between these older and state of the art deep learning approaches on standard benchmark datasets.