Action-aware anomaly detection in surveillance videos using normal behavior modeling
TL;DR
Qualitative and quantitative results confirm that modeling normal action semantics provides a robust and scalable solution for real-world surveillance anomaly detection.
Abstract
Detecting anomalous events in surveillance videos is a critical yet challenging task due to the rarity, diversity, and unpredictable nature of abnormal activities. Existing methods often rely on fully supervised annotations or weakly supervised multiple instance learning frameworks that require labeled anomalous videos and complex training strategies. In this work, we propose AAAD (Action-Aware Anomaly Detection), a framework that models normal human behavior using learned action embeddings and detects anomalies as semantic deviations. The proposed method first segments videos into fixed-length clips and extracts compact action embeddings using a frozen ResNet18 backbone. An autoencoder is then trained exclusively on embeddings derived from normal clips, enabling the model to learn the distribution of normal actions without requiring anomaly labels. During inference, anomalies are detected based on reconstruction error in the embedding space, allowing temporal localization of abnormal events at the clip level. Experiments conducted on the UCF-Crime dataset demonstrate that the proposed approach effectively distinguishes normal and abnormal activities, achieving 69.0% AUC with a separation ratio of 1.25x. Our method outperforms unsupervised baselines including k-NN (56.4%), One-Class SVM (60.5%), Isolation Forest (54.5%), and Clustering (57.0%). Qualitative and quantitative results confirm that modeling normal action semantics provides a robust and scalable solution for real-world surveillance anomaly detection.