Integrated Spatial–Temporal Framework for Video Anomaly Detection in Surveillance Systems

Abstract

Video-based anomaly detection seeks to discover anomalous events, such as crimes, fires, or medical emergencies, by utilizing both spatial and temporal features of video data. Traditional surveillance systems are frequently limited to minimal recording, requiring human analysts for post-event assessment, resulting in delayed responses during crucial occurrences. To address these issues, we present a multi-layered approach to detecting video anomalies that can deal with both temporal and spatial components of video data. The input video is initially obtained from the dataset and undergoes frame conversion. The extracted key frames are then preprocessed for further analysis. To obtain multi-scale spatial characteristics from each frame, the first layer uses a spatial Pyramid pooling network (SPP-Net) along with a convolutional neural network (CNN). These spatial features are then passed to an optimized bi-directional gated recurrent unit (Opt-Bi-GRU) enhanced with Multi-Head Self-Attention (MHSA), which analyzes the temporal dynamics and captures both forward and backward dependencies across frames. Finally, a capsule network (CapsNet) processes the output of the Bi-GRU, identifying complex patterns that may indicate abnormalities over time. The proposed method is implemented using Python. The proposed model performs better than existing methods in terms of F1-score, specificity, sensitivity, accuracy, recall, precision, FPR, and FNR. The proposed model achieves the highest accuracy of 98.2%, 98.87%, and 98.52%, respectively, utilizing the UBI-fights, UCF-crime, and UCSD pedestrian datasets. These results demonstrate that the proposed framework provides an automated, reliable, and effective solution for real-time anomaly detection in surveillance systems.