FASTe: Framework With Application-Driven Spatio-Temporal Efficiency for Video Anomaly Detection

Abstract

Video anomaly detection (VAD) plays a crucial role in modern surveillance systems. However, practical deployment remains challenging due to three key limitations: the difficulty of handling variable-length videos, the lack of fine-grained frame-level anomaly localization, and the high computational complexity of using existing models in real-time. To address these challenges, we propose FASTe, a lightweight and spatio-temporal efficient framework for real-time anomaly detection. FASTe introduces 1) a LogSumExp-based multiple instance learning (MIL) aggregation strategy for robust training on variable-length inputs, 2) frame-level anomaly localization under weak supervision, without requiring dense labels, and 3) spatio-temporal decoupling via adaptive pooling, reducing attention complexity from <inline-formula> <tex-math notation="LaTeX">$O(T^{2} \times H^{2} \times W^{2})$ </tex-math></inline-formula> to <inline-formula> <tex-math notation="LaTeX">$O(T^{2})$ </tex-math></inline-formula>. Evaluated on the UCF-Crime dataset, our approach achieves a receiver operating characteristic area under the curve (ROC-AUC) of 94.57%, achieving improved performance under weak supervision. The proposed framework offers a practical and scalable solution for real-time anomaly detection in resource-constrained surveillance environments.