Learning Fine-Grained Video Anomaly Detection from Normal Videos

Abstract

Video anomaly detection (VAD) aims to identify abnormal events in videos. Due to the lack of high-quality training data with detailed annotations, current VAD methods can only produce video-level predictions. To remedy this, several methods attempt to synthesize pseudo video anomalies. However, these methods suffer from low realism and coarse annotations, which limits their performance in real-world scenarios. In this paper, we propose a framework for unsupervised anomaly video generation from solely normal videos, leveraging VLMs to generate structured textual descriptions of anomalies conditioned on the perception of this video. Then, abnormal segments are synthesized using VLMs based on the synthetic textual descriptions. As our framework is highly controllable, video-level and region-level labels can be obtained to provide fine-grained annotations. On top of the synthetic data, we develop a fine-grained VAD network to simultaneously produce video-level, frame-level, and region-level predictions. Experiments show that our method achieves remarkable fine-grained VAD performance.