Evaluating the Effectiveness of AI-Generated Data for Video-Based Action Recognition
TL;DR
This study evaluates AI-generated videos for action recognition across convolutional and transformer-based architectures using real, synthetic, and hybrid datasets and demonstrates that incorporating synthetic videos effectively supports model performance while significantly reducing annotation overhead.
Abstract
Human action recognition relies heavily on large-scale annotated video datasets, which are costly and time-consuming to curate, while AI-generated videos offer a promising alternative data source, their effectiveness for training action recognition models remains insufficiently explored. This study evaluates AI-generated videos for action recognition across convolutional and transformer-based architectures using real, synthetic, and hybrid datasets. To ensure generative diversity and consistency, a structured prompt engineering pipeline combining action descriptions, environmental contexts, and camera viewpoints was developed. Synthetic datasets were generated using the Grok Imagine and Meta AI Vibes video generation models and paired with a 15-class subset of the Human Motion Database 51 (HMDB51) to construct the Generative Synthetic Human Action Recognition Dataset (GenSynth-HARD). To mitigate domain shift arising from discrepancies between real and AI-generated videos, a Conditional Domain Adversarial Network (CDAN) with a dynamically scaled Gradient Reversal Layer (GRL) was integrated for domain feature alignment. The best-performing hybrid model achieved a Top-1 accuracy of 79.92% on the HMDB51 subset, demonstrating that incorporating synthetic videos effectively supports model performance while significantly reducing annotation overhead.