Action Recognition in Sports Videos Using Multiscale Convolutional Networks with Long Short-Term Memory (LSTM)-Based Temporal Modeling.

Abstract

Action recognition in sports videos remains challenging because of complex motion dynamics, occlusion, and high intra-class variability. Although existing deep learning approaches, including CNN-BiLSTM and transfer learning-based models, have demonstrated effectiveness in human activity recognition, their performance may be limited in sports scenarios with rapid, diverse movements. Many existing methods rely on single-scale convolutional filters, which may not effectively capture both fine-grained and coarse motion characteristics simultaneously. To address this limitation, this study proposes a Multi-Scale Convolutional Neural Network (MSCNN) integrated with a Long Short-Term Memory (LSTM) network for sports video action recognition. The MSCNN extracts spatial representations at multiple receptive fields through parallel convolutional kernels, enabling the learning of both detailed and contextual motion features. These features are subsequently processed by the LSTM to capture temporal dependencies and motion continuity across consecutive frames. Experimental evaluation was conducted on the UCF11, UCF Sports, and JHMDB benchmark datasets. The proposed MSCNN-LSTM model achieved classification accuracies of 98.3%, 95.4%, and 81.7%, respectively, outperforming the comparative approaches evaluated in this study. An ablation study further demonstrated the contribution of multi-scale feature extraction and temporal modeling to overall performance. These findings demonstrate the potential of the proposed framework to combine spatial and temporal information for sports video action recognition.