Skip to content
Conference Open access

Skeleton-based action recognition using two-stream ST-GCN and lightweight ViT

Oct 2026 · The International Conference on Optical Communication and Optoelectronic Technology · Vol 14371, pp. 143710T - 143710T-12 · 1 citation · 17 references
Engineering

Abstract

Aiming at the issues of insufficient spatio-temporal feature extraction, fixed topological structure of skeletal graphs, and inadequate modeling of long-range joint dependencies in skeleton-based action recognition, a method named 2s-ViTGCN is proposed, which integrates a two-stream spatial-temporal graph convolutional network (ST-GCN) with a lightweight Vision Transformer (ViT). The method constructs a skeleton dataset using a pose estimation algorithm and employs a dual-stream architecture that takes joint coordinates and bone vectors as input to fuse multi-modal information. By embedding a lightweight ViT module into the spatio-temporal graph convolution, dynamic adjacency matrix learning and long short-term spatio-temporal dependency modeling are achieved. Experiments on a self-constructed dataset demonstrate that the proposed method outperforms current mainstream models in recognition accuracy. The ablation tests indicate that the small ViT unit and two-stream structure are also beneficial to the performance. This paper can be used to increase the precision and reliability of skeleton based action recognition by building topology adaptively as well as using multi-modal features.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.