Semantic-guided causal alignment for video event anticipation

Abstract

Video event anticipation is the task of inferring future action behaviors from a partially observed sequence of events. However, due to the limited representational capacity of purely visual models, the anticipation framework struggles to decouple genuine motion intents from complex scenes, thereby forming spurious correlations with background objects and degrading its generalization ability. To address this limitation, we propose a novel Semantic-guided Causal Alignment (SCA) framework that introduces supplementary textual representations from vision-language models to guide the model in mitigating contextual bias. To this end, we construct a front-end causal deconfounder module that dynamically re-weights visual channels using text semantics to suppress background noise. Furthermore, we propose a semantic alignment objective to explicitly constrain the visual latent goals distribution within an objective linguistic anchor. By modeling the semantic relationships between text priors and visual latent goals, the framework achieves causal debiasing and ultimately performs robust future event anticipation using the debiased representations. Experimental results on the EPIC-KITCHENS-55 (EK55) and EPIC-KITCHENS-100 (EK100) benchmarks demonstrate that the proposed method consistently outperforms existing mainstream approaches.