Skip to content
Conference

Toward event anticipation: multimodal semantic contrastive learning for video action anticipation

Oct 2026 · International Conference on Computer Vision, Robotics and Automation Engineering · Vol 14353, pp. 143530H - 143530H-6 · 0 citations · 13 references
Engineering

Abstract

Video action anticipation aims to predict an upcoming action before it actually occurs based only on previously observed video frames, and can be viewed as a fine-grained form of event anticipation in streaming videos. Existing methods mainly rely on visual temporal modeling to infer future representations, while recent LLM-based approaches typically introduce language as an external prior or high-level predictor, making the injected semantics relatively coarse and indirect and thus unable to provide explicit fine-grained supervision for future feature generation, which is critical to anticipation. To address this issue, we propose a multimodal semantic contrastive learning framework that introduces language-derived fine-grained semantic supervision into future representation learning. Specifically, we first construct action-aware text prototypes by prompting a large language model to generate multiple detailed descriptions for each action and encoding them with a VLM text encoder, and then align the generated future features with the corresponding prototypes through contrastive learning to improve the semantic discriminability of future representations. Extensive experiments on EPICKITCHENS- 100 and THUMOS’14 demonstrate the effectiveness of the proposed method.

View source

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.