G-VTM: A Multimodal Vision-Trajectory Model for Generalized Vehicle Trajectory Prediction
TL;DR
G-VTM, a generalized vision-trajectory model, is proposed, which captures global map semantics while modeling scenario-and direction-aware interaction based on intuitive visual perception and achieves strong generalized performance under heterogeneous traffic conditions.
Abstract
Generalized vehicle trajectory prediction across diverse junctions, including urban intersections and roundabouts, remains a fundamental task in Cooperative Vehicle–Infrastructure Systems (CVIS). This study faces two key challenges: (1) Generalize across junctions with heterogeneous map semantics and traffic behavioral patterns, where the former arises from differences in road topologies and traffic regulations, and the latter reflects diverse behavioral intentions of road users; (2) Scenario-adaptive interaction modeling, where single-modality trajectory learning captures local spatio-temporal correlation, but lacks map constraint and direction-aware interaction contexts. To overcome these challenges, we propose G-VTM, a generalized vision-trajectory model. G-VTM models fine-grained behavioral patterns and relative spatial interaction from trajectory modality. At the vision modality, G-VTM captures global map semantics while modeling scenario- and direction-aware interaction based on intuitive visual perception. Experiments on multiple real-world datasets collected by unmanned aerial vehicles (UAVs) demonstrate that our method achieves strong generalized performance under heterogeneous traffic conditions. The code is provided at https://github. com/zxyhaclyon/G-VTM.