Roadside video-based 2D digital twin representation and short-term trajectory prediction for urban intersections

Abstract

Intersections are among the most critical scenes for traffic sensing, analysis, and motion forecasting. In fixed roadside wide-angle videos, however, accurate digital twin construction is hindered by strong perspective distortion, spatial mapping inconsistency across different image regions, occlusion-induced trajectory fragmentation, detection jitter, and abrupt transitions between observation and prediction segments during sample construction. To address these issues, this paper proposes a practical pipeline for roadside-video-based 2D digital twin representation and short-term trajectory prediction. First, a five-zone partitioned calibration strategy is designed to map pixel trajectories to a unified 2D world plane while reducing region-dependent projection errors. Second, a multi-stage trajectory extraction and cleaning workflow is developed to obtain structured vehicle trajectories suitable for prediction. Third, a Transition Consistency Constraint (TCC) is introduced during sliding-window sample generation to remove windows with unnatural observationto- future jumps and improve spatio-temporal consistency. Finally, the proposed pipeline is validated using recurrent forecasting backbones on a self-built intersection dataset and on the DR_USA_Intersection_EP0 scene from the INTERACTION benchmark. On the self-built dataset, the LSTM model with TCC and low-learning-rate fine-tuning achieves an Average Displacement Error (ADE) of 0.627 m and a Final Displacement Error (FDE) of 0.913 m, corresponding to reductions of about 60.6% and 67.3% over a constant-velocity baseline. Additional experiments with BiGRU further confirm that the proposed data-quality pipeline remains beneficial under an alternative recurrent backbone. On INTERACTION EP0, recurrent models clearly outperform the constant-velocity baseline, while the gain brought by TCC becomes scene dependent under the relative-coordinate setting. These results show that the proposed pipeline improves both the usability of roadside-video-based 2D digital twins and the reliability of short-term trajectory prediction in real intersection scenes.