TBJC: A Targetless Trajectory-Based Joint Calibration Framework for Roadside Multimodal Fusion Perception

Abstract

Joint calibration is essential for ensuring the geometric consistency and reliability of multimodal fusion perception. However, existing methods are difficult to apply directly to roadside perception systems deployed at urban intersections because reliable external calibration cues are often unavailable. To address this challenge, we propose a targetless trajectory-based joint calibration framework, termed TBJC. The framework exploits object trajectories observed by heterogeneous sensors and comprises three core modules: trajectory tracking, cross-modal trajectory matching, and extrinsic parameter estimation. By leveraging the strong spatiotemporal correlations and stable geometric consistency embedded in cross-modal traffic trajectories, TBJC establishes trajectory-level correspondences between 2-D image observations and 3-D light detection and ranging (LiDAR) observations. These correspondences are then used to estimate the camera–LiDAR extrinsic transformation. Building on this framework, we develop TBJC-Alpha as a practical implementation for camera–LiDAR calibration. In TBJC-Alpha, trajectory tracking and cross-modal matching combine learning-based association scoring with global assignment optimization. For extrinsic parameter estimation, efficient perspective-n-point (EPnP) first generates an instantaneous pose estimate, after which random sample consensus (RANSAC) is applied to historical pose estimates to suppress transient outliers and improve estimation stability. TBJC-Alpha is systematically compared with representative targetless calibration methods on the V2X-Seq and TUMTraf datasets. Ablation studies examine the effects of key algorithmic components, while sensitivity analyses evaluate performance under different time-window lengths and trajectory densities. The results demonstrate that TBJC-Alpha achieves competitive performance in accuracy, stability, efficiency, and reliability across the evaluated datasets.