A Joint Intrinsic and Extrinsic LiDAR–Camera Spatiotemporal Calibration System for Roadside Perception

Abstract

Roadside perception achieves comprehensive traffic scene understanding with multiple infrastructure-mounted sensors, and LiDAR–camera fusion plays a vital role. A critical prerequisite for effective fusion is accurate spatiotemporal calibration, including intrinsic calibration, extrinsic calibration, and time synchronization. To ensure precise spatiotemporal calibration of different sensors, calibration targets are commonly used to provide distinct reference points. However, existing target-based calibration methods typically require manual detection steps, constraining the acquisition of temporal information for time synchronization. In addition, in roadside scenarios, target placement is restricted to the drivable road surface, limiting the intrinsic calibration performance. Moreover, most existing methods solely rely on a single 2-D reprojection error for optimization, resulting in suboptimal calibration performance. To alleviate these issues, we propose a joint intrinsic and extrinsic spatiotemporal calibration system using a novel target consisting of a high-reflectivity circular marker and four AprilTags. With this target, we first design an automatic target detection method based on physical and geometric rules to track target trajectories, eliminating the need for manual intervention. Then, we propose a joint optimization method to estimate the intrinsic and extrinsic parameters using trajectories from different sensors. After that, we represent the trajectories as Gaussian mixture models (GMMs) to jointly refine the extrinsic parameters both in 3-D space and on the 2-D plane. Finally, time synchronization is achieved based on the spatially aligned trajectories. In simulator evaluations, the proposed method achieves maximum mean translation, rotation, and time synchronization errors of 1.5 cm, 0.13°, and 16 ms, respectively, outperforming existing methods. Real-world experiments further validate the effectiveness of our proposed method.