LiDAR-Supervised Monocular Depth Estimation via Cross-Modal Supervision

Abstract

Accurate depth perception is a cornerstone of autonomous driving, yet LiDAR sensors—the primary source of metric depth—remain costly and operationally complex. In this paper, we propose a cross-modal supervision framework that uses sparse LiDAR depth maps solely during training, enabling camera-only dense depth inference at test time. A ConvNeXt-base encoder with an FPN neck and a lightweight depth decoding head is trained with a log-scale L1 loss, gradient consistency term applied exclusively at valid LiDAR pixels (~0.7% pixel density), and an image-guided edge-aware smoothness loss operating on all pixels, alongside a two-phase backbone freeze-then-unfreeze strategy to stabilize early convergence. Evaluated on a large-scale Korean highway dataset of 64,840 frames, our model achieves AbsRel of 0.0675, RMSE of 3.907 m, and $\delta \lt 1.25$ accuracy of 0.943, demonstrating that ultra-sparse LiDAR supervision is sufficient to train competitive monocular depth estimators.