Skip to content
Book Open access

IMPEL: Can Pretrained Image Diffusion Enable Realistic LiDAR Point Cloud Generation?

Nov 2026 · Proceedings of the 34th ACM International Conference on Multimedia · 0 citations · 4 references

Abstract

LiDAR data generation holds considerable potential for scalable 3D scene perception in autonomous systems. Existing works typically adapt image-domain generative architectures and train from scratch, but the lack of comparable-scale LiDAR data bottlenecks their accuracy in modeling 3D geometric details. The notable semantic consistency between LiDAR and image modalities provides an opportunity to utilize the image diffusion model pretrained on massive image data for LiDAR generation. However, the substantial modality gap poses a significant challenge for straightforward adaptation. To address this question, we propose IMPEL, a novel framework for LiDAR point cloud generation by exploiting the prior knowledge contained in the pretrained image diffusion. Specifically, we first utilize a Hough voting-based method to project 3D LiDAR point clouds into 2D range images accurately. We then propose a dual spatial consistency optimization strategy to guide the fine-tuning of the pretrained image diffusion model to perceive structural semantics and spatial relationships from both 2D and 3D aspects. Additionally, we design a LiDAR-specific ControlNet that integrates the LiDAR scanning characteristics into the control mechanism for camera-to-LiDAR conditional generation, which holds significant practical value. Experiments conducted on the 64-beam and 32-beam scenarios demonstrate the effectiveness of our framework, substantiating that the rich prior knowledge embedded in the pretrained image diffusion model can be effectively transferred to LiDAR point cloud generation. Code is available at https://github.com/SINCOSLab/IMPEL.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.