Vernata: Self-Supervised Learning of LiDAR Point Representations

for Point Cloud Representation Learning

LiDAR-Supervised Monocular Depth Estimation via Cross-Modal Supervision

Accurate depth perception is a cornerstone of autonomous driving, yet LiDAR sensors—the primary source of metric depth—remain costly and operationally complex. In this paper, we propose a cross-modal supervision framework that uses sparse LiDAR depth maps solely during training, enabling camera-only dense depth inference at test time. A ConvNeXt-base encoder with an FPN neck and a lightweight depth decoding head is trained with a log-scale L1 loss, gradient consistency term applied exclusively at valid LiDAR pixels (~0.7% pixel density), and an image-guided edge-aware smoothness loss operating on all pixels, alongside a two-phase backbone freeze-then-unfreeze strategy to stabilize early convergence. Evaluated on a large-scale Korean highway dataset of 64,840 frames, our model achieves AbsRel of 0.0675, RMSE of 3.907 m, and $\delta \lt 1.25$ accuracy of 0.943, demonstrating that ultra-sparse LiDAR supervision is sufficient to train competitive monocular depth estimators.

GhostPoint: Self-Supervised Representation Learning by Hallucinating Occluded LiDAR Structure

3D object detection from LiDAR point clouds is a core problem in autonomous driving. Recent advances in self-supervised learning (SSL) enable scalable pretraining and transfers well to per-point tasks such as semantic and panoptic segmentation, but transfer to 3D detection remains weaker. We analyze recent SSL methods and find that most objectives are defined only on measured LiDAR returns from visible surfaces, leaving occluded and unobserved regions unconstrained. This visible-surface bias can be sufficient for point-wise prediction, but 3D detection requires robustness to missing structure. To address this gap, we propose GhostPoint, an SSL framework that hallucinates latent features in local neighborhoods around discovered instances, generated via a novel instance voxel dilation. In GhostPoint, an encoder processes observed returns, and an additional predictor infers neighborhood representations from observed context. In addition to standard encoder-level supervision, we introduce a predictor-level supervision scheme on sampled voxels from generated neighborhoods. Specifically, observed (visible/masked) voxels match teacher-encoder targets, while unobserved voxels match teacher-predictor hallucinations. This design encourages the learned representation to explicitly model structure beyond observed returns. Extensive evaluations on nuScenes and Waymo demonstrate that our method achieves state-of-the-art performance, consistently improving downstream 3D detection, especially under sparse scans and limited labels.

Attention-driven Cross-modal Self-supervised Learning for Label-efficient Hyperspectral-LiDAR DSM Classification

Abstract. Remote sensing acquisition systems rely on a range of platforms, from drones to satellite missions, to record multimodal Earth surface data. This fact encourages the preparation of datasets with complementary properties, thereby increasing their discriminative potential. A common complementary combination is between Hyperspectral and LiDAR-generated digital surface model data. While engaging, this fusion poses challenges for specific applications. Multiple works fuse these modalities at the feature level using vector concatenation, maximization, or averaging. Although functional, these methods omit target interactions between the modalities. Another challenge in remote sensing is the quantity and quality of labels required by deep learning methods, which are expensive, error-prone, and difficult to scale. We address the challenges above by proposing a self-supervised processing framework based on cross-modal attention that effectively fuses features at multiple levels, thereby exploiting complementary information across data streams. Specifically, our method is founded on a pseudo-Siamese network that reweights each modality’s features with information from the other via a mirrored cross-modal attention. The network’s objective is to maximize the similarity between the feature representations of both streams. A fusion network builds a latent representation using the learned encoders and attention modules. Then, a k-Nearest Neighbor classifier categorizes each sample within the representation using ten labels per class. Our experiments show that our spatial- and channel-spatial cross-modal attention approaches outperform well-established fusion methods for label-efficient land cover classification across datasets. Our findings lay the groundwork for fusion methods that effectively exploit inter-stream data relationships to encourage complementarity.

Seed-to-Semantics: Few-Shot Prototype-Guided Progressive Learning for Hyperspectral and LiDAR Classification

Deep learning-based fusion of hyperspectral images (HSI) and LiDAR has achieved strong performance in multimodal remote sensing classification, but its success is heavily constrained by the high cost of pixel-wise annotation. In extremely label-scarce regimes, such as 2-5 labeled samples per class, conventional deep models are prone to severe overfitting, while standard semi-supervised learning (SSL) methods often suffer from confirmation bias because pseudo-labels are generated from unstable early-stage representations. To address these challenges, we propose Prototype-Guided Progressive Learning (PGPL), a unified framework for few-shot HSI-LiDAR classification. Instead of relying solely on model confidence in latent space, PGPL first constructs a reliable initialization pool directly in the original data domain using spectral-angle and elevation-consistency cues, and then progressively expands the training set through class-balanced pseudo-label admission and temporal confidence stabilization. In this way, the framework improves pseudo-label reliability during both initialization and subsequent self-training. Extensive experiments on three benchmark datasets demonstrate that PGPL consistently outperforms state-of-the-art supervised and semi-supervised baselines under the corresponding 2-5-shot settings, achieving overall accuracy gains of 4.64% points on Houston, 1.16% on Trento, and 3.92% on MUUFL over the strongest competing methods, while also yielding higher pseudo-label purity. The source code will be publicly available at https://github.com/zhangyiyan001/PGPL

CVSD-Reg: Cross-Modal Visual Semantic Prior Distillation for Robust LiDAR Registration

Learning-based global point cloud registration has achieved remarkable progress, yet its reliance on geometric representations makes existing methods sensitive to variations in point density, scan pattern, viewpoint, and sensor characteristics. We propose CVSD-Reg, a robust global LiDAR registration framework that distills visual semantic priors from a vision foundation model into LiDAR representations. In Stage 1, a Point Transformer V3 student learns from a frozen DINOv2 teacher through contrastive distillation and spherical-manifold alignment, which preserves the hyperspherical geometry of the teacher embedding space. Self-supervised InfoNCE consistency and soft $\mathrm{SE}(3)$ invariance further encourage viewpoint-robust descriptors. In Stage 2, the distilled representation is adapted to registration through correspondence learning, density-aware point-dropout augmentation, and end-to-end pose optimization. With a single checkpoint, CVSD-Reg generalizes to both single-sensor and zero-shot cross-sensor scenarios without sensor-specific adaptation and remains entirely camera-free at inference. On KITTI, nuScenes, and HeLiPR, CVSD-Reg achieves strict success rate (SR@0.5\,m/$1^\circ$) of 97.7$\%$, 99.0$\%$, and 99.3$\%$, respectively, including 97.3$\%$ on sparse 16-beam Velodyne scans. It outperforms state-of-the-art geometric registration methods by up to 44.0 percentage points without requiring camera inputs or post-hoc ICP refinement.