OccTTA: Fast and Memory-Efficient Test-Time Adaptation for 3D Occupancy Prediction

Abstract

3D semantic occupancy prediction (OCC) is a crucial task in vision-centric autonomous driving, focusing on reconstructing fine-grained 3D geometry and semantics of the scene surrounding the ego vehicle. Despite recent progress, OCC still faces a significant challenge in generalizing to real-world scenarios due to the unpredictable domain shifts between training data and diverse testing environments. A promising solution to this challenge is test-time adaptation (TTA), which enhances model generalization by updating its parameters to align with the testing domain during inference. However, applying existing TTA methods to OCC is non-trivial as: i) Practical deployment of OCC on resource-constrained onboard devices imposes requirements on memory efficiency. ii) OCC must rapidly adapt to dynamic scenes in latency-sensitive autonomous driving scenarios, posing a challenge for fast test-time adaptation. To address the above issues, we propose a novel fast and memory-efficient Occupancy Test-Time Adaptation (OccTTA) method. Specifically, i) to achieve memory-efficient TTA, we propose a novel TTA framework for OCC based on lite-boost units; ii) to enable fast TTA, we propose a novel forward-backward adaptation strategy that synergizes computationally lightweight forward updates with selective backward optimization. Specifically, our strategy selectively triggers backward updates only for divergent inputs, while employing a k-NN classifier to perform efficient forward updates for accelerating the models responsiveness to domain shifts, thereby simultaneously improving adaptation efficiency and achieving superior performance. Moreover, to address the class imbalance problem in OCC, we introduce an adaptive class-balanced loss that dynamically prioritizes challenging classes, enabling more robust TTA. Extensive experiments on the nuScenes-C benchmark verify the effectiveness of our method.