UncNeRF: Uncovering Heavily Occluded Object With Multi-View Clues

Abstract

Neural Radiance Fields can achieve photo-realistic rendering results, but the occlusion in front of the target object is a common and extreme scenario in practice that cannot be neglected. The prevailing works attempt to remove the occlusions using external 2D visual priors, which are not constrained to provide 3D-consistent guidance for the specific scenarios. In this paper, we propose UncNeRF, which utilizes multi-view clues from captured defective images to uncover the heavily occluded object. Specifically, we provide additional multi-view complementary optimization supervisions using object-centric forward warping and enhance the target object reconstruction by sampling pseudo-training views and introducing external spatial-relation regularization. To evaluate the reconstruction performance of occluded objects, we present the challenging and diverse Heavy Occlusion Removal (HOR) dataset consisting of synthetic and real-world scenes, whose target objects to be reconstructed are heavily occluded. Experimental results show that our method achieves state-of-the-art performance in heavy occlusion removal compared to other methods.