Neural monocular depth sensing in metalens-integrated cameras

Abstract

Compact monocular depth sensing is critical for three-dimensional perception in space-constrained platforms such as robotics, wearable devices, augmented reality (AR)/virtual reality (VR), and autonomous systems. However, metalens imaging introduces spatially varying blur and optical imperfections, which severely degrade conventional depth estimation networks. Here, we propose MetaDepth, a neural depth sensing framework tailored for metalens-integrated cameras, enabling efficient depth reconstruction within an ultra-compact form factor. A hybrid measurement – simulation pipeline is developed to generate physics-consistent training data using experimentally measured point spread functions. The proposed network integrates task-driven enhancement, deformable-attention fusion, and adaptive depth regression to suppress metalens-induced non-uniform blur and recover continuous depth maps. Experiments on both synthetic and real metalens images demonstrate accurate and geometrically consistent depth estimation. This work establishes a general optics – algorithm co-design paradigm for compact computational imaging systems and supports practical depth sensing in miniaturized vision platforms.