Colon3R is presented, a cross-domain semi-supervised framework built on pretrained VGGT that transfers coupled camera, depth, and pointmap geometry from labeled phantom and simulated data to unlabeled in-vivo colonoscopy without requiring target-domain geometric annotations.
Abstract
Monocular colonoscopic 3D reconstruction is important for surgical robotic colonoscopy, but remains challenging due to weak texture, specular reflections, limited view overlap, and non-rigid tissue motion. Conventional multi-view 3D reconstruction methods rely on stable correspondences and approximate rigidity, which are often violated in colonoscopy. Existing endoscopic methods often rely on domain-specific supervision, whereas there are not enough in-vivo labeled data available to adapt geometry foundation models to clinical colonoscopy. We present Colon3R, a cross-domain semi-supervised framework built on pretrained VGGT that transfers coupled camera, depth, and pointmap geometry from labeled phantom and simulated data to unlabeled in-vivo colonoscopy without requiring target-domain geometric annotations. Unlike source-only fine-tuning, which learns only from phantom and simulated data, Colon3R directly exploits unlabeled in-vivo video through teacher-derived cross-view supervision. Our proposed hierarchical quasi-rigid reliability selects reliable supervision at the sequence, directed-pair, and pixel levels, while source-preserving adaptation retains the learned coupled geometry during target-domain adaptation. Extensive experiments demonstrate that our method achieves superior overall performance over state-of-the-art approaches in depth, pointmap, and camera pose estimation. Qualitative comparisons on real in-vivo colonoscopy further show substantially more complete and geometrically consistent reconstructions than competing methods under clinical domain shift. The code is publicly available at https://github.com/vglx/Colon3R.
We introduce iMED, the first synchronized multi-endoscope dataset for robot-assisted minimally invasive surgery (RAMIS), designed to address a fundamental limitation in surgical vision benchmarking: the absence of independent held-out viewpoints. Existing surgical datasets capture single-trajectory sequences from a sin...
Sierra Bonilla, Feng Jiang, C. Nwoye et al.· European Conference on Compu...· 1 citation
Geometric understanding supports endoscopic navigation and robotic assistance, but learning reliable endoscopic geometry faces two challenges: scarce geometric annotations and ambiguity between camera motion and tissue deformation. We present SCOPE-4D, an endoscopic 4D geometry foundation model that jointly predicts ca...
Chao-Yi Zhou, Zhong-Pai Gao, Anwesa Choudhuri et al.· 0 citations
Capsule endoscopy serves as a perception front end in medical cyber-physical systems, where monocular reconstruction converts gastrointestinal observations into spatial representations for scene understanding and digital-twin-oriented analysis. However, weak texture, specular reflection, low illumination and limited vi...
Cheng-Li Yang, Zi-Peng Zhang, Meng-Ya Hu et al.· IEEE Transactions on Industr...· 0 citations
Understanding 3D scene dynamics in minimally invasive surgery (MIS) is a fundamental step toward physically intelligent and autonomous surgical systems. However, existing public datasets offer limited scalability and diversity in real-world clinical scenarios. Moreover, annotations typically focus on either tissue or i...
Jia-Xin Guo, Wen-Zhen Dong, Tao-Yu Wu et al.· 0 citations
Gastric endoscopy is central to early gastric cancer detection, but conventional monocular endoscopes provide a narrow field of view, making complete inspection operator-dependent and often lengthening examinations. Existing 3D reconstruction from monocular video enables post-capture review, yet still depends on narrow...
Hayato Yamaguchi, K. Takenaka, Hideki Koike et al.· Proceedings of the 2026 ACM...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.