This work studies the effect of extrinsic reliability and pose conditioning on feed-forward 3DGS, and proposes a calibration-aware pipeline that anchors reconstructed scenes to the robot's metric workspace and presents a renderable, metric-pose-anchored dataset with scene-level reliability information for robot manipulation research.
Abstract
Robot manipulation models primarily reason from 2D observations while acting in the 3D physical world. To bridge this gap, recent work has augmented robot data with geometric priors such as depth, point clouds, and 3D trajectories, while renderable 3D Gaussian representations provide another promising form of 3D supervision. However, 3DGS representation is designed mainly for photometric fidelity and may not preserve real-world metric scale, particularly when the supplied camera extrinsics are unreliable. We study the effect of extrinsic reliability and pose conditioning on feed-forward 3DGS, and propose a calibration-aware pipeline that anchors reconstructed scenes to the robot's metric workspace. Our experiments show that pose conditioning improves novel-view fidelity, while its geometric benefit depends on the reliability of the injected extrinsics. Using this pipeline, we present a renderable, metric-pose-anchored dataset with scene-level reliability information for robot manipulation research. Our dataset is available at https://huggingface.co/datasets/wonguen/3DROID
VI3, a model-agnostic framework that metrically anchors a pretrained 3DFM using only IMU readings, is introduced, acting as a fine refinement under well-conditioned motion and as a strong prior when motion is less informative.
E. Lozano, Alberto Jaenal, Javier Civera· 2 citations
This work introduces a calibration pipeline that combines learned stereo depth with a joint factor graph, pooling all episodes collected from the same physical robot to recover its shared kinematic parameters alongside per-scene extrinsics, and trains DepthWorld, a Stable Video Diffusion-based world model that jointly...
Jai Bardhan, Josef Sivic, Vladimír Petrík· 2 citations
Reliable 6D object pose estimation is a foundational capability for robotic manipulation in Industry 4.0 and unstructured warehouse environments. Current state-of-the-art methods, such as PVN3D and SAM-6D, predominantly rely on “Model-Based” paradigms that necessitate high-fidelity CAD models of the target objects for...
J. Lou, S. Shavetov, N. Tang· Journal of Instrument Engine...· 0 citations
Recovering 3D digital humans from a single 2D image is an ill-posed computer vision problem due to the loss of depth information. Probabilistic 3D human pose estimation compensates for this by estimating a set of 3D hypotheses from a prior distribution via generative models. However, most prior work focuses only on 3D...
Cuong Le, Bao-Long Tran, P. Melnyk et al.· 0 citations
Experiments show improved depth accuracy and temporal consistency over scale-only calibration, demonstrating how localization geometry can support both pose recovery and dense robot perception.
Jia-Rong Lian, Zhen-Hua Xiao, Zhao-Yang Zhang et al.· 0 citations
It is indicated that LiDAR-conditioned diffusion models can induce structured 3D representations from 2D supervision alone, with a modality-dependent manifold that locally unifies near a shared bottleneck.
Samed Doğan, Nico Leuze, Alfred Schöttl· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.