Gekko is a network that jointly performs cross-view completion, masked autoencoding, and per-pixel prediction of this relative improvement of the cross-view reconstruction error over a masked-autoencoder error, providing an additional binocular signal for all masked regions without any ground-truth 3D annotation.
Abstract
Self-supervised pre-training via cross-view completion learns strong features for 3D vision from co-visible regions of image pairs. However, the reference view provides little information for reconstructing non-co-visible patches, implicitly yielding a monocular training signal in these regions. We introduce Gekko, which turns this limitation into a useful signal. The relative improvement of the cross-view reconstruction error over a masked-autoencoder error is a self-supervised proxy for co-visibility: large improvements indicate co-visible regions, negligible ones non-co-visible areas. Gekko is a network, trained from scratch, that jointly performs cross-view completion, masked autoencoding, and per-pixel prediction of this relative improvement, providing an additional binocular signal for all masked regions without any ground-truth 3D annotation. Under identical architectures and training data, Gekko consistently outperforms CroCo on zero-shot correspondence estimation, relative pose estimation, and pointmap regression, with up to 6 times higher accuracy at the strictest relative-pose threshold and a 22% drop in end-point error on ETH3D. The extra channel it learns is itself a strong co-visibility detector on unseen scenes, and Gekko's frozen features outperform released cross-view backbones of comparable or larger size. It can also be trained directly from raw videos with a simple stride-based curriculum, removing the cumbersome 3D preprocessing prior methods require while matching models trained on curated data. Code and pre-trained models are publicly available.
The Intermodal Dual-MAE Framework with a Similarity-based RL Masking Strategy (SRLM) is proposed, which adaptively masks informative positions by leveraging cross-modal similarity and reinforcement learning, thus narrowing the modality gap.
Zhi-Xin Cheng, Jia-Cheng Deng, Xiao-Tian Yin et al.· IEEE transactions on multime...· 1 citation
This paper introduces dynamic self-distillation (DSD) to provide additional supervision during training, enhancing the model’s ability to learn fine-grained geometric representations, and designs the asymptotic bidirectional feature aggregation network (ABFANet), which enhances multi-scale features through progressive...
Xiao-Gang Song, Jian Liu, Qin Zhao et al.· The Visual Computer· 0 citations
B2TFPose is presented, a training-free zero-shot method for 6DoF pose estimation of unseen objects from RGB images, establishing state-of-the-art performance among training-free RGB methods and outperforming trained counterparts including GigaPose and GenFlow, at competitive inference speed.
It is argued that efficient Real-ISR requires not only a shorter sampling trajectory, but also specialized modeling of faithful reconstruction and perceptual detail synthesis, and proposed PixelIR, a fidelity-perception decoupling framework built upon pixel-space image-residual flow matching.
Bingtian Qiao, Yue Shi, Yong Guo et al.· 0 citations
We argue that learning visual representations without labels requires a training signal jointly complete across three non-overlapping objectives: semantic invariance across augmented views, patch-level spatial prediction, and representational non-degeneracy. We formalize these as the observation, prediction, and regula...
Over a century ago, Henri Poincar\'e argued that a motionless observer cannot acquire the notion of space. Yet, most visual representation learning methods operate on individual images, while those that leverage multiple views rely on RGB reconstruction, entangling geometry with appearance. We propose Poincar3, a self-...
David Nordström, T. Loiseau, Vincent Lepetit et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.