This work proposes a novel framework that overcomes this limitation by tightly coupling a DFM with a cascade MVS pipeline through a bidirectional mutual refinement strategy, and introduces a prior-guided cost volume refinement mechanism that effectively integrates multi-view and monocular information via attention-based fusion and discretized depth bins.
Abstract
Deep learning-based Multi-View Stereo (MVS) has advanced significantly but often generalizes poorly to unseen scenes, particularly in occluded areas or regions with limited view overlap. To mitigate this, recent approaches integrate Depth Foundation Models (DFMs) into MVS pipelines to provide monocular depth priors. However, existing methods typically rely on a static, one-way fusion scheme, which fails to fully exploit the complementary strengths of both modalities. We propose a novel framework that overcomes this limitation by tightly coupling a DFM with a cascade MVS pipeline through a bidirectional mutual refinement strategy. Our method leverages MVS depth to resolve the scale ambiguity in monocular predictions, while the monocular depth, in turn, enhances the structural completeness and fine-grained detail of the MVS estimate. Furthermore, we introduce a prior-guided cost volume refinement mechanism that effectively integrates multi-view and monocular information via attention-based fusion and discretized depth bins, thereby promoting local geometric consistency. Extensive experiments demonstrate that our method outperforms state-of-the-art MVS approaches on standard benchmarks, producing more complete and generalizable depth maps with sharp boundaries. Furthermore, although not explicitly designed for sparse-view settings, our framework generalizes remarkably well, competing favorably with even dedicated sparse-view methods while maintaining a superior accuracy-efficiency trade-off.
A unified spatial-temporal depth alignment framework that jointly resolves cross-view and cross-time inconsistencies without distinguishing foreground/background without distinguishing foreground/background is proposed.
Yongzhe Lyu, Shao-Fei Wang, Yi-Xin Chen et al.· 0 citations
Marigold V2 achieves state-of-the-art results when applied to other dense regression tasks, such as surface normals estimation and intrinsic image decomposition, and achieves state-of-the-art results when applied to other dense regression tasks, such as surface normals estimation and intrinsic image decomposition.
Igor Pavlovic, Thiemo Wandel, Anton Obukhov et al.· ACM Transactions on Graphics· 4 citations· ⚡2
The key insight is that DA3's cross-view features, though trained only for depth consistency, implicitly encode motion-discriminative signals when combined with best-match feature search across frames, which could enable much denser temporal sampling that prior methods cannot support.
Xin-Hao Xiang, Wei-Yang Li, Zhi-Jie Zheng et al.· 0 citations
PePESeg3D achieves state-of-the-art performance in both multi-scale segmentation and scene reconstruction, highlighting the importance of integrating perception priors into both geometry optimization and feature learning for accurate multi-scale 3D segmentation.
CMambaDepth is proposed, a self-supervised framework that achieves efficient multi-scale feature fusion and fine-grained contextual modeling via channel-wise selective state propagation and a Hybrid Attention Module is introduced to combine large-kernel local context and Manhattan self-attention for complementary spati...
Xue-Lian Xiang, Jia-Yao Liu, He-Qi Xiang et al.· 0 citations
This work formally introduces the novel problem of multi-view fisheye image generation conditioned on volumetric semantic representations and presents two distinct methods, which achieve robust photorealistic multi-view fisheye image generation and highlight the specific advantages of the cross-domain strategy for hand...
Matthias Neuwirth-Trapp, Begüm Altunbas, Jiaying Wang et al.· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.