Dynamic Consistency-Aware Multi-View Learning with SAM3 for 3D Medical Image Segmentation

Abstract

Three-dimensional medical image segmentation is critical for clinical diagnosis and treatment planning. Medical images typically exhibit heterogeneous characteristics of low inter-slice resolution and high in-plane resolution, leading 3D CNNs to lose fine edge details via cross-slice feature averaging and traditional 2D slice-based methods to produce discontinuous results due to neglected inter-slice spatial relationships. Multi-view learning shows potential in addressing these issues but lacks sample-adaptive dynamic consistency awareness, as conventional fixed-weight fusion cannot adaptively calibrate view discrepancies for heterogeneous medical data. To tackle this, we propose a dynamic consistency-aware multi-view framework enhanced with SAM3 for 3D medical image segmentation. The framework embeds pairwise cross-view similarity-driven dynamic gating into the multi-view pipeline: decomposing 3D images into three orthogonal views, extracting fine-grained features with a frozen SAM3 encoder, and using a dynamic consistency-aware fusion module that computes pairwise view similarity to dynamically calibrate resolution-induced discrepancies, capture inter-slice and cross-view spatial dependencies, and reconstruct unified 3D features, which are then mapped to segmentation masks via a decoder. Experiments on Automated Cardiac Diagnosis Challenge (ACDC) and infrapatellar fat pad (IPFP) datasets demonstrate that our method achieves superior segmentation accuracy over state-of-the-art approaches, with promising technical performance that supports further multicenter prospective clinical validation.