DDFNet: A Depth-decoupled Dual-directional Guided Fusion Network for Semantic Segmentation

Abstract

To address the issues that VMamba exhibits relatively weak local detail preservation, particularly prone to boundary blurring, and that existing CNN–VMamba fusion methods overlook the distinct characteristics of features at different depths, this paper proposes a semantic segmentation network based on depth-decoupled dual-directional fusion. The network consists of three core modules. First, a cross-branch feature warm-up alignment module aligns the feature distributions of the CNN and VMamba branches via lightweight bidirectional cross-attention before fusion, thereby mitigating the interference of feature discrepancy on subsequent fusion. Second, a shallow boundary bidirectional guidance module employs Sobel boundary gating for forward boundary guidance from the CNN to VMamba, and simultaneously uses semantic consistency responses for reverse semantic guidance from VMamba to the CNN, enabling boundary details and global semantics of high-resolution shallow features to be effectively preserved. Finally, a deep semantic bidirectional fusion module lets VMamba dominate global semantic modeling while the CNN provides only local saliency supplementation, thus avoiding boundary guidance failure at low resolutions. The improved network is evaluated on the Cityscapes dataset. Experimental results demonstrate that the network alleviates the boundary blurring of VMamba and the insufficient feature fusion of CNN–VMamba architectures, achieving a mean Intersection over Union of 78.43%.