Rethinking Decoder Design for Transformer-Based Polyp Segmentation

Abstract

Accurate polyp segmentation in colonoscopy images is critical for early detection and prevention of colorectal cancer, yet remains challenging due to variable polyp appearance, size, and boundary ambiguity. While recent transformer-based encoders such as Mix Vision Transformer (MiT) have demonstrated strong feature extraction capability, they are typically paired with decoder architectures originally designed for CNN feature hierarchies, creating a potential feature utilization bottleneck. Through systematic comparison of seven encoder–decoder combinations, we find that while MiT-B2 improves Dice by 3.8–20.7% over ResNet50, the decoder choice introduces up to 4.1% per-dataset Dice variation (0.7% on average) with the same encoder, revealing decoder design as an underexplored bottleneck. Based on this analysis, we propose a Progressive Hierarchical Decoder (PHD) with progressive channel projection and efficient skip fusion tailored for transformer feature hierarchies. PHDSeg achieves 0.918 mean Dice on Kvasir-SEG and 0.903 on CVC-ClinicDB over three independent runs, outperforming the best baseline trained under identical conditions by 1.3% average Dice across five benchmarks. We additionally investigate frequency-domain feature enhancement and discover that high-frequency augmentation improves challenging datasets (ETIS: + 1.7%) but degrades well-defined cases (CVC-ClinicDB: −6.3%), providing practical guidance for frequency-based methods in medical image segmentation.