Comparative Analysis of Loss Functions for Semantic Segmentation: An Empirical Study on Cityscapes Dataset

Abstract

— Semantic segmentation remains a fundamental challenge in computer vision, where the choice and weighting of loss functions significantly impact model performance. This study presents a comprehensive comparative analysis of individual versus combined loss functions with systematic weight ablation for semantic segmentation using modified Attention U-Net and DeepLabV3+ architectures on the Cityscapes dataset. We systematically evaluate seven weight configurations across three loss components (Cross-Entropy, Dice, Focal) through rigorous ablation studies, and validate our findings across two architectures to ensure generalizability. Through extensive experimentation across 20 epochs with 2,975 training and 500 validation images, our results demonstrate that the Dice-dominant weighting configuration (0.5:1.0:0.5 for CE:Dice:Focal) achieves superior performance with 57.83% mean Intersection over Union (mIoU) on Attention U-Net and 58.35% mIoU on DeepLabV3+, representing 7.78% improvement over the best individual loss function. Comprehensive ablation studies reveal that weight configuration critically affects performance, with Dice-dominant weighting consistently outperforming equal weighting (55.59% mIoU) and individual loss functions. Qualitative analysis demonstrates substantial improvements in boundary delineation and small object detection, with boundary IoU improving by 1.41% and challenging class performance (trucks, pedestrians) improving by 5 – 21%. Statistical analysis reveals that Cross-Entropy provides the most efficient training with a 75.4% loss reduction, while Dice loss exhibits convergence challenges, resulting in only a 34.5% reduction. Our findings conclusively demonstrate that optimized combined loss function weighting achieves better segmentation performance than both individual approaches and naive equal weighting strategies, with consistent improvements across different network architectures.