Reducing False Negatives in AI-Based Breast Histopathology: A Clinically Oriented Evaluation of Deep Learning Models Under Domain Shift

Abstract

Background/Objectives: Deep learning approaches have demonstrated strong performance in breast histopathology image classification; however, reliable generalization across heterogeneous acquisition environments remains challenging due to domain shift. In clinical practice, missed malignant cases are particularly critical because they may directly affect diagnostic decisions and patient outcomes. This study systematically investigates the behavior of modern deep learning architectures and adaptation strategies under realistic cross-domain conditions, with particular emphasis on malignant case detection and false-negative reduction. Methods: Three modern architectures—ConvNeXt-Tiny, Swin-Tiny, and MaxViT-Tiny—were initially trained on a large-scale breast histopathology dataset and subsequently evaluated on the BreaKHis dataset using strict patient-level separation to avoid information leakage. Three transfer settings were investigated: direct zero-shot transfer, head-only adaptation, and full fine-tuning. Performance was evaluated independently across four magnification levels (40×, 100×, 200×, and 400×) using accuracy, precision, sensitivity, F1-score, ROC–AUC, PR–AUC, and false-negative rates. Results: Direct zero-shot transfer produced substantial performance degradation across all architectures, with mean false-negative rates ranging from 75.85% to 90.11%, highlighting the limited transferability of source-domain representations under heterogeneous acquisition conditions. Both adaptation strategies substantially improved performance and reduced missed malignant cases to below 10%. Swin-Tiny under head-only adaptation achieved the most favorable malignant detection profile, reaching a mean sensitivity of 97.36% while reducing the average false-negative rate to 2.64%. In contrast, MaxViT-Tiny achieved the highest mean ROC–AUC value (0.849) after full fine-tuning, although this did not correspond to the lowest false-negative burden. Conclusions: The findings demonstrate that maximizing global discrimination performance does not necessarily correspond to optimal malignant detection under cross-domain conditions. Sensitivity and missed-case analysis provide complementary information beyond conventional discrimination metrics and may support more informed model assessment. Furthermore, the proposed methodology provides a reproducible framework for investigating adaptation performance in AI-assisted breast histopathology systems.