Skip to content

Lesion Segmentation from Imperfect Medical Data via Weakly Supervised Learning with SAM and Explainable Vision Transformer.

Sep 2026 · IEEE journal of biomedical and health informatics · Vol PP · 0 citations
Medicine

TL;DR

A hybrid weakly supervised framework that integrates the Segment Anything Model with an Explainable Vision Transformer for robust and interpretable lesion segmentation and delivers noise-robust segmentation, clinically transparent predictions, and efficient label utilization under imperfect supervision is proposed.

Abstract

Accurate lesion segmentation remains a critical yet challenging task in medical imaging due to scarce pixel-level annotations, inter-observer variability and modality-specific noise. Conventional fully supervised models require exhaustive manual labeling, which limits scalability and clinical applicability. To address these challenges, this paper proposes a hybrid weakly supervised framework that integrates the Segment Anything Model (SAM) with an Explainable Vision Transformer (ViT) for robust and interpretable lesion segmentation. The SAM module converts weak annotations such as bounding boxes and sparse points into high-quality pseudo-masks refined through ensemble consistency, shape priors, and confidence masking, reducing dependency on dense labels. A ViT-based multiple-instance learning (MIL) module then performs weakly supervised classification and mask refinement using uncertainty-aware optimization and self-training, ensuring resilience to noisy supervision. A SAM-ViT alignment loss enforces consistency between pseudo-masks and transformer attention maps, improving anatomical plausibility. Additionally, an explainable AI (XAI) mechanism aligns attention rollout and gradient-based saliency with lesion masks, embedding interpretability directly into the training process. This unified pipeline delivers noise-robust segmentation, clinically transparent predictions, and efficient label utilization under imperfect supervision. Experimental results on CAMELYON16/17 and BraTS datasets demonstrate state-of-the-art performance, achieving AUCs of 0.972, 0.949 and 0.941, with a Dice coefficient of 0.883 and mIoU of 0.826, outperforming existing weakly supervised and SAM-only baselines. This framework establishes a scalable and interpretable foundation for clinical-grade lesion segmentation, reducing annotation cost while improving reliability in real-world medical imaging workflows.

View source

Similar papers

Open access Sep 2026

Uncertainty-guided decoupled complementary network for binary semi-supervised medical image segmentation

Semi-supervised medical image segmentation methods have drawn wide attention as they reduce reliance on heavily annotated data. However, existing models suffer from confirmation bias with limited annotations, and structural or parameter coupling hinders self-correction, especially for medical images with ambiguous boun...

Dong-Sheng Wang, Xiao-Han Lang · 0 citations
Sep 2026

A semi-supervised computer-aided diagnosis network for reducing clinical annotation burden under limited annotation

Results indicate that FFT-EMatch can effectively improve pseudo-label reliability and boundary-aware representation learning under limited annotation settings, highlighting its promise for annotation-efficient medical image segmentation and its potential applicability in practical medical image analysis scenarios.

Neng-Zhao Luo, Yan-Min Luo, Yu-Tian Lin et al. · 0 citations
Sep 2026

Dynamic adaptation and boundary-aware learning for semi-supervised medical image segmentation.

DABAL is proposed, a semi-supervised framework designed to improve both supervision reliability and contour localization and introduces a Dynamic-static Domain Adaptive Adapter (DDAA) into the Segment Anything Model (SAM) encoder to preserve stable structural priors while providing input-dependent feature compensation...

Wei-Yan Zeng, Zhi-Ming Cheng, Bin Lin et al. · 0 citations
Preprint Sep 2026

Weakly-supervised Kidney Tumor Classification from CT Scans with Multi-Instance Learning and Anatomical Filtering

This work investigates two complementary methods to address CT scan analysis challenges: multi-instance learning (MIL) and anatomical filtering, and shows that anatomical filtering with the Compass model is critical for the out-of-distribution generalization of embedding-based ABMIL.

Joonas Ariva, Dmytro Fishman · 0 citations
Conference Oct 2026

DMFNet: a semi-supervised dual-path multiscale fusion network for lesion segmentation in breast ultrasound

Accurate lesion segmentation in breast ultrasound images is crucial for early diagnosis and treatment planning, yet remains highly challenging due to inherent imaging characteristics, including low signal-to-noise ratio, speckle noise artifacts, blurred boundaries, and significant inter-lesion appearance variability. F...

Ya-Dong Lan, Yu Liu, Zai-Yi Liu et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.