Aug 2026· Physics in Medicine and Biology· 0 citations
Medicine
TL;DR
This work demonstrates that MGTP-Seg not only provides an accurate, interpretable, and clinically relevant solution for automatic GTV delineation, but also offers a novel methodological framework to fuse spatial priors with semantic knowledge in medical image analysis.
Abstract
Accurate delineation of the gross tumor volume (GTV) is critical for determining the efficacy of radiotherapy in esophageal cancer. Conventional segmentation methods either rely solely on end-to-end learning from imaging features, which often struggle to address small tumor volumes and ambiguous boundaries, or incorporate coarse masks as spatial priors but fail to integrate the pathological semantics essential for clinical decision-making. This disconnect can lead to segmentation results that are poorly aligned with clinical practice. To address these limitations, this study proposed the Mask Guidance and Text Prompting Segmentation (MGTP-Seg) framework. Based on UNETR, MGTP-Seg innovatively integrates a mask guidance branch and a text prompting branch. The mask guidance branch utilizes pre-segmented masks from nnU-Net to provide spatial priors, while the text prompting branch dynamically integrates clinically relevant semantics from large language models into the visual feature space via learnable prompt tuning. Through adaptive fusion and bidirectional alignment, these branches enable synergistic integration of imaging details, spatial priors, and high-level clinical knowledge in an end-to-end manner. Experiments on multi-center datasets confirm that MGTP-Seg delivers accurate and robust segmentation on both internal and external validation sets. This work demonstrates that MGTP-Seg not only provides an accurate, interpretable, and clinically relevant solution for automatic GTV delineation, but also offers a novel methodological framework to fuse spatial priors with semantic knowledge in medical image analysis.
Background: Accurate glioma subregion delineation is important for radiotherapy planning and longitudinal monitoring, but manual contour correction is time-consuming. Models such as nnU-Net may generalize imperfectly and lack clinician-directed text correction. Purpose: We investigated adapting a three-dimensional (3D) vision-language foundation model for text-guided brain tumor segmentation refinement. Methods: We developed a lightweight VoxTell-based framework. Pretrained VoxTell generated initial masks. Oracle prompts derived from segmentation errors encoded target, action, location, imaging evidence, edit size, and preservation constraints. Frozen Qwen/VoxTell prompt embeddings were injected through trainable projections into its multiscale decoder conditioning; other weights remained frozen. Training, validation, and testing used 901, 100, and 250 BraTS-GLI cases. Cross-dataset transfer was evaluated on 100 meningioma, metastasis, pediatric tumor, and UPENN-GBM cases. Results: On the internal test set using post-contrast T1-weighted input, correct instructions improved subregion Dice similarity coefficient (DSC; enhancing tumor, edema, and necrotic/non-enhancing core) from $0.774\pm0.158$ to $0.796\pm0.137$. They outperformed blank prompts ($0.762\pm0.155$; Holm-adjusted $p<0.001$, $d_z=0.71$) and contradictory prompts ($0.770\pm0.163$; $p<0.001$, $d_z=0.48$). In cross-dataset testing, correct instructions improved DSC from $0.527\pm0.287$ to $0.550\pm0.278$ and outperformed contradictory instructions ($0.504\pm0.275$; $p<0.001$, $d_z=0.43$). Conclusion: A 3D vision-language foundation model can perform instruction-guided refinement of glioma subregion segmentations. Sensitivity to correct, blank, and contradictory prompts suggests text-dependent contour editing rather than nonspecific post-processing, supporting further evaluation as a clinician-in-the-loop tool.
Zach Eidex, Yunyan Lin, M. Safari et al.· 0 citations
Accurate segmentation of the gross tumor volume from computed tomography images is a core step in the development of precise radiotherapy planning for esophageal cancer, which directly affects treatment efficacy and normal tissue protection. Recently, foundation models represented by the segment anything model (SAM) have been increasingly applied in medical image segmentation, and several studies have extended them to tumor target segmentation with promising progress. However, existing SAM-based segmentation methods rely on manual prompts, leading to significant limitations in esophageal cancer. Manual localization of tumor prompt points is difficult, and inappropriate prompts easily cause segmentation errors, increasing the risk of damage to organs-at-risk during radiotherapy. To address this, this study proposes a segmentation method based on SAM that streamlines the inference process by using a coarse-to-fine prompt generation strategy. Inspired by the stepwise refinement of clinical target volume delineation, the core design lies in a coarse-to-fine prompt generation strategy. Specifically, coarse segmentation results generated by nnUNet are first converted into initial prompts to provide global anatomical priors for SAM. Furthermore, a dynamic iterative prompt update mechanism is introduced. During inference, the iterative correction prompts are derived solely from the model’s own prediction uncertainty, forming a closed-loop refinement that gradually improves segmentation accuracy. Experimental results based on multi-center datasets show that this method, which iteratively updates prompts based on its own uncertainty, achieves better segmentation performance than existing single-pass methods on both internal and external validation sets. The framework is highly consistent with the logic of clinical target volume delineation and can provide a reliable efficient scheme for the formulation of precise radiotherapy plans for esophageal cancer.
Yuxuan Yao, Hongfei Sun, Chengwei Chen et al.· Physics in Medicine and Biol...· 0 citations
Brain tumor is one of the most challenging neurological diseases to diagnose and even a minor inaccuracy in the tumor characterization can be fatal. An accurate and reliable brain tumor segmentation from 3D MRI images is a fundamental requirement for an effective diagnosis, treatment planning and assessment of outcome in neuro-oncology. Due to infiltrative growth of tumors, heterogeneity in its structure and diffuse boundaries of tumor regions, brain tumor segmentation is quite critical and challenging. Even a minor error in delineation can adversely affect surgical resection and radiotherapy planning. To address these challenges, this study proposes a region-adaptive ensemble framework that integrates the complementary strengths of two capable 3D segmentation models, SegResNet and Swin UNETR through a staged fusion strategy: simple averaging, region-adaptive soft weighting (RSW), and a disagreement-based region-specific refinement (DRE) for high-conflict voxels. The CNN-based SegResNet is capable in capturing fine-grained local textures and well-defined tumor cores due to its convolutional local bias whereas Transformer-based Swin UNETR is capable in modeling long range contextual dependencies across MRI volume due to its hierarchical Transformer architecture. These two models are finetuned on BraTS 2020 dataset and then integrated using a dynamic voxel-wise disagreement-based fusion strategy that adaptively weights model predictions based on parameters like regional confidence, historical performance and level of disagreement. The multi-run experimental evaluation of the architecture on BraTS 2020 dataset is able to achieve impressive dice scores of 0.9447 ± 0.0021 in Whole Tumor (WT), 0.9231 ± 0.0033 in Tumor Core (TC) and 0.9071 ± 0.0041 in Enhancing Tumor (ET) regions. These results indicate that the Region Adaptive fusion with a disagreement-based refinement between convolutional and transformer-based models leads to a robust framework for brain tumor segmentation, that upon further research and validation might turn out to be suitable for clinical decision making and treatment planning.
Precise delineation of lesion boundaries is a cornerstone of radiological image analysis, directly impacting downstream prognostic accuracy. However, manual annotation of 3D volumetric data is labor-intensive and expert-dependent, creating a significant bottleneck in clinical workflows. While multimodal text-guided segmentation has gained traction, glioma research remains constrained by data scarcity and strict ethical regulations, resulting in a lack of integrated datasets and domain-specific methodologies. To bridge this gap, we introduce TextBraTS, a high-quality, open-access dataset featuring aligned 3D MRI, expert-verified textual descriptions, and genotype information. We establish a Data-Centric Paradigm encompassing an LLM-driven Clinical Narrative Standardization Protocol that distills unstructured reports into structured semantic priors. Furthermore, we propose a Multi-level Gated Fusion (MGF) network to dynamically integrate these expert-curated semantic attributes into 3D vision backbones. Benchmarked against state-of-the-art methods, our framework demonstrates superior precision in both glioma segmentation and molecular subtyping. This work quantifies the "Semantic Gain" of structured textual priors as a proof-of-concept, providing a controlled benchmark for the potential of structured text guidance and providing a robust foundation for integrated radiogenomic analysis and computer-aided diagnosis research.
Xiaoyu Shi, Rahul Kumar Jain, Yinhao Li et al.· IEEE journal of biomedical a...· 0 citations
Text-guided medical image segmentation leverages clinical semantics to improve lesion delineation, yet many existing models bind cross-modal fusion, supervision, and decoder design into a task-specific architecture. Such tight coupling makes it difficult to reuse language guidance modules across heterogeneous vision and text backbones, and often requires redesigning the network when the encoder pair changes. This paper presents BTHA, a backbone-transferable hierarchical adapter framework for text-guided medical image segmentation. BTHA is built around a stable feature-level interface: given multi-scale visual features and a text representation, it injects semantic guidance through shape-preserving adapters while maintaining the decoder-side tensor contract. To make this interface effective, we introduce a Hierarchical Coarse-to-Fine Supervision Strategy that decomposes learning into global image-text alignment, multi-scale auxiliary localization, and boundary-aware final mask refinement. We further design a Scale-Adaptive Gated Semantic Guidance (SAGSG) adapter, where resolution-specific gates adaptively control textual injection and channel recalibration suppresses redundant cross-modal responses. Evaluations across diverse vision and text backbones show that the same adapter and supervision design remains effective across convolutional and transformer-based visual encoders as well as different language encoders. Experiments on four public datasets further demonstrate that BTHA improves strong text-guided baselines with modest computational overhead.
Yung-Hsing Liu, Xuan Fang, Haijin Zeng et al.· 0 citations