Jul 2026· International Journal of Remote Sensing· Vol 47, pp. 7118 - 7169· 0 citations· 33 references
TL;DR
This work advances WSSS for remote sensing images by addressing key limitations of CAM-based pseudo-labels generation, offering a cost-effective alternative to fully supervised approaches and facilitating broader applications in geographic information science and earth observation.
Abstract
ABSTRACT Remote sensing image semantic segmentation plays a pivotal role in converting complex image data into quantifiable geographic spatial information, underpinning applications such as disaster assessment, urban planning, and agricultural resource investigation. Fully supervised semantic segmentation methods rely heavily on labour-intensive pixel-level annotations, prompting a shift towards weakly supervised semantic segmentation (WSSS) that utilizes image-level annotations. However, remote sensing images are characterized by dense, intricate targets and the absence of distinct backgrounds, leading to challenges, such as sparse activation of local regions, incomplete localization in class activation maps (CAMs), and noisy, rough boundaries in pseudo-labels generated from image-level supervision. To address these issues, we propose a deep learning method with pixel relationship constraints for WSSS in remote sensing images. Specifically, we design an image reconstruction (IR) loss function to provide pixel-level supervision, enhancing the completeness of CAMs; a pixel relationship constraint (PRC) module to strengthen the global correlation of target regions and improve detailed information extraction; and an intersection optimization strategy (IOS) based on the Segment Anything Model (SAM) to refine pseudo-labels and segmentation results by mitigating noise. Here, we show that our method achieves mean Intersection over Union (mIoU) values of 63.85%, 71.20%, and 40.96% on the Vaihingen, Potsdam, and iSAID datasets, respectively, reaching 89.25%, 89.02%, and 65.66% of the performance of fully supervised methods. This work advances WSSS for remote sensing images by addressing key limitations of CAM-based pseudo-labels generation, offering a cost-effective alternative to fully supervised approaches and facilitating broader applications in geographic information science and earth observation. The code is available at https://github.com/CHENDL-SHEN/PRCCAM.
Image-level weakly supervised remote sensing semantic segmentation aims to learn pixel-level land-cover prediction using only image-level labels, greatly reducing the annotation cost of fully supervised methods. Class activation map (CAM)-based methods are widely used for this task, but they usually focus on the most discriminative regions, leading to incomplete activation and inaccurate boundaries. Recently, vision Transformer (ViT)-based methods have been introduced to alleviate the limitation of CAMs by exploiting token relations and attention mechanisms. However, remote sensing images often contain dense land-cover regions with subtle interclass differences, and patch tokens in deep ViT layers may become oversmoothed without explicit patch-level supervision, weakening local semantic discrimination. Moreover, large intraclass variations and frequent category co-occurrence make image-specific class tokens prone to semantic drift across different remote sensing images. To address these problems, we propose a global and neighbor-aware token learning (GNATL) framework. GNATL contains two complementary modules: neighbor-aware patch token learning (NPTL) and global class token learning (GCTL). NPTL exploits overlapping regions between neighboring crops to construct implicit patch-level constraints, thereby alleviating patch token oversmoothing. Global class token learning (GCTL) dynamically maintains global class tokens as category-level prototypes to guide image-specific class tokens toward stable category semantics. Experiments on the International Society for Photogrammetry and Remote Sensing (ISPRS) Potsdam, ISPRS Vaihingen, and DeepGlobe Land Cover datasets show that GNATL achieves mean Intersection over Union (mIoU) scores of 56.38%, 47.74%, and 62.75%, outperforming the best compared methods by 2.63%, 4.57%, and 2.36%, respectively.
Mansu Gu, Jing Bai, Rui-Zhe Guan et al.· IEEE Transactions on Geoscie...· 0 citations
Pixel-level annotation of remote sensing imagery is costly, motivating weakly supervised semantic segmentation (WSSS) using only image-level labels. However, class activation maps (CAMs) often highlight only discriminative sub-regions and fail to separate adjacent land-cover regions, particularly in remote sensing scenes characterized by densely co-occurring land-cover classes and substantial variations in object scale. To address these limitations, we propose a three-stage framework that integrates complementary priors from Contrastive Language–Image Pre-training (CLIP), Self-Distillation with No Labels version 2 (DINOv2), and the Segment Anything Model (SAM). First, a lightweight CLIP adapter aligns vision–language priors with remote sensing imagery, while sigmoid-based multi-label decoupled distillation replaces class-competitive distillation with independent class-wise supervision, producing more complete CAMs. Second, DINOv2-guided feature clustering decomposes large merged regions before SAM prompt generation, while Spatial–Semantic Constraints are used to construct confidence-guided point-and-box prompts and reject excessively expanded or semantically inconsistent masks, thereby generating reliable pseudo-labels. Finally, a compact segmentation network is initialized with the weights learned in Stage 1 and retrained using the refined pseudo-labels generated in Stage 2, eliminating the need for foundation models during inference. Experiments on the Potsdam, LoveDA, and DeepGlobe datasets show that the proposed method achieves mean intersection over union (mIoU) scores of 53.16%, 52.66%, and 62.98%, respectively, outperforming state-of-the-art WSSS baselines by 6.55, 1.16, and 1.27 percentage points, respectively. These results demonstrate the effectiveness and generalizability of the proposed framework across diverse remote sensing scenarios under image-level supervision.
A lightweight and efficient framework that integrates CLIP and DINO foundation models to address three challenges: semantic misalignment between generic text prompts and RSI-specific visuals; static CAM quality; and incomplete object coverage is proposed.
Xin Li, Nicola Genzano, M. Gianinetto et al.· ISPRS Annals of the Photogra...· 0 citations
Weakly supervised semantic segmentation has emerged as a promising approach for reducing the annotation burden associated with dense pixel-level labelling. Among weak supervision strategies, point supervision offers an attractive compromise by requiring only a small subset of labelled pixels while preserving meaningful semantic information. However, effective optimisation under sparse supervision requires loss functions capable of excluding unlabelled regions from the training process. This study presents an implementation framework for point-supervised remote sensing image segmentation using Partial Cross Entropy Loss. Dense segmentation masks obtained from the LoveDA dataset were converted into sparse point annotations through random pixel sampling with a point ratio of 1%. A subset containing 200 image-mask pairs was constructed and partitioned into training and validation sets. A U-Net architecture with a ResNet-34 encoder was employed as the segmentation backbone, while Partial Cross Entropy Loss was implemented using the ignore-index mechanism available in PyTorch to restrict optimization to labelled pixels only. Functional validation was performed through forward propagation, loss computation, and gradient backpropagation. Successful parameter updates and finite loss values confirmed the correct integration of sparse point supervision with the encoder-decoder segmentation network. The proposed framework provides a simple, modular, and reproducible implementation for point-supervised semantic segmentation and establishes a foundation for future investigations involving weakly supervised and semi-supervised remote sensing image analysis.
Loraine Mutune, Tecla Mutave Kyalo, J. Mutinda et al.· Asian Journal of Research in...· 0 citations
Multimodal fusion methods have shown great potential in remote sensing image analysis, but existing approaches rely heavily on massive amounts of annotated data. This is not only costly and time-consuming but also prone to subjective bias. To address this issue, we propose a category-prior-based self-supervised framework, CGSNet, which uses category prior maps extracted from multispectral images as supervisory signals for end-to-end training. An adaptive confidence-weighted pseudo-label generation mechanism is designed to alleviate noise and errors in prior maps by replacing binary labels with continuous confidence maps, enabling the learning of uncertain interclass features. In addition, a multispectral feature-guided refinement strategy utilizes color and texture information to calibrate class transition regions and enhance the discriminative power of pseudo-labels in complex scenes. A dynamic mask selection strategy further enhances the model’s robustness and generalization capabilities through progressive learning. Experiments demonstrate that CGSNet achieves state-of-the-art performance without the need for human annotation, achieving an Mean Intersection over Union (mIoU) score of 78.46% on the Gaofen image dataset (GID) (vegetation) dataset and 79.58% on the Zurich (vegetation) dataset—12.22% and 15.07% higher than existing methods, respectively—while exhibiting strong cross-dataset zero-shot generalization capabilities. The code will be available at https://github.com/NUAALISILab
Jiahang Liu, Jian Cui, Mao-yin Guo et al.· IEEE Transactions on Geoscie...· 0 citations
Referring remote sensing image segmentation (RRSIS) aims to segment target ground objects in high-resolution remote sensing imagery according to textual descriptions. Due to complex backgrounds and large-scale variations, accurately aligning linguistic semantics with spatial regions remains a challenging problem. Existing RRSIS methods are predominantly segmentation-centric, where textual descriptions mainly serve as conditional guidance for pixel-wise prediction rather than explicitly enforcing semantic consistency between visual regions and linguistic representations. This limitation often leads to suboptimal fine-grained alignment and insufficient suppression of background-dominant regions. To address these challenges, we propose a novel RRSIS framework centered on structure-constrained semantic contrastive learning (CL), termed S2CLNet. Departing from the segmentation-driven paradigm, instead of relying solely on segmentation supervision, S2CL introduces region-level structural constraints to perform fine-grained CL in the joint embedding space. This design encourages semantically corresponding visual regions and linguistic representations to be more closely aligned while effectively separating target regions from irrelevant background areas. Furthermore, we develop a dynamic modality balancing module (DMBM) to enhance cross-modal interaction under complex remote sensing scenarios. The DMBM jointly models bidirectional cross-modal attention (BCA) and dynamically adjusts the relative contributions of visual and linguistic modalities according to scene complexity and linguistic specificity, thereby facilitating more adaptive and robust multimodal feature integration. Extensive experiments on three public benchmarks, RefSegRS, RRSIS-D, and RISBench, demonstrate that the proposed method achieves superior performance compared with state-of-the-art approaches. The code will be publicly available at https://github.com/45degreesl
Sen Lei, Shuai Li, Xin-Yu Xiao et al.· IEEE Transactions on Geoscie...· 0 citations