Skip to content

MpGI: Multiperspective Generation and Integration for High-Quality Remote Sensing Image–Text Datasets

2026 · IEEE Transactions on Geoscience and Remote Sensing · Vol 64, pp. 5637721-5637721 · 0 citations · 52 references

Abstract

Recent advances in remote sensing (RS) vision-language foundation models (VLFMs) rely heavily on large-scale paired image–text data. However, existing dataset construction methods mainly emphasize data scale while overlooking a more fundamental limitation: the lack of structured and comprehensive semantic representation. Since RS imagery inherently contains multilevel and multiperspective semantics, including object attributes, spatial layouts, scene context, and functional relationships, single-pass caption generation often produces incomplete, biased, or weakly aligned descriptions. To address this issue, we propose a multiperspective semantic distillation paradigm for constructing high-quality RS image–text datasets. Based on this paradigm, we introduce a two-stage framework termed multiperspective generation and integration (MpGI). In the first stage, we perform annotation-guided knowledge elicitation using multiple multimodal large language models (MLLMs), where structured annotations are transformed into model-understandable instructions to extract diverse and complementary visual-semantic descriptions. In the second stage, we conduct semantic distillation and alignment-aware compression using large language models (LLMs), which integrate complementary descriptions and reorganize the resulting semantics into compact captions better suited for vision-language learning. With this framework, we construct HQRS-IT-210K, a high-quality RS image–text dataset containing approximately 210K images and 1.26M image–text pairs. We fine-tune CLIP to obtain HQRS-CLIP, which outperforms previous state-of-the-art RS vision-language models across multiple downstream tasks, including zero-shot classification (ZSC), few-shot classification (FSC), cross-modal retrieval, and semantic localization (SeLo), while using only 4.2% of the training data used by previous approaches. We further fine-tune CoCa on the full dataset to obtain RS-CoCa, which demonstrates strong RS image captioning (RSIC) capability and can generate captions comparable to manual annotations. These results show that improving semantic quality and image–text alignment is a more effective direction than merely increasing data volume for advancing RS vision-language modeling. The dataset, pretrained models, and code are available at https://github.com/YiguoHe/HQRS-210K-and-HQRS-CLIP

View source

Similar papers

Open access 2026

GeoVP: A Unified Visual Prompting Framework for Multisource Remote-Sensing Image Understanding

GeoVP is proposed as a visual prompting MLLM for multisource RS image understanding, enabling unified image-level and region-level understanding under different prompt granularities and demonstrating the effectiveness of explicit visual prompting for prompt-conditioned region understanding in multisource RS imagery.

Le Yu, Yuan-Wen Wang, Xiao-Tong Qi · 0 citations
Sep 2026

VPRef: A Cross-Domain Benchmark for Referring Remote Sensing Image Segmentation

This paper establishes the first cross-domain RRSIS benchmark, designated as the Vaihingen-Potsdam Referring (VPRef) dataset, and develops a tailored parameter-efficient domain adaptation baseline anchored on the Segment Anything Model (SAM3) via Low-Rank Adaptation (LoRA).

Quan-Wei Liu, Tao Huang, Jia-Qi Yang et al. · 0 citations
Open access 2026

Dual-Level Prototype Alignment via Cross-Attention for Few-Shot Remote Sensing Semantic Segmentation

DLPANet is proposed, a novel dual-level prototype alignment network centered on Prototype-Guided Spatial Attention, enabling simultaneous modeling of scene context and fine-grained details and demonstrates that the decoupled dual cross-attention mechanism provides superior prototype-query alignment compared to prior gl...

Mustafa Alawadi, M. Fateh · 0 citations
2026

S2CLNet: Structure-Constrained Semantic Contrastive Learning for Referring Remote Sensing Image Segmentation

Referring remote sensing image segmentation (RRSIS) aims to segment target ground objects in high-resolution remote sensing imagery according to textual descriptions. Due to complex backgrounds and large-scale variations, accurately aligning linguistic semantics with spatial regions remains a challenging problem. Exist...

Sen Lei, Shuai Li, Xin-Yu Xiao et al. · 2 citations
2026

Focused Adapter: Enhancing Fine-Grained Attention for Remote Sensing Image–Text Retrieval

The emergence of large-scale vision–language models (VLMs) has significantly advanced remote sensing image–text retrieval (RSITR) by providing powerful cross-modal semantic priors. However, when adapted to the remote sensing (RS) domain, these models struggle to capture fine-grained representations due to their inheren...

Wen-Liang Du, Xiao-Yu Xu, Jia-Qi Zhao et al. · 0 citations
Jul 2026

StructPointNet: explicit geometric prior extraction and reuse for efficient multimodal remote sensing semantic segmentation

Experimental results demonstrate that StructPointNet delivers competitive segmentation accuracy, and the model proves to be scalable and hardware-friendly, offering a practical paradigm for efficient multimodal interpretation in resource-limited settings.

Si-Wei Wei, Chao-Jie Wang, Ruo-Xi Wang et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.