2026· IEEE Transactions on Geoscience and Remote Sensing· Vol 64, pp. 5521715-5521715· 0 citations· 75 references
Abstract
The multimodal large language models (MLLMs) have achieved breakthroughs in remote sensing applications because of their powerful cross-modal understanding, enabling effective alignment between linguistic descriptions and visual data. However, the existing MLLM-based approaches still suffer from challenges, such as inaccurate region localization, limited domain adaptability, and high computational costs. To overcome these limitations, we propose a training-free visual prompting MLLM called TFVP-MLLM, which is built on visual prompting guidance and a scoring feedback mechanism to achieve high-quality remote sensing image classification under low resource constraints. Our TFVP-MLLM is composed of the category proposal generator (CPG) and the score feedback refiner (SFR). The CPG is responsible for generating high-quality candidate categories by integrating the proposed geographic priors injection module (GPIM) and hybrid-granularity region attention loss (HGRAL). Meanwhile, SFR uses an iterative generate and score process to enhance the accuracy and reliability of the candidate categories. As two critical components of CPG, GPIM incorporates prior geographic knowledge to significantly improve the model’s domain adaptation, while HGRAL optimizes visual token representations with several frozen modules to reduce computational requirements. To validate the effectiveness of the proposed TFVP-MLLM, the comprehensive experiments are conducted on three benchmark datasets UCM, AID, and DIOR-RSVG. For the scene classification task, TFVP-MLLM achieves the state-of-the-art performance among training-free multimodal approaches with overall classification accuracies of 97.8% and 82.4% on the UCM and AID datasets, respectively. For the regional object classification task, TFVP-MLLM achieves remarkable scores of 92.4% semantic similarity (SS) and 88.6% semantic intersection over union (SIOU) on the DIOR-RSVG dataset, which surpasses the other training-free methods and even achieves comparable results against supervised approaches. In addition, we perform a series of ablation studies to confirm the effectiveness of each module. The source code is available at https://github.com/Camellia86/TFVP-MLLM
Recent advances in prompt learning and multimodal large language models (MLLMs) have improved interactive image understanding. However, fine-grained remote-sensing (RS) interpretation remains challenging because text-only instructions are often insufficient to precisely specify regions of interest in complex scenes, and visual prompting methods developed for natural images generalize poorly to heterogeneous RS data. To address these challenges, GeoVP is proposed as a visual prompting MLLM for multisource RS image understanding. GeoVP supports point, box, and free-form prompts, enabling unified image-level and region-level understanding under different prompt granularities. It employs a hybrid vision encoder to extract multiscale semantic and structural features and a region-aware encoder to convert heterogeneous prompts into unified region representations. These cues are integrated with language instructions for fine-grained RS reasoning. A one-stage training strategy is adopted to improve cross-domain adaptation across natural-image and RS domains. In addition, an auxiliary Pixel-Level Localization Module provides qualitative mask-based visualization cues for prompted regions. GeoVP-650 K is constructed as a 654K-scale image–prompt–text triplet dataset covering optical, synthetic aperture radar, and infrared imagery. GeoVP achieves an average zero-shot classification accuracy of 79.05% on AID and an average cross-task classification accuracy of 87.55% on UCMerced. It also obtains an average semantic intersection over union of 98.25% on DIOR-RSVG under box prompts, demonstrating the effectiveness of explicit visual prompting for prompt-conditioned region understanding in multisource RS imagery.
Le Yu, Yuan-Wen Wang, Xiaotong Qi· IEEE Journal of Selected Top...· 0 citations
The rapid development of multimodal large language models (MLLMs) has introduced a flexible paradigm for remote sensing image scene understanding (RSISU), enabling natural-language interaction with remote sensing imagery. However, a systematic understanding of the capability boundaries, cross-task generalization, and task-specific limitations of existing remote sensing MLLMs (RS-MLLMs) is still lacking. This paper presents a systematic survey and diagnostic evaluation of MLLMs for RSISU. We review the technical evolution of RS-MLLMs, focusing on model design, multimodal learning, training data, and downstream capabilities. We further compare RS-MLLMs with general-purpose computer vision MLLMs (CV-MLLMs) across diverse RSISU tasks and benchmarks. RS-MLLMs remain competitive in domain-specific settings, particularly remote sensing visual grounding and high-resolution visual question answering. More notably, general-purpose CV-MLLMs can match or even outperform these specialized models on several RSISU tasks without remote sensing-specific fine-tuning. These findings demonstrate the strong transferability of general-purpose CV-MLLMs and show that current RS-MLLMs do not consistently outperform them across diverse RSISU tasks. Current MLLMs also face limitations in spatial and relational reasoning, fine-grained visual understanding, instruction diversity, and generalization across heterogeneous task formats. Based on these findings, we outline future directions toward reliable evaluation, multimodal and high-resolution reasoning, efficient deployment, and tool-augmented remote sensing agents. This survey provides a systematic reference for developing robust, generalizable, and practical MLLMs for RSISU.
Qiwei Ma, Chunping Qiu, Xinjun Cheng et al.· 0 citations
Abstract. Zero-shot visual recognition with vision-language models (VLMs) has shown strong generalization to unseen categories in natural-image benchmarks, yet its effectiveness in remote-sensing (RS) imagery remains less explored. In this paper, we investigate whether meta-prompting with large language models (LLMs) can improve zero-shot scene classification in RS by automatically generating semantically rich class descriptions. Building on the Meta-Prompting for Visual Recognition (MPVR) framework, we evaluate three open-source LLMs, Mixtral-8×7B, Qwen 2.5 7B, and LLaMA 3.1 8B, as prompt generators across five RS benchmark datasets. The resulting descriptions are encoded with several VLMs, including CLIP, MetaCLIP, RemoteCLIP, and CLIP-LAION-RS, and compared against generic single-template and handcrafted domain-specific prompting baselines. Our results show that LLM-generated prompts are competitive with, and in several cases improve upon, manually designed templates, while revealing that the gains depend on both the dataset and the visual backbone. Overall, the study highlights the potential of open-source LLMs as scalable prompt generators for zero-shot remote-sensing recognition and provides insight into the transferability of meta-prompting beyond natural-image domains.
Antonis Promponas, Eirini Baltzi, Valsamis Ntouskos et al.· The International Archives o...· 1 citation
Stable cross-domain feature alignment is indispensable for earth observation classification, which is fundamentally hampered by radiometric gaps between generic pre-training images and aerial remote sensing data. Vision-language pre-trained models exhibit strong zero-shot capability on ordinary photos yet suffer severe accuracy loss on satellite and aerial imagery. While LoRA tuning cuts partial training costs, backbone parameter fine-tuning still brings considerable GPU memory overhead in training. Relying on frozen DeepSeek V4 MoE text LLM and static SigLIP vision encoder, this study designs a slim cross-modal projection subnet to eliminate feature distribution gaps between modalities. Stacked residual MLPs constitute the sole learnable part, containing roughly 20M parameters for visual-text latent space matching. The model is evaluated collectively on EuroSAT, PatternNet and RSSCN7, covering nearly 60,000 aerial images with 55 separate scene classes. Recorded aggregate classification precision reached 99.80% across the unified multi-source testing pool. Compared with LoRA-dependent VL-ZSDA-RS benchmark schemes, the adjustable parameter scale shrinks by over half, alongside a 46% cut in peak GPU memory usage. Layer-wise ablation trials reflect unstable matching performance under shallow projection layouts; five stacked transformation layers deliver the most balanced tradeoff between computation overhead and inter-modal alignment quality. External trainable mapping subnetworks, as the test records suggest, unlock visual discrimination capacity for unmodified text-only large language models, supplying a low-hardware threshold tuning route for earth observation research groups constrained by computing resources.
Yuxiang Qiao· Applied and Computational En...· 0 citations
UniRS-Instruct is presented, a high-quality, diversified, and unified multimodal instruction-following dataset for RSI understanding that unifies diverse tasks, including image captioning, visual question answering, visual grounding, and region-level captioning, into a consistent format.
Linrui Xu, Yuhan Wang, Ling Zhao et al.· IEEE Journal of Selected Top...· 0 citations
Recent advances in large language models (LLMs) have reshaped semantic analysis. Opinion Extraction (OE) for Science and Technology Intelligence (STI) requires concise core opinions from large information streams. Off-the-shelf models struggle to filter noise from these streams and show limited structured-output reliability in zero-shot multilingual and multi-modal settings. To address information overload and extraction defocus, this study proposes a multimodal core-opinion extraction framework in which visual evidence serves as a contextual anchor for textual judgment. Using VideoLLaMA2 (VL2) and VideoLLaMA2.1 (VL2.1) as the base models, we apply Quantized Low-Rank Adaptation (QLoRA) fine-tuning on a curated dataset of 2,194 multilingual and multimodal samples. Under the selected Image-Augmented setting, fine-tuned VL2.1 generates structured JSON core-opinion outputs, achieving 64.98% Precision, 42.15% Recall, 51.14% F1-score, and 74.00% sample-level accuracy. Relative to the zero-shot VL2.1 setting, it raises the F1-scores of Spanish and Russian from 4.83% and 0.45% to 46.05% and 51.93%, respectively. The framework further incorporates a Fuzzy Cumulative Prospect Theory-based post-extraction triage module for case-level value assessment, providing a case-level value signal for downstream STI screening.
Sheng Hong, Xuanqi Wang, Jiachen Wang et al.· 0 citations
