Knowledge-Guided Vision-Language Models Prompt Learning for Open-Vocabulary Remote Sensing Image Scene Classification

Abstract

Remote sensing open-vocabulary scene classification aims to recognize unseen scene categories beyond the predefined training set by leveraging external knowledge and vision-language models. Prompt learning has emerged as an effective paradigm to adapt pretrained vision-language models to open-vocabulary remote sensing image processing tasks by dynamically encoding task-specific knowledge into textual prompts. However, most existing approaches mainly rely on category names derived from training data, which lack fine-grained semantic descriptions and attribute-level associations that are crucial for complex remote sensing scenes. This limitation often degrades generalization when encountering novel classes and diverse geospatial domains. To address this problem, we propose KnowProKD, a knowledge-guided framework that incorporates diverse external knowledge and constraint-based regularization to enhance base-to-novel generalization. Specifically, KnowProKD consists of three components: 1) leveraging large language models (LLMs) to generate diverse attribute and description knowledge for constructing knowledge-aware prompts, improving cross-modal semantic alignment; 2) introducing knowledge-based constraints to regularize training and enhance robustness to novel category recognition; and 3) designing a dual-branch logits calibration mechanism that exploits attribute and description prompts to balance performance across base and novel classes. Extensive experiments on eight challenging remote sensing scene classification benchmarks demonstrate that KnowProKD achieves favorable performance compared with recent prompt-learning baselines and generalizes well to previously unseen scenarios. Furthermore, we apply KnowProKD to cross-regional urban land-use mapping and statistical analysis across representative cities from six continents, validating its practical effectiveness in real-world remote sensing applications.