Large Language Model-Guided Structural Alignment Domain Adaptation for Open-Set Remote Sensing Scene Classification
TL;DR
A novel large language model-guided structural alignment (LLMSA) method is proposed, which effectively handles the intraclass diversity of scenes by leveraging comprehensive representation capability of LLMs and delineates the semantic boundaries clearly and achieves a favorable balance between classification accuracy on known category and unknown category recognition.
Abstract
For open-set domain adaptation (OSDA) in remote sensing scene classification, it is essential to establish precise semantic boundaries for different scenes. Existing vision–language models usually achieve OSDA with fixed templates based on category labels. However, such templates lead to coarse category representations, which make it difficult to describe the diverse scene information within the same category and cause confusion among similar scenes. Furthermore, the fragmented and interleaved background increases scene complexity and interferes with interdomain category knowledge transfer. Inspired by the above-mentioned issues, this article proposes a novel large language model-guided structural alignment (LLMSA) method. Specifically, an LLM-driven semantic generation module is introduced to establish diverse and scalable fine-grained attribute descriptions, which effectively handles the intraclass diversity of scenes by leveraging comprehensive representation capability of LLMs. Under the guidance of scalable attributes, the structural alignment module is employed to mine the relative relationship of structural elements, which alleviates the interference from complex backgrounds and effectively suppresses negative knowledge transfer. Experiments on six cross-domain scenarios with three widely used public datasets demonstrate that LLMSA delineates the semantic boundaries clearly and achieves a favorable balance between classification accuracy on known category and unknown category recognition.