2026· IEEE Transactions on Geoscience and Remote Sensing· Vol 64, pp. 5525114-5525114· 0 citations· 54 references
Abstract
The multimodal fusion of hyperspectral image (HSI) and light detection and ranging (LiDAR) data has advanced remote sensing (RS) classification. However, existing methods are mainly based on the closed-set assumption and are thus less effective in real-world open-set scenarios where unknown categories may appear during inference, due to the lack of representative unknown samples and the semantic gap across modalities. To address this, we propose OpenGPL, a novel multimodal open-set recognition (OSR) framework that integrates contrastive language-image pretraining (CLIP) with generative models. First, to tackle the problem of missing unknown samples, we propose a dual-level pseudo-unknown generation strategy. At the pixel level, we employ a diffusion model with a multimetric filtering mechanism to synthesize high-quality pseudo-unknown samples. At the feature level, we introduce a normalizing flow to sample features near the decision boundary, thereby tightening the feature distribution of known classes via an outlier exposure (OE)-inspired strategy. Second, to bridge the multimodal semantic gap, we design a positive–negative prompt learning mechanism. By maximizing the similarity between features and positive prompts while pushing them away from negative prompts, this scheme explicitly constructs a more discriminative semantic space, effectively enlarging the boundary between known and unknown categories. Finally, we adopt a dual-verification mechanism that fuses energy scores with similarities to negative prompts to accurately reject unknown classes. Comprehensive experiments conducted on three benchmark HSI-LiDAR datasets demonstrate that the OpenGPL achieves competitive performance in recognizing known classes and detecting unknown categories in open-set scenarios.
Compared with cross-domain few-shot learning (CDFSL), open-set CDFSL (OS-CDFSL) is more challenging because it must accurately classify known classes across domains while reliably rejecting unknown samples. Under this setting, domain shift causes inconsistent spectral distributions for samples from the same class and c...
Chen Ding, Si-Rui Zheng, Yi-Zhou Dong et al.· IEEE Transactions on Geoscie...· 0 citations
Pixelwise classification of hyperspectral images (HSI) is a fundamental task in remote sensing, but its deployment in real-world applications is often limited by the scarcity of labeled data and the presence of unknown classes at inference time. In this work, we propose Few-shot Open-set hyperspectral recognition from...
M. B. Pereira, J. D. dos Santos, Laurent Najman· IEEE Journal of Selected Top...· 0 citations
Vision-language multimodal learning has exhibited remarkable advantages in few-shot hyperspectral image (HSI) classification, where prompt learning effectively enhances feature-extraction accuracy and representation quality by guiding the model to focus on critical information. However, static prompts lack flexibility,...
Yu-Hang Li, Jin-Rong He, Xiang-Qing Zhang et al.· IEEE Transactions on Geoscie...· 1 citation
While the landscape of closed-set oriented object detection has been revolutionized in recent years, extending this success to the instances of unseen categories, the core challenge of open-vocabulary object detection (OVD), remains a formidable hurdle. Prevailing methods often resort to a dedicated student detector to...
Yan-Qing Yao, Xiang Yuan, Gong Cheng· IEEE Transactions on Geoscie...· 0 citations
Open-set few-shot face recognition (OSFSFR) requires both strong generalization from scarce samples and reliable rejection of unknowns. Existing methods discard base-class knowledge after pre-training, while pseudo-labeling suffers from noisy labels and base–novel distribution gaps. We propose the BEDCM, a unified fram...
Hyperspectral image (HSI) fusion aims to generate high-resolution HSI by integrating low-resolution hyperspectral data with auxiliary high-resolution sources (e.g., panchromatic (PAN), RGB, or MSI). While recent deep learning-based HSI fusion approaches have achieved promising results, they are typically designed for s...
Shaoxiong Hou, Jiahui Qu, Wen-Qian Dong et al.· IEEE Transactions on Geoscie...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.