Skip to content
Open access

OpenGPL: Multimodal Open-Set Recognition via Dual-Level Pseudo-Unknown Generation and Positive–Negative Prompt Learning

2026 · IEEE Transactions on Geoscience and Remote Sensing · Vol 64, pp. 5525114-5525114 · 0 citations · 54 references

Abstract

The multimodal fusion of hyperspectral image (HSI) and light detection and ranging (LiDAR) data has advanced remote sensing (RS) classification. However, existing methods are mainly based on the closed-set assumption and are thus less effective in real-world open-set scenarios where unknown categories may appear during inference, due to the lack of representative unknown samples and the semantic gap across modalities. To address this, we propose OpenGPL, a novel multimodal open-set recognition (OSR) framework that integrates contrastive language-image pretraining (CLIP) with generative models. First, to tackle the problem of missing unknown samples, we propose a dual-level pseudo-unknown generation strategy. At the pixel level, we employ a diffusion model with a multimetric filtering mechanism to synthesize high-quality pseudo-unknown samples. At the feature level, we introduce a normalizing flow to sample features near the decision boundary, thereby tightening the feature distribution of known classes via an outlier exposure (OE)-inspired strategy. Second, to bridge the multimodal semantic gap, we design a positive–negative prompt learning mechanism. By maximizing the similarity between features and positive prompts while pushing them away from negative prompts, this scheme explicitly constructs a more discriminative semantic space, effectively enlarging the boundary between known and unknown categories. Finally, we adopt a dual-verification mechanism that fuses energy scores with similarities to negative prompts to accurately reject unknown classes. Comprehensive experiments conducted on three benchmark HSI-LiDAR datasets demonstrate that the OpenGPL achieves competitive performance in recognizing known classes and detecting unknown categories in open-set scenarios.

Read PDF

Similar papers

2026

Prompt-Guided Cross-Network Distillation for Open-Set Cross-Domain Few-Shot Hyperspectral Image Classification

Compared with cross-domain few-shot learning (CDFSL), open-set CDFSL (OS-CDFSL) is more challenging because it must accurately classify known classes across domains while reliably rejecting unknown samples. Under this setting, domain shift causes inconsistent spectral distributions for samples from the same class and c...

Chen Ding, Si-Rui Zheng, Yi-Zhou Dong et al. · 0 citations
Open access 2026

Few-Shot Open-Set Hyperspectral Pixelwise Classification With Spatial Aware Label Propagation

Pixelwise classification of hyperspectral images (HSI) is a fundamental task in remote sensing, but its deployment in real-world applications is often limited by the scarcity of labeled data and the presence of unknown classes at inference time. In this work, we propose Few-shot Open-set hyperspectral recognition from...

M. B. Pereira, J. D. dos Santos, Laurent Najman · 0 citations
Open access 2026

Dual-Prompt-Driven Cross-Modal Fusion Learning for Few-Shot Hyperspectral Image Classification

Vision-language multimodal learning has exhibited remarkable advantages in few-shot hyperspectral image (HSI) classification, where prompt learning effectively enhances feature-extraction accuracy and representation quality by guiding the model to focus on critical information. However, static prompts lack flexibility,...

Yu-Hang Li, Jin-Rong He, Xiang-Qing Zhang et al. · 1 citation
Open access 2026

Dual-Path Learning Toward Open-Vocabulary Object Detection in Remote Sensing Images

While the landscape of closed-set oriented object detection has been revolutionized in recent years, extending this success to the instances of unseen categories, the core challenge of open-vocabulary object detection (OVD), remains a formidable hurdle. Prevailing methods often resort to a dedicated student detector to...

Yan-Qing Yao, Xiang Yuan, Gong Cheng · 0 citations
Open access Sep 2026

An Energy-Guided Distribution Calibration Method for Open-Set Few-Shot Face Recognition

Open-set few-shot face recognition (OSFSFR) requires both strong generalization from scarce samples and reliable rejection of unknowns. Existing methods discard base-class knowledge after pre-training, while pseudo-labeling suffers from noisy labels and base–novel distribution gaps. We propose the BEDCM, a unified fram...

Xiao-Le Zhou, Chen-Xiao Liao, Hao Ouyang et al. · 0 citations
2026

UHFusion: Prompt-Guided Multi-Task Learning Network for Hyperspectral Image Fusion via Multi-Type Mixture-of-Experts

Hyperspectral image (HSI) fusion aims to generate high-resolution HSI by integrating low-resolution hyperspectral data with auxiliary high-resolution sources (e.g., panchromatic (PAN), RGB, or MSI). While recent deep learning-based HSI fusion approaches have achieved promising results, they are typically designed for s...

Shaoxiong Hou, Jiahui Qu, Wen-Qian Dong et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.