Semisupervised Adaptation of Vision-Language Models for Image Classification
TL;DR
Results on the UC Merced (UCM) and NWPU benchmarks indicate that SE-CLIP significantly outperforms existing semi-supervised approaches and provides a viable solution for adapting VLMs to the remote sensing domain with minimal human intervention.
Abstract
Vision-language models (VLMs), like contrastive language-image pretraining (CLIP), have shown significant potential in handling natural images, yet their performance is often limited by the distinct characteristics of satellite imagery. While parameter-efficient adaptation techniques exist, their efficacy is frequently limited by the scarcity of annotated samples. In this letter, we propose self-evolutionary CLIP (SE-CLIP), a semisupervised framework designed for recursive label mining in scene classification. The approach follows a dual-phase pipeline, where an initial warmup on a few annotated seeds is followed by a recursive discovery phase that iteratively identifies high-confidence samples from unlabeled pools. To maintain the integrity of the evolving support set, we employ a class-balanced selection strategy that prevents the model from being dominated by easily learned categories. Results on the UC Merced (UCM) and NWPU benchmarks indicate that SE-CLIP significantly outperforms existing semi-supervised approaches. The framework provides a viable solution for adapting VLMs to the remote sensing (RS) domain with minimal human intervention.