Skip to content
Book Open access

Reliable Pseudo-Labeling via Pairwise Discriminative Comparison for Unsupervised Prompt Tuning

Nov 2026 · Proceedings of the 34th ACM International Conference on Multimedia · 0 citations · 3 references

Abstract

Vision-Language Models (VLMs) suffer significant performance degradation when deployed in specialized target domains, and existing adaptation methods are hindered by noisy pseudo-labels. We observe that VLMs typically retain the correct class among their top candidates but fail to distinguish it from visually or semantically similar confounders. This suggests that discriminative knowledge is not lost but diluted: the feature dimensions useful for separating one class pair can differ from those useful for another, whereas standard cosine similarity treats all dimensions uniformly. We therefore propose Pairwise Discriminative Comparison for Reliable Pseudo-Labeling (DisCo), which decomposes classification into pairwise comparisons in class-specific subspaces and aggregates their evidence for robust pseudo-label generation. An efficient average-mask score reduces per-sample computation from O(K2d) to O(Kd) while emphasizing dimensions that are consistently discriminative across class pairs. The refined pseudo-labels guide iterative prompt learning through knowledge distillation and subspace-aware prototype alignment. Experiments on three domain-adaptation benchmarks and eleven general-recognition datasets show that DisCo consistently outperforms prior unsupervised baselines and remains competitive with few-shot supervised methods. Code is available at https://github.com/ssbhss/DisCo.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.