PromptMatch: Semi-supervised visual prompt tuning for medical image classification.

Abstract

Visual prompt tuning (VPT) efficiently adapts foundation models for medical image classification but remains dependent on large labeled datasets. To overcome this, we explore semi-supervised learning (SSL) for label-efficient VPT. In VPT, while the backbone is well-regularized by large-scale pre-training to exhibit stable representations, the newly introduced prompts are initialized from scratch and remain sensitive to perturbations. Motivated by this discrepancy, we propose PromptMatch, a framework that explicitly regularizes the prompt space. By adversarially perturbing prompts on unlabeled data, it enforces prediction consistency across prompt variations to stabilize the adaptation process. Furthermore, we introduce (1) a Deep Prompt Supervision (DPS) module that applies deep supervision to prompt embeddings, utilizing previously neglected intermediate prompt information for SSL; and (2) an Instance-Specific Prompt (ISP) module, which generates instance-specific prompts to capture the unique characteristics of individual samples, overcoming the limitations of shared prompts in VPT. We evaluate PromptMatch on three real-world medical image classification datasets. Our extensive experiments demonstrate that it surpasses other SSL counterparts which apply perturbations to features or samples. PromptMatch reduces the reliance on labeled data, matching fully supervised performance with only 2.86% of the labels. Code is available at https://github.com/fdu-farm/PromptMatch.