Skip to content
Book Open access

PAIR: Prototype-Aware Inertia Rectification for Open-Vocabulary Multi-Label Recognition

Nov 2026 · 0 citations · 39 references

Abstract

Open-vocabulary multi-label recognition (OV-MLR) aims to predict multiple labels beyond seen training categories by transferring knowledge from vision-language models. Existing methods typically rely on either raw patch-level matching or a single global token, which makes them vulnerable to seen-category bias under seen-only supervision. In particular, patch features tend to collapse onto a few competitive seen categories, causing predictions to be dominated by seen-category responses, and limiting generalization to unseen labels. To address this issue, we propose PAIR (Prototype-Aware Inertia Rectification), a unified framework that reduces seen-category bias from both local and global perspectives. First, PURE (Prototype Recovery with Uncertainty-aware Region Enhancement) reorganizes patch tokens into teacher-guided latent prototypes, and refines them via transport-based prototype recovery, yielding more transferable local representations. Second, AIR (Alignment Inertia Rectification) builds a prototype-conditioned global representation, and rectifies global alignment through hard-negative shaping and local-to-global consistency regularization. By jointly rectifying local prototype learning and global representation alignment, PAIR provides a more balanced basis for open-vocabulary prediction. Extensive experiments on MS-COCO and NUS-WIDE show that PAIR consistently outperforms the strong OV-MLR baseline, demonstrating its effectiveness across both benchmarks.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.