Skip to content

Hierarchical Prompt Learning for Hyperbolic Vision-Language Models

Sep 2026 · arXiv.org · Vol abs/2609.24276 · 0 citations · 49 references
Computer Science

TL;DR

Six hierarchical metrics and embedding analyses show that the hierarchical prompt learning plug-in for frozen hyperbolic VLMs produces more taxonomically consistent predictions and induces a hierarchy-consistent organization of parent, class, and image embeddings in hyperbolic space.

Abstract

Hyperbolic vision-language models (VLMs) represent image and text features in a geometry naturally suited to hierarchy, but their adaptation to downstream tasks has largely relied on fixed prompts. Existing prompt learning methods, meanwhile, treat class labels as a flat set and do not exploit available taxonomic structure. We address this gap with a hierarchical prompt learning plug-in for frozen hyperbolic VLMs. Given a fixed offline parent-class hierarchy, it augments a class prompt learner with a separate parent prompt learner, parent-level supervision, hyperbolic entailment regularization, and parent-feedback logit fusion. We instantiate the method with CoOp, CoCoOp and MaPLe, yielding HyPLO, CoHyPLO and MaHyPLO. Across the standard 11-dataset benchmark, all variants improve base-to-new generalization and cross-dataset transfer, and remain comparable to their prompt learning baselines under domain shift. Six hierarchical metrics and embedding analyses show that the method produces more taxonomically consistent predictions and induces a hierarchy-consistent organization of parent, class, and image embeddings in hyperbolic space. Its gains are largest when novel classes must be placed within a fixed taxonomy, and smallest for fine-grained confusions among sibling classes or shifts affecting only the image distribution.

View source

Similar papers

#computer vision Preprint Aug 2026

Hyper3-CLIP: Hierarchy-Conditioned Hyperbolic Vision-Language Training

Hyper3-CLIP is presented, a hierarchy-conditioned hyperbolic VLM that combines global, local, and global-local contrastive learning with query-conditioned visual pooling and improves R@5 and R@10 retrieval on COCO and Flickr, as well as multi-label classification on VOC and COCO, while remaining competitive on hierarch...

Matin Mahmood, Antonio Rueda-Toicen, Mohamed ElBassat et al. · 0 citations

ProCAP: Probabilistic Cross-Attentive Prompt Learning for Vision-Language Models

ProCAP is proposed, a probabilistic cross-attentive prompt learning framework that improves cross-modal interaction and training stability without updating any CLIP weights: it learns both visual and textual prompt tokens and links them through stacked bidirectional multi-head cross-attention so the two branches refine...

H. Abbas, Fatemeh Daneshfar, M. Abdar · 0 citations
#artificial intelligence Preprint Sep 2026

PETR: Prompt Ensembling with Training-free Routing for Vision-Language Models

Prompt learning efficiently adapts vision-language models (VLMs) to downstream tasks, but gains on seen classes often come at the expense of generalization to unseen classes. To address this limitation, we propose prompt ensembling with training-free routing (PETR), whose key innovation is a carefully designed dual-pro...

Wei-Han Cai, Hao Tan, Xin-Ping Gao et al. · 0 citations
Preprint Oct 2026

Unsupervised Long-Tailed Adaptation of Vision-Language Models

Adapting vision-language models to downstream tasks has achieved remarkable success by leveraging pseudo-labels generated from unlabeled data. Existing methods typically assume a uniform unlabeled data distribution, and thus the resulting pseudo-label distribution is likewise uniform. However, real-world data distribut...

Ke-Liang Chen, Ya-Xin Hou, Hui Liu et al. · 0 citations

Dual-Mode Low-Rank Learner with Bridge-Prototype Ensemble for Vision-Language Class-Incremental Learning

DuLBE is introduced, which couples dual-mode low-rank learning with a bridge-prototype ensemble classifier for exemplar-free CIL and achieves state-of-the-art CIL performance while retaining the high parameter efficiency of low-rank tuning.

Chi-Yuan He, Zi-Huan Qiu, Fan-Man Meng et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.