Skip to content

Making Large Vision–Language Models Better Few-Shot Learners

Sep 2026 · IEEE Transactions on Image Processing · Vol 35, pp. 10286-10299 · 0 citations · 85 references
Medicine Computer Science

Abstract

Few-shot classification (FSC) aims to emulate the human ability to rapidly learn new concepts from a handful of examples. Large Vision-Language Models (LVLMs), with their rich prior knowledge and powerful visio-linguistic understanding capabilities, are emerging as a highly promising paradigm for FSC. This paper investigates the challenges of applying LVLMs to FSC tasks and identifies two core learning bottlenecks. The first is the model’s inherent positional bias, such as favoring last options in textual choices. The second is the insufficient learning problem, where the model tends to rely on its vast pre-trained knowledge, rather than genuinely generalizing new knowledge from the support samples provided in the current task. Furthermore, the support-query paradigm in FSC presents a significant efficiency challenge, as the long sequence structure resulting from multiple image-text inputs leads to high inference costs. To address these challenges, we first correct the model’s positional bias by constructing positionally balanced meta-tasks for instruction fine-tuning. To enhance the model’s generalized learning, we introduce a semantic-guided background generation strategy to break spurious visual correlations between foreground and background. Furthermore, we propose a hard negative mining strategy to organize more efficient instruction fine-tuning, compelling the model to focus on more discriminative feature information. Finally, we apply a two-stage token pruning strategy for inference acceleration, significantly reducing the model’s inference latency. Experiments demonstrate that our proposed methods achieve superior performance and efficiency across multiple FSC benchmarks. Code is released at https://github.com/xyfore/TLVLMG

View source

Similar papers

Conference Open access Sep 2026

Inductive Visual Logic for Few-Shot Out-Of-Distribution Adaptation in VLMs

Generative vision-language models (VLMs) such as Qwen-VL and LLaVA achieve strong zero-shot performance on tasks overlapping with their pretraining distribution, yet fail on specialized domains where the required discriminative features were never learned, a regime we term distant out-of-distribution (OOD). Standard ad...

Hung-Jen Chen, Yuek F. Ho, Ting-Yao Huang et al. · 0 citations
Preprint Aug 2026

What Can Low Resource Languages Learn From Each Other?

This work proposes PSMC (Pre-train, Specialize, Merge, Merge, and Co-train), a data-efficient framework that capitalizes on a cross-script "transfer effect" to fuse language-specific experts into a unified, high-performance multilingual back- bone.

P. Achyuth, Kahaan Shah, Chetan Arora · 0 citations
Preprint Aug 2026

MLLMCLIP: Feature-Level Distillation of MLLM for Robust Vision-Language Representations

Compared to prior CLIP-enhancement methods, MLLMCLIP achieves state-of-the-art compositional accuracy while delivering consistent gains on standard zero-shot classification and image-text retrieval, showing that feature-level distillation strengthens both compositional and general vision-language representation capabil...

Jongsuk Kim, Qi-Yu Wu, Zhuoyuan Mao et al. · 0 citations
#artificial intelligence Preprint Oct 2026

What Makes Synthetic Hard Negatives Work in Vision-Language Pretraining?

Synthetic hard negatives generated in the representation space have proven effective for unimodal self-supervised learning, but transferring this idea to vision-language pretraining is not straightforward. We analyze six representation-space synthesis strategies and identify two failure modes in their transfer to visio...

Nikos Giakoumoglou, Paschalis Giakoumoglou, Andreas Floros et al. · 0 citations
#artificial intelligence Preprint Sep 2026

Learning to Learn from Context: Synthetic Training from Perturbed Public Documents

This work attempts to make use of high-quality public documents with small perturbations and empirically finds that LLMs can successfully generate context-dependent reasoning traces and answers, which are then used to train a student model that substantially improves the performance on CL-bench.

Hao Wu, Yang Xiao, Yu-Song Sun et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.