Sep 2026· IEEE Transactions on Image Processing· Vol 35, pp. 10286-10299· 0 citations· 85 references
MedicineComputer Science
Abstract
Few-shot classification (FSC) aims to emulate the human ability to rapidly learn new concepts from a handful of examples. Large Vision-Language Models (LVLMs), with their rich prior knowledge and powerful visio-linguistic understanding capabilities, are emerging as a highly promising paradigm for FSC. This paper investigates the challenges of applying LVLMs to FSC tasks and identifies two core learning bottlenecks. The first is the model’s inherent positional bias, such as favoring last options in textual choices. The second is the insufficient learning problem, where the model tends to rely on its vast pre-trained knowledge, rather than genuinely generalizing new knowledge from the support samples provided in the current task. Furthermore, the support-query paradigm in FSC presents a significant efficiency challenge, as the long sequence structure resulting from multiple image-text inputs leads to high inference costs. To address these challenges, we first correct the model’s positional bias by constructing positionally balanced meta-tasks for instruction fine-tuning. To enhance the model’s generalized learning, we introduce a semantic-guided background generation strategy to break spurious visual correlations between foreground and background. Furthermore, we propose a hard negative mining strategy to organize more efficient instruction fine-tuning, compelling the model to focus on more discriminative feature information. Finally, we apply a two-stage token pruning strategy for inference acceleration, significantly reducing the model’s inference latency. Experiments demonstrate that our proposed methods achieve superior performance and efficiency across multiple FSC benchmarks. Code is released at https://github.com/xyfore/TLVLMG
Generative vision-language models (VLMs) such as Qwen-VL and LLaVA achieve strong zero-shot performance on tasks overlapping with their pretraining distribution, yet fail on specialized domains where the required discriminative features were never learned, a regime we term distant out-of-distribution (OOD). Standard ad...
Hung-Jen Chen, Yuek F. Ho, Ting-Yao Huang et al.· European Conference on Compu...· 0 citations
This work proposes PSMC (Pre-train, Specialize, Merge, Merge, and Co-train), a data-efficient framework that capitalizes on a cross-script "transfer effect" to fuse language-specific experts into a unified, high-performance multilingual back- bone.
P. Achyuth, Kahaan Shah, Chetan Arora· 0 citations
Compared to prior CLIP-enhancement methods, MLLMCLIP achieves state-of-the-art compositional accuracy while delivering consistent gains on standard zero-shot classification and image-text retrieval, showing that feature-level distillation strengthens both compositional and general vision-language representation capabil...
Jongsuk Kim, Qi-Yu Wu, Zhuoyuan Mao et al.· 0 citations
Synthetic hard negatives generated in the representation space have proven effective for unimodal self-supervised learning, but transferring this idea to vision-language pretraining is not straightforward. We analyze six representation-space synthesis strategies and identify two failure modes in their transfer to visio...
Nikos Giakoumoglou, Paschalis Giakoumoglou, Andreas Floros et al.· 0 citations
This work attempts to make use of high-quality public documents with small perturbations and empirically finds that LLMs can successfully generate context-dependent reasoning traces and answers, which are then used to train a student model that substantially improves the performance on CL-bench.
Hao Wu, Yang Xiao, Yu-Song Sun et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.