Concept-Guided Prediction Refinement for Zero-Shot Style Classification
TL;DR
Concept-Guided Prediction Refinement (CGPR), a novel training-free framework for zero-shot painting style classification, is proposed, which consistently improves both Top-1 accuracy and balanced accuracy over vanilla zero-shot inference without requiring additional labeled data or model fine-tuning.
Abstract
Recent vision-language models (VLMs) have demonstrated impressive zero-shot image classification capabilities without requiring task-specific training. Although these models have shown promising performance in artwork understanding, zero-shot painting style classification remains challenging due to subtle stylistic differences and high visual similarity across artistic styles. Existing zero-shot approaches classify each painting independently based solely on image–text similarity, thereby overlooking semantic relationships among paintings that share common visual characteristics. To address this limitation, we propose Concept-Guided Prediction Refinement (CGPR), a novel training-free framework for zero-shot painting style classification. CGPR first constructs a concept hypergraph by associating paintings with discriminative visual concepts encoded by a pretrained VLM. The concept information is then combined with a visual $k$ -nearest-neighbor graph to establish concept-guided visual neighborhoods, through which prediction scores are iteratively refined according to the uncertainty of each prediction. Experimental results using four representative VLMs on the WikiArt and MultitaskPainting100k benchmarks demonstrate that CGPR consistently improves both Top-1 accuracy and balanced accuracy over vanilla zero-shot inference without requiring additional labeled data or model fine-tuning.