Multi-level visual-language models feature learning for generalizable anomaly detection
TL;DR
A Multi-Level Feature Learning (MLFL) framework is proposed to enhance the zero-shot capability of CLIP via hierarchical alignment and introduces a Generalized Prompt Distillation (GPD) module that distills object-aware prompts into a unified representation.
Abstract
Zero-shot anomaly detection (ZSAD) aims to identify anomalies in target datasets without accessing their samples. Although CLIP and other large-scale vision language models show strong generalization, their potential for multi-level feature extraction in ZSAD remains underexplored. To address this, we propose a Multi-Level Feature Learning (MLFL) framework to enhance the zero-shot capability of CLIP via hierarchical alignment. MLFL adopts a two-stage training paradigm: Multi-Level Text Prompt Tuning (MLTP) and Multi-Level Text-Image Feature Alignment (MLFA). MLTP learns object-agnostic and object-aware prompts tailored to different encoder blocks. MLFA aligns textual and visual features using linear layers for shallow blocks and a Deep Feature Alignment (DFA) module for deep blocks. To compress parameters and preserve semantics, we introduce a Generalized Prompt Distillation (GPD) module that distills object-aware prompts into a unified representation. Experiments on seven industrial datasets achieve state-of-the-art performance, and deployment tests on edge devices demonstrate the potential applicability of the framework in practical industrial scenarios.