Robustness of Zero-Shot CLIP to Contextual Variations in Industrial Anomaly Detection: A Factorized Sensitivity Analysis

Abstract

Automated optical inspection is crucial for maintaining quality in modern manufacturing. While deep-learning-based anomaly detection (AD) has shown promise, its deployment is often hindered by “contextual variations,” namely benign environmental changes such as lighting or camera noise that are irrelevant to product defects. Vision-Language Models (VLMs), such as Contrastive Language–Image Pre-training (CLIP), are expected to be robust to such variations because their dual-modality training aligns diverse visual concepts with high-level semantic descriptions. However, it remains unclear whether CLIP truly distinguishes object-localized defects from these environmental variations in industrial inspection. This paper presents a factorized evaluation framework to quantify the sensitivity of zero-shot CLIP to 17 contextual and object-localized variation types across five optical-path categories. Experiments on MVTec AD reveal that zero-shot CLIP exhibits strongly variation-specific behavior: background foreign objects can degrade the area under the receiver operating characteristic curve (AUROC) by over 14 points, whereas fog and blur can lower anomaly scores for normal images with contextual variations, producing superficial AUROC improvements. A complementary evaluation on RobustAD further shows AUROC degradation and high fixed-threshold false-positive rates under real acquisition shifts. Background masking, prompt variants, and alternative scoring methods did not fully mitigate these variation-dependent failure patterns. Further spatial and prompt-based analyses suggest that CLIP often recognizes contextual variations themselves, but its anomaly score is not consistently tied to defects on the target object surface. These findings highlight a practical robustness gap in zero-shot VLM-based industrial inspection and provide guidance for developing context-aware anomaly detection systems.