YOLO-OCR vs. Vision–Language Models Under Limited Data Availability: A Car-Plate Recognition Case Study

Abstract

Automatic license-plate recognition (ALPR) systems typically rely on a two-stage pipeline that combines object detection with optical character recognition (OCR), whereas recent vision–language models (VLMs) enable direct image-level recognition without task-specific training. This paper presents a comprehensive comparison between these paradigms under limited target-domain data availability. We evaluate a plate-specific YOLO11x detector combined with four OCR backends and five locally hosted open-weight VLMs, without training or fine-tuning on a study-specific Serbian dataset containing 435 images and 643 visible license plates, annotated with image-level plate counts, registration transcriptions, and plate-level bounding boxes. Performance is assessed using strict end-to-end success, exact-match accuracy, character error rate, readable-plate recall, plate-count accuracy, and inference speed. The comparison is further validated on the OpenALPR-EU and UC3M-LP benchmarks. Finally, we investigate the effect of increasing target-domain annotation budgets by incrementally fine-tuning the YOLO detector on UC3M-LP to determine when a conventional detector–OCR pipeline overtakes zero-shot VLMs. The results demonstrate complementary strengths rather than a universal winner. YOLO11x combined with FastPlateOCR achieves the highest transcription accuracy and is approximately 13× faster than the best-performing VLM; the three general-purpose OCR backends collapse on identical detections, so these figures characterize the strongest available plate-specific recognizer rather than detector–OCR pipelines in general, while Qwen3-VL provides superior image-level plate-count accuracy in the zero-shot setting, an advantage concentrated in single-plate scenes. The annotation-budget study shows that a few hundred labeled localization annotations substantially improve the conventional pipeline, enabling it to surpass zero-shot VLMs in character recovery, although VLMs retain an advantage in image-level plate counting. These findings provide practical guidance for selecting ALPR systems according to annotation availability, deployment constraints, and application objectives.