Skip to content
Conference

Examining the Visual Capabilities of Multimodal Large Language Models for Automotive Applications

Jul 2026 · International Conference on Computer Aided Design · pp. 1-6 · 0 citations · 18 references

Abstract

Automatic recognition and classification of vehicle damages is an important research direction in modern computer vision and artificial intelligence, playing an increasingly significant role in industrial and practical applications. Traditional computer vision-based approaches can recognize and classify objects with high accuracy; however, achieving task-specific performance typically requires large amounts of annotated data, time-consuming training or fine-tuning, and extensive parameter optimization. This process is not only resource- and cost-intensive but also limits the rapid adaptability of the technology. The aim of this research is to investigate how effectively the latest Multimodal Large Language Models (MLLMs) can recognize types of vehicle damage in a zero-shot setting, i.e., without fine-tuning, and to evaluate how their performance can be further improved through prompt engineering and fewshot prompting. MLLMs have the advantage of being able to provide multiple forms of information from a single query and supplement their outputs with natural-language explanations. In contrast, traditional models are generally designed to perform only one predefined task. Therefore, within the framework of this project, the performance of a fine-tuned YOLO-based computer vision model is compared with that of MLLMs in vehicle damage classification. This comparison highlights a modern, data-efficient approach that achieves competitive performance through prompt engineering and in-context learning, eliminating the need for additional model training and opening new directions for automotive applications.

View source

Similar papers

Conference Aug 2026

An integrated YOLOv11 action recognition and vision-language model framework for intelligent kindergarten safety monitoring

Tests show that this real-time intelligent monitoring, safety early warning, and analysis framework can quickly and accurately detect personnel, identify various actions, issue safety early warnings in a timely manner, and complete intelligent scene analysis, meeting the actual usage needs of kindergartens.

Xiaojian Rao, Lin Fan, Yong Tian et al. · 0 citations
Open access Aug 2026

Developing and evaluating automated deep learning and human-in-the-loop vision–language systems for microplastic characterization

Microplastic (MP) pollution poses escalating environmental risks, demanding efficient and reproducible tools for morphological characterization of plastic particles. Traditional manual microscopy is labour-intensive, operator-dependent, and poorly suited to large-scale monitoring. This study presents a comparative evaluation of two distinct artificial intelligence paradigms for the analysis of optical microscope images of microplastics. The first paradigm is a domain-specific, multi-task deep learning (DL) classifier based on EfficientNet-B0 with transfer learning, trained on an in-house dataset of approximately 700 annotated microscope images to simultaneously predict microplastic shape/type (five classes), color (10 classes), and surface texture (two classes). The second paradigm employs the Claude Vision API as a zero-shot vision–language model (VLM), augmented with a structured human-in-the-loop (HITL) mechanism allowing domain experts to provide targeted guidance for ambiguous particles. Both systems were evaluated on an identical, independent test set using accuracy, macro-averaged precision, recall, and F1-score. The DL classifier achieved F1-scores of 91.2%, 88.5%, and 85.1% for shape/type, color, and texture classification, respectively. In contrast, the VLM achieved raw F1-scores ranging between 72 and 81% across the evaluated tasks, which improved substantially to approximately 84–89% following expert-guided refinement. These results demonstrate that the trained DL model excels in high-throughput, reproducible screening, while the VLM-HITL system offers enhanced interpretability and flexibility for ambiguous cases. This comparative framework, deployed as a freely accessible web application via Hugging Face Spaces, provides practical insights into the deployment trade-offs between domain-specific and generalist AI approaches for environmental microplastic analysis.

Mahmoud Sami, Sara El-Kerkary, Marina S. Fawzy · 0 citations
Open access Aug 2026

APPLICATION OF LLM + ZERO-SHOT LARGE MODELS FOR FRUIT OBJECT DETECTION

Efficient and flexible agricultural image annotation is crucial for intelligent crop monitoring in smart agriculture, yet conventional detection models are limited by fixed class labels and require extensive manual annotations. This study presents a zero-shot annotation framework that integrates OWLv2, Google’s second-generation open-vocabulary vision model, with large language models (e.g., GPT-3.5, DeepSeek V1) to enable multilingual, natural language-driven fruit recognition in smart agriculture. A user-friendly interface was developed to support individual or batch image annotation with adjustable sensitivity to meet diverse field requirements. Experimental evaluations demonstrated the framework's strong generalizability and semantic understanding capabilities, allowing recognition of unseen fruit categories and attributes such as ripeness or color. The system significantly reduces annotation time and labor costs, while enhancing accessibility through natural language interaction. To ensure a robust evaluation of generalizability, a cross-domain protocol was employed using a novel dataset from 2025. Results showed that OWLv2 achieved an F1-score of 0.80 and an mAP of 0.8301, significantly outperforming the pre-trained YOLO11 (F1: 0.74, mAP: 0.60) in zero-shot scenarios. OWLv2 exhibited superior flexibility and required no task-specific dataset retraining, although its computational demands remain higher than lightweight models like YOLO11. Notably, while the LLM (DeepSeek) introduced a total one‑time API latency of 598.3 ms (called only once for processing multiple images). the actual core computational latency of OWLv2 was only 257.7 ms per image. Despite a total processing time of 891.9 ms (including visualization output), the framework demonstrates superior recall (0.9080) and semantic flexibility without retraining. These results verify the enormous application potential of OWLv2 and similar zero-shot models in agriculture, providing scalable solutions for automated annotation, real-time monitoring, and large-scale data collection.

Ying-Dong Qin, Hao-yu Song, Jing-Yi Li et al. · 0 citations
Review Aug 2026

Deep Learning–Driven Visual Intelligence for Chemical Instrument Monitoring: A Scoping Review From an Expert Systems Perspective

Automated visual recognition in industrial environments has become a key enabler for intelligent monitoring systems, particularly in safety‐critical domains such as chemical plants. Unlike general industrial vision tasks, chemical instrument monitoring operates under stringent constraints, including harsh environments, legacy non‐digital devices, and high reliability requirements for decision‐making. In this context, deep learning‐based visual recognition should not be viewed as an isolated perception task, but rather as a critical component within expert systems that support operational decision‐making. This paper presents a comprehensive scoping review of deep learning–driven visual intelligence for industrial applications, with a specific focus on chemical instrument monitoring from an expert systems perspective. A total of 127 studies published between 2015 and 2025 are systematically analysed, covering object detection, image segmentation, and optical character recognition (OCR), along with their integration into practical monitoring pipelines. Beyond categorizing models, this review emphasizes the evolution from standalone perception models toward system‐level solutions that incorporate edge deployment, data‐efficient learning, and explainability. Key challenges are identified, including data scarcity due to proprietary industrial datasets, limited model generalization across dynamic environments, computational constraints in real‐time deployment, and the lack of interpretability required for safety‐critical decision support. Importantly, this review highlights the gap between high‐performing vision models and their reliable integration into expert systems for chemical operations. Emerging trends are discussed, including synthetic data generation, domain adaptation, foundation models (e.g., SAM and open‐vocabulary detection), and multimodal fusion, with a critical evaluation of their applicability and limitations in chemical instrument scenarios. By reframing industrial visual recognition within the broader context of expert systems, this work provides a structured understanding of current capabilities and outlines future directions for developing trustworthy, efficient, and human‐centric intelligent monitoring systems in chemical industries.

Qirui Li, Hai-Yang Luo, Zhiping Peng et al. · 0 citations