UVLM: A Universal Vision-Language Model Loader for Reproducible Multimodal Benchmarking

Abstract

Vision–Language Models (VLMs) have emerged as powerful tools for image understanding tasks, yet their practical deployment remains hindered by significant architectural heterogeneity across model families. This paper introduces UVLM (Unified Vision–Language Model), a pip-installable Python (v3.9+) package that provides a unified interface for loading, configuring, and running multiple VLM architectures on custom image analysis tasks. UVLM currently supports two major model families which differ fundamentally in their vision encoding, tokenization, and decoding strategies: LLaVA-NeXT and Qwen2.5-VL. The package abstracts these differences behind a single inference function and eliminates all architecture-specific code from the user’s workflow. UVLM is organized as eight modular Python components (model loading, dual-backend inference, response parsing, consensus validation, batch processing, prompt assembly, model registry, and utilities) and can be deployed in three modes: Google Colab for zero-install cloud access, local Jupyter notebooks for on-premises GPU use, and as a programmatic API for integration into automated pipelines. Key features include a multi-task prompt builder supporting four response types (numeric, category, boolean, text), a consensus validation mechanism based on majority voting, a flexible token budget (up to 1500 tokens) for custom reasoning strategies, and built-in truncation detection. The package is designed for extensibility: adding a new VLM family requires implementing one backend-specific inference section and adding entries to the model registry, without modifying any other module. An illustrative example on 120 street-view images across 16 model configurations is provided to demonstrate the software’s evaluation workflow.