TUNGVLM: A High-Efficiency Structured-Output Multimodal Framework For Vehicle Recognition

Abstract

Vehicle make, model, and color recognition (MMCR) is a critical perception task in intelligent transportation systems (ITS), requiring both high accuracy and low latency. While recent vision-language models provide strong semantic representation capabilities, their autoregressive decoding process introduces semantic uncertainty and substantial computational overhead, limiting real-time deployment. To address these challenges, we propose TungVLM, a deterministic multimodal framework for efficient vehicle recognition. The proposed architecture reformulates the language backbone as a global semantic stabilizer to enhance semantic consistency and robustness, while a make-guided hierarchical classification head design guides the model toward fine-grained discrimination among visually similar vehicle models. To improve domain generalization, global-scale vehicle datasets are jointly optimized with real-world Taiwanese vehicle imagery. Experimental results on the unseen CHNTU surveillance dataset demonstrate an exact match accuracy of 81.29% with only 129 ms inference latency, outperforming GPT-4o (71.53%) and Florence-2 (50.13%). These results validate the effectiveness of TungVLM for real-time, high-throughput ITS applications.