Skip to content
Preprint

VIVAS: Vitalizing Visual Perception in VLM Pre-training via Vision-language Unified Autoregressive Supervision

Aug 2026 · 1 citation · 47 references
Computer Science

TL;DR

VIVAS is proposed, a framework built upon the unified token space paradigm, which introduces a dense-structural-semantic vision tokenizer, which expands the textual vocabulary into a unified vision-language vocabulary by incorporating a visual vocabulary.

Abstract

While Vision-Language Models (VLMs) demonstrate strong capabilities, they continue to suffer from a critical limitation: insufficient fine-grained visual perception, which fundamentally limits their multimodal understanding. We attribute this bottleneck to text-dominant optimization biases during pre-training, which encourage the model to overlook fine-grained visual details, thereby limiting the capability of multimodal understanding. We investigate that overcoming this bottleneck requires two key elements: (1) a unified token space paradigm that ensures stable training dynamics, and (2) a modality-aligned dense visual supervision signal enriched with both structural granularity and semantic information to capture critical visual representations. Based on these insights, we propose VIVAS, a framework built upon the unified token space paradigm, which introduces a dense-structural-semantic vision tokenizer, which expands the textual vocabulary into a unified vision-language vocabulary by incorporating a visual vocabulary. During pretraining, VIVAS performs vision-language unified autoregressive supervision over both visual details and linguistic content, thereby enhancing visual perception to improve multimodal understanding. Trained end-to-end on 12.4T tokens, VIVAS achieves state-of-the-art performance across 7 tasks and 39 multimodal benchmarks.

View source

Similar papers

Preprint Aug 2026

UVU: Improving Multimodal Understanding via Vision-Language Unified Autoregressive Paradigm

UVU effectively synergizes pixel-level visual perception with semantic-level visual understanding, internalizing visual reconstruction capabilities and unlocking the facilitative role of visual supervision in enhancing understanding in the pre-training stage.

Zhe-Han Kan, Xing-Hua Jiang, Yu-Bo Zhu et al. · 1 citation
Sep 2026

ConvCue: Complementary Visual Inductive Biases for Vision-Language Models

Results show that pretrained convolutional representations, when integrated through learned adaptation and fusion, can improve the visual understanding of existing VLMs without replacing their original visual encoders.

Zi-Xuan Lan, Shi-Chu Sun · 0 citations
Review

Toward Multimodal Understanding in Low-Level Vision: The Rise of Vision-Language Models

This paper presents the first comprehensive survey of VLM applications in low-level vision, proposing a unified taxonomy that organizes existing literature into two complementary paradigms, and identifies critical open challenges, including semantic-pixel misalignment and computational efficiency.

Feng-Yang Xiao, May May Liqeliqe Mar. Liqe Mar., Mariusz Dec et al. · 0 citations
#artificial intelligence Preprint Sep 2026

VoT: Vision-of-Thought for Unified Multimodal Representation Alignment

Vision-of-Thought (VoT), a framework that introduces a discrete visual-thinking layer between vision-language models (VLMs) and diffusion transformers (DiTs) that improves semantic alignment and provides a structured interface for interpretable and controllable generation is proposed.

Jingxiang Sun, Chao Liao, Zheng-Xiong Luo et al. · 0 citations
Preprint Aug 2026

When Vision Becomes Text: Visual Token Pruning via Cross-Modal Residual Guidance in VLMs

This paper revisit VLM inference and presents a new efficient guidance scheme that complements similarity-based guidance, and proposes Cross Modal Residual (CMR), a training-free visual token compression method that combines CMR, text-attention relevance, and residual-space diversity to retain task-relevant and complem...

Congyang Ou, Rui-Ke Song, Yang Zhou et al. · 1 citation
Conference Open access Sep 2026

Inductive Visual Logic for Few-Shot Out-Of-Distribution Adaptation in VLMs

Generative vision-language models (VLMs) such as Qwen-VL and LLaVA achieve strong zero-shot performance on tasks overlapping with their pretraining distribution, yet fail on specialized domains where the required discriminative features were never learned, a regime we term distant out-of-distribution (OOD). Standard ad...

Hung-Jen Chen, Yuek F. Ho, Ting-Yao Huang et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.