Skip to content

VideoMM: Adaptive Macro-Micro Inference for Efficient Video MLLMs

Sep 2026 · arXiv.org · Vol abs/2609.16722 · 1 citation · 41 references
Computer Science

TL;DR

VideoMM is introduced, which marks a paradigm shift from model-centric downsizing to adaptive perceptual granularity and accelerates inference by 2.73 times over current leading methods, establishing a highly scalable paradigm for long-video understanding.

Abstract

Scaling Multimodal Large Language Models (MLLMs) to long-form video understanding is bottlenecked by the explosion of visual tokens, which saturates context windows and incurs prohibitive costs. Current solutions predominantly rely on auxiliary models for token reduction but face a fundamental dilemma: lightweight encoder-driven approaches often overlook critical semantic information, whereas heavyweight MLLM-driven reduction negates the efficiency gains. {In this work, we identify a more fundamental inefficiency underlying this dilemma: while fine-grained visual details are essential for detailed understanding, they are largely redundant for the preliminary task of selecting semantically relevant regions. } Motivated by this, we introduce \textbf{VideoMM}, which marks a paradigm shift from model-centric downsizing to adaptive perceptual granularity. Specifically, our framework {decouples selection from reasoning} by executing semantic filtering on a cost-effective \textit{Macro Proxy} (derived from downscaled frames), and projecting the selected regions onto high-fidelity \textit{Micro Tokens} for detailed understanding only when necessary. Extensive evaluations show that VideoMM significantly outperforms existing solutions. It achieves a 6.13$\times$ speedup and a 7.4\% accuracy gain over full-context baselines on LongVideoBench, and further accelerates inference by 2.73$\times$ over current leading methods, establishing a highly scalable paradigm for long-video understanding. Our code is available at: https://github.com/adfh917k/VideoMM.

View source

Similar papers

#artificial intelligence Preprint Oct 2026

VETO: Video Efficient Token Optimization for Vision Language Models

The key design insight is hierarchical ordering: by first compressing spatial dimensions, VETO drastically reduces the cost of subsequent global temporal matching, bypassing the efficiency wall of single-axis approaches, with an advantage that grows with modern fully-fused attention infrastructure.

Gueter Josmy Faure, Hao-Ping Wang, Min-Hung Chen et al. · 0 citations
#computer vision Preprint Aug 2026

PACE: A Unified Condense-and-Extract Paradigm for Fast VLM Inference

PACE (Pixel-Adaptive Condense and Extract), a training-free inference framework that accelerates both the vision encoder and the Large Language Model (LLM) via a unified Condense-and-Extract paradigm, is proposed.

Jun-Jie Liu, Shengyuan Ye, Xu Chen · 0 citations
#artificial intelligence Preprint Sep 2026

KeyRec: Bounded Visual Memory for Streaming and Long-Video Understanding

KeyRec is a training-free framework for constructing bounded visual memory that preserves fine-grained recent observations in a visual cache and organizes historical evidence into a structured event bank and achieves the best compressed performance in 13 of 15 settings using only 10\% of the dense decoder-facing visual...

Zi-Han Chen, Xuejian Rong, Xiao-Juan Wang et al. · 0 citations
#machine learning Preprint Sep 2026

GleanVID: Complementary Token Selection for Efficient Video Large Language Models

Video Large Language Models (VideoLLMs) have achieved strong video understanding capabilities but incur substantial inference overhead due to the large number of visual tokens. Existing VideoLLM token compression methods largely rely on selection-independent scoring, overlooking cross-frame complementarity and conseque...

Shuo Yang, Chang-Bai Li, Rui Tang et al. · 0 citations
#machine learning Conference Open access Oct 2026

V-CoLA: Vision Token Compression with Linear Attention

Vision-language models (VLMs) have demonstrated impressive capabilities but suffer from substantial computational overhead, as vision tokens dominate the input sequence. This motivates vision token compression as a key direction to alleviate the burden. However, with the emergence of hybrid architectures incorporating...

Hao Jiang, Yi-Ru Mao, Tian-Peng Bu et al. · 0 citations
Sep 2026

AdaCompVL: Adaptive Compression of Spatiotemporal and Cross-Modal Redundancy for Efficient Video–Language Learning

Multimodal large language models (MLLMs) have recently extended from static image understanding to video comprehension, but representing videos as frame-level token sequences incurs substantial computational overhead. Existing visual token compression methods typically rely on uniform sampling or single-dimension redun...

Jian-Xin Ma, Shi-Bo Jin, Lu-Juan Dang et al. · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.