Skip to content
Book Open access

DATP: Training-Free Depth-Aware Token Pruning for Vision-Language Models

Nov 2026 · Proceedings of the 35th ACM International Conference on Information and Knowledge Management · 0 citations · 4 references

Abstract

The massive redundancy of visual tokens in Vision-Language Models (VLMs) creates a critical computational bottleneck during large language model (LLM) prefill. Recent training-free pruning methods provide plug-and-play acceleration, yet their pruning decisions often rely on limited visual states or delayed in-LLM signals, which may underexploit both the layer-wise evolution of visual representations and the query-specific relevance of local regions. We propose DATP (Depth-Aware Token Pruning), a training-free early-pruning framework that selects compact visual tokens before they enter the LLM. DATP combines two complementary signals: Depth-aware Layer Aggregation (DALA), which aggregates multi-layer visual saliency trajectories to construct a high-recall structural prior, and Cross-modal Attention Gating (CMAG), which introduces a lightweight prompt-conditioned relevance signal to suppress visually salient but task-irrelevant tokens. Experiments on LLaVA-v1.5, LLaVA-NeXT, and Qwen2.5-VL show that DATP achieves a favorable accuracy-efficiency trade-off under aggressive token budgets. On LLaVA-v1.5, DATP preserves 95.7% of the full-token performance while retaining only 11% of the visual tokens. Across the reported multimodal reasoning and hallucination-sensitive benchmark settings, DATP generally outperforms the evaluated training-free pruning baselines.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.