Skip to content
Preprint

FlexVDR: Learning to Allocate Matryoshka Representation Budgets for Visual Document Retrieval

Oct 2026 · 0 citations · 18 references
Computer Science

TL;DR

FlexVDR is introduced, a framework for query-adaptive budget allocation within a single retriever and a Budget Allocator predicts route-specific retrieval utility from early query features and combines these predictions with a configurable resource preference to select a route.

Abstract

Visual document retrieval relies on rich multi-vector representations to capture fine-grained evidence in document pages, but encoding and matching these representations can be computationally expensive. Existing compression and elastic representation methods expose smaller configurations, yet making multiple budgets available does not determine which configuration a query should use. The allocation decision must estimate the retrieval value of alternative representations before incurring their full computation, while accommodating different resource preferences. We introduce FlexVDR, a framework for query-adaptive budget allocation within a single retriever. We construct a joint family of routes over encoder depth, embedding width, and token count. Retrieval supervision and distillation train the compact routes, while a native bypass preserves the full-budget reference. A Budget Allocator predicts route-specific retrieval utility from early query features and combines these predictions with a configurable resource preference to select a route. Experiments across five backbones on ViDoRe V1, V2, and V3 public show that FlexVDR retains much of the native retrieval quality while selecting compact routes for individual queries. A configurable resource preference allows the same retriever to adjust its computation without retraining or switching models.

View source

Similar papers

Preprint Sep 2026

RandSlot: Learning Compact Visual Document Representations with Random Soft Tokens

Visual document retrieval requires expressive representations to match queries with evidence distributed across text, tables, and page layouts. Multi-vector representations capture fine-grained information, but storing and comparing many vectors introduces substantial retrieval costs. In this paper, we introduce RandSl...

De-Wen Guo, Shi Yu, Ling-Xiao Zhang et al. · 0 citations
Preprint Oct 2026

GARRET: Generative Feedback for Region Retrieval in Visual Documents

Multimodal retrieval-augmented generation over visually rich documents requires evidence at a finer granularity than entire pages. Existing visual retrievers typically index or return full pages, whereas training region-level retrievers often requires costly query-to-region annotations or noisy model-generated labels....

Wei-Jian Jian, Si-Yang Chen, Yu-Guang Yang et al. · 0 citations
Preprint Oct 2026

EVIE: Evidence-Vector-Informed Embeddings for Visual Document Retrieval

This work introduces EVIE (Evidence-Vector-Informed Embeddings), a family of native visual document retrievers integrating three key innovations: evidence-judged data governance, which uses a multimodal judge to identify answer-bearing positives and filter unreliable negatives, and Hierarchical agglomerative index comp...

Zi-Fei Wang, Wei Wen, Qian Ji et al. · 1 citation
#computer vision Sep 2026

ColNanoVDR: Document-Free Query Distillation for Multi-Vector Visual Document Retrieval via Optimal Transport

ColNanoVDR is presented, to the authors' knowledge the first framework to bring this document-free distillation to multi-vector VDR, and it is proved that the resulting alignment cost bounds the MaxSim score difference on every page.

Zhu Liu, Zi-Yi Wang, Yao Zhang et al. · 1 citation
Preprint Sep 2026

MAGIC: Marginal-Guided Compression with Optimal Transport for Efficient Visual Document Retrieval

Recent visual document retrieval (VDR) systems such as ColPali use multi-vector page embeddings, in which patch-level vectors enable fine-grained evidence matching but incur substantial index storage and MaxSim scoring overhead. Post-hoc merging offers a practical route to efficient VDR by reducing this cost without re...

Xu Yuan, Huai-Zhi Liu, Wen-Qi Fan et al. · 0 citations
Preprint Oct 2026

Aligning LLMs with Sparse Retrieval via Adaptive Sparse Supervision

While large language models (LLMs) offer strong semantic representations, their application to first-stage retrieval incurs substantial latency and storage costs. Neural sparse retrievers such as SPLADE provide efficient retrieval but rely on relatively compact encoders with limited representational capacity. To bridge...

Tong-Na Chen, Ming-Han Li, Guo-Dong Zhou · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.