Hierarchical Document Parsing and Multi-Agent Collaborative Decision-Making for Long-Document Visual Question Answering
Abstract
Long-document visual question answering requires models to locate sparse question-relevant evidence across many pages and understand fine-grained textual and visual content. Processing an entire document at high resolution is computationally expensive, whereas reduced visual fidelity and irrelevant pages can impair both answer generation and evidence localization. To address these challenges, we propose HiDoc-AV, a framework that integrates hierarchical document parsing with collaborative decision-making between multiple agents and a verifier. It uses coarse-to-fine, multi-resolution processing to locate evidence efficiently while preserving detailed visual information for answering. Its unified pipeline coordinates low-resolution evidence-page retrieval, necessity-aware pruning, high-resolution evidence-grounded answering, multi-agent candidate generation, semantic consensus, and verifier arbitration. This comprehensive design tightly couples efficient document exploration with reliable cross-page reasoning rather than treating retrieval and answering as isolated stages. Heterogeneous VLM agents propose evidence--answer pairs, and a verifier arbitrates their predictions to improve decision reliability. For efficient deployment, we further distill the framework into a compact HiDoc-AV-Distill student that requires no multi-agent collaboration at inference time. Experiments on a multilingual long-document VQA benchmark show that the full HiDoc-AV framework achieves an Overall Score of 0.87, outperforming the strongest individual VLM baseline by 15 absolute points. HiDoc-AV-Distill retains most of this capability with an Overall Score of 0.85, demonstrating the effectiveness and deployability of the proposed framework.