SmolDocling, a compact 256M-parameter vision-language model (VLM), is fine-tune to perform end-to-end key-value extraction directly from document images, jointly solving identification, localization, and association in a single pass without OCR preprocessing.
Abstract
Document processing pipelines traditionally cascade optical character recognition (OCR) engines with downstream models for structured information extraction, leading to multi-stage error propagation. We fine-tune SmolDocling, a compact 256M-parameter vision-language model (VLM), to perform end-to-end key-value extraction directly from document images, jointly solving identification, localization, and association in a single pass without OCR preprocessing. We extend DocTags with specialized key, value, region, and link tags, enabling many-to-many relationships in a unified output sequence. To address data limitations, we design an augmentation pipeline combining synthetic form filling and graph-based crops that preserve complete key-value subgraphs. We further introduce a layout-aware evaluation framework extending text matching with spatial bounding box verification. On FUNSD, XFUND, and a large-scale private dataset, our model outperforms larger zero-shot VLM baselines under layout-aware evaluation, while being 27 times smaller than Qwen2.5-VL (7B) and over 5 times faster at inference. The model weights will be released publicly after publication.
Results show that token-level implicit layout analysis is an effective and practical approach for accelerating VLM-based OCR systems, and proposes LayoutLite, a lightweight plug-and-play module for efficient document OCR.
JZ-Tab is proposed, the first framework dedicated to the automated recognition of Jianzi Pu, which functions as an end-to-end structured visual information extraction system for mixed-script documents and introduces music-structured generation, injecting sequential regularities derived from symbolic music logic into the learning process.
Zehan Li, Fu Zhang, Zhijun Liu et al.· Proceedings of the 32nd ACM...· 0 citations
SAYRE is presented, a scene-aware document synthesis framework for generating scalable KIE training data without hand-crafted template design, and error analysis shows that synthesized training reduces field-level errors by improving schema-aware extraction over dense tables, business identifiers, and contract clauses.
Zhipeng Xu, Zulong Chen, Qing Liu et al.· 0 citations
M3Grounder sets a new state-of-the-art in grounded DocVQA, advancing from coarse boxes to hierarchical, fine-grained and contextually grounded mask evidence.
Venkata Kesav Venna, Sai Madhusudan Gunda, Jyothi Swaroopa Jinka et al.· 1 citation
Document parsing aims to transform unstructured documents into structured and machine-readable representations. Recent advances in Vision-Language Models (VLMs) have significantly advanced document parsing. However, existing approaches still face two major challenges. First, decoupled VLM-based methods heavily rely on accurate layout analysis, where geometric distortions in camera-captured documents can introduce cascading errors. Second, although end-to-end VLM-based methods alleviate the dependence on explicit layout detection, they often suffer from redundant generation, hallucinations, and insufficient structural reasoning in high-resolution scenarios. To address these challenges, we propose NaviDC-OCR, a unified framework for document parsing. NaviDC-OCR introduces deformation-aware learning to incorporate geometric perception into VLMs and proposes an adaptive sampling mechanism for complex layout representation. Furthermore, a content-structure decoupled learning strategy is developed to explicitly model formula grammars and table structures, enabling more effective structured representation learning. Extensive experiments demonstrate that NaviDC-OCR achieves state-of-the-art performance across diverse document parsing benchmarks. It obtains overall scores of 96.87, 88.53 and 78.41 on OmniDocBench v1.6, Wild-OmniDocBench, and PureDocBench, respectively, and ranks first in the ICDAR 2026 Sci-ImageMiner Challenge. These results validate the effectiveness and generalization capability of NaviDC-OCR in complex document parsing scenarios.
Peng Cai, Zhaofan Zou, Shifa Liu et al.· 0 citations
A five-tier architectural model is devised that incorporates a dedicated scene graph generation module tailored for archival data, aiming to enhance element detection and three-tier attention fusion module that integrates scene graph, text, and cross-modal features to ensure precise feature alignment.
