Skip to content
Preprint

SciDocBench: A Workflow-Centered Benchmark and Data Pipeline for Scientific Document Understanding

Sep 2026 · 1 citation · 134 references
Computer Science

TL;DR

This work introduces SciDocBench, a workflow-centered benchmark targeting capability diagnosis with verifiable training-data construction for scientific-document assistants, and SciDocIR, a structured representation of scientific document objects, layout and cross-reference relations, and provenance.

Abstract

Scientific papers require models to integrate evidence across text, equations, figures, tables, code, and datasets while preserving its provenance. Beyond answer correctness, scientific reading requires verifiable outputs from operations such as evidence localization, definition extraction, and consistency checking. We introduce SciDocBench, a workflow-centered benchmark targeting these operations through 124 expert-authored and difficulty-screened questions across seven research-assistant capability groups, 19 subtasks, and five scientific domains. Each question is instantiated in four matched settings formed by pairing its bilingual variants with the All Images First and Markdown Interleaved document representations, yielding 496 evaluation instances. The strongest evaluated model, Claude-Opus-5, scores 62.6 out of 100, with remaining gaps in evidence localization, structured information extraction, cross-document synthesis, and robustness to document representation. To convert these diagnostics into scalable training signals, we introduce SciDocIR, a structured representation of scientific document objects, layout and cross-reference relations, and provenance. Using SciDocIR, we construct SciDocDataset, which contains 4K supervised fine-tuning instances and 10K reinforcement-learning instances, built on 14 verifiable training subtasks. Post-training Qwen3.6-27B on task-aligned data improves its SciDocBench score from 40.03 to 45.33 with supervised fine-tuning and to 45.74 with subsequent reinforcement learning. Both adapted models preserve DocVQA and InfoVQA performance and improve ChartQA accuracy over the original model by 0.80 and 3.40 points, respectively. Together, SciDocBench, SciDocIR, and SciDocDataset connect capability diagnosis with verifiable training-data construction for scientific-document assistants.

View source

Similar papers

Book Open access Oct 2026

SciClaim2Explore: Claim-Guided Exploration of Scientific Data Evidence

Digital libraries provide access to research articles and related datasets and code, if available. However, even if a scientific statement is linked to supporting data, turning such relations into executable analyses requires separate workflows. This gap is increasingly urgent as computational reproducibility remains a...

H. Ghaemi, Zakieh Alizadehsani, Markus Stocker · 0 citations
Conference Open access Oct 2026

SciFigBench: A Benchmark for Scientific Method Figure Understanding

The Scientific Figure Benchmark is introduced, a benchmark for evaluating core capabilities required for scientific method figure understanding, and SciFigCue, a training-free visual prompting module that externalizes arrow-induced dependencies as visual relational evidence for method-figure reasoning.

Li-Jian Wu, Zheng Li, Yuxuan Liu et al. · 0 citations
Review Open access Sep 2026

Accelerating metadata annotation in collaborative research centers: a hybrid AI workflow for biomedical entities

Abstract Background Collaborative Research Centers rely on FAIR-compliant, richly structured metadata, yet manual annotation is a major bottleneck. We implemented a search-augmented large language model (LLM) workflow within a local research data management system to pre-annotate biomedical entities, using human-in-the...

M. Watter, F. Engel, Aref Kalantari et al. · 0 citations
#artificial intelligence Preprint Oct 2026

From Papers to Mechanisms: An Evidence-Grounded Knowledge Substrate for Scientific Language Models

Scientific language models often access literature through untyped text chunks, which fragment the functional and evidential structure required for mechanism-rich questions. We introduce an evidence-grounded mechanism knowledge substrate that organizes scientific literature into provenance-linked evidence units, role-t...

Qiu-Hui Chen, Yi-Bo Liu, Tao Dai et al. · 0 citations
#artificial intelligence Review Open access Oct 2026

SurveyGraph: Generating Academic Surveys via Methods-Entity Knowledge Graphs

Most automated survey generation systems retrieve papers as flat documents, limiting their ability to capture research structure. We present SurveyGraph, a framework built around a Methods-Entity Knowledge Graph (MEKG) encoding method-level follow-on, competition, and shared-resource relations. The graph supports core-...

Yi-Fan Yuan, Yan-Cheng Zhu, Zuo-Yu Zhang et al. · 0 citations
Book Open access Oct 2026

A Provenance-First Discovery Layer for Research Gaps, Limitations, and Future Directions in ETDs

Electronic theses and dissertations (ETDs) often state research gaps, limitations, and directions for future work. These statements can support literature review and research planning, but they are difficult to find in long, structurally varied documents. We present a provenance-first pipeline that turns such statement...

Amr Aboelnaga, Hajra Klair, Hoda Eldardiry et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.