ReFigBench, a benchmark and evaluation framework built on 1,000 real overview figures retrieved from arXiv papers with full provenance, is introduced, exposing the tension between fidelity and editability as the central challenge for practical multimodal document agents.
Abstract
Multimodal coding agents are expected to turn visual inputs into usable artifacts, and they act through a harness, the layer of tools, context management, and execution environment around the model. Existing evaluations often isolate short tool calls, API traces, or screenshot resemblance, and a low score under these proxies cannot say whether the model saw poorly, planned poorly, or was failed by its harness. We study scientific overview figure reconstruction, an agent task in which a source image must become an editable PowerPoint slide that preserves text, topology, layout, and native document structure. We introduce ReFigBench, a benchmark and evaluation framework built on 1,000 real overview figures retrieved from arXiv papers with full provenance. Coding agents from four model families reconstruct every figure under two workflows, direct code generation and a specialized PPTX workflow, and the strongest model runs inside two commercial harnesses, yielding ten configurations. Evaluation combines deterministic artifact checks, repeated automated scoring by judges from two model families, and blinded human comparisons. Perception remains a bottleneck that iterative rendering only partly repays. Whether workflow effort converts into quality depends on the model together with its harness, since the same model gains from the specialized workflow inside one harness and loses inside the other, and the harness shifts scores even under an identical direct prompt. The specialized workflow erases native connectors in every configuration, human judges still prefer its renderings in most matchups, and even the strongest agent falls short of the rubric ceiling. These results expose the tension between fidelity and editability as the central challenge for practical multimodal document agents.
SciFigure2Code is introduced, an AI-reconstructed benchmark that instead evaluates presentation recovery: generating editable Python programs that preserve how a scientific panel is arranged and read, and provides an auditable testbed for agents that construct editable, visually faithful scientific figure presentations...
Wen-Tao Li, Yi-Bo Wu, Yi-Zhe Chen et al.· 0 citations
This work introduces Code4Scene, a benchmark of 190 Unreal Engine cases built from human-assembled scenes that evaluates coding agents on two complementary settings under a shared execution interface, and exposes a gap between plausible 3D generation and reliable spatial reasoning and state control.
Xiao-Kang Ye, Siddhant Hitesh Mantri, Zi-Meng Chen et al.· arXiv.org· 3 citations
PPTBench establishes a measurable testbed for studying visual coding and advancing agents toward more reliable visual creation, and shows that agents can generally produce valid slide files, but struggle to produce high-quality reconstructions that faithfully recover the semantics and visual structure of the target.
Xiao-Qiu Wang, Yi-Zhe Chi, Wen-Yi Li et al.· 0 citations
Screen-recorded web editing requests contain weak deictic expressions such as ``this''and ``there,''whose referents depend on speech, cursor trajectories, page state, and edit history. Such requests require intent recovery beyond the explicit specifications assumed by many existing web-editing benchmarks. We introduce...
This work introduces VisEditBench, a benchmark of 1,395 human-annotated visualization code-editing tasks grounded in realistic visualization workflows and failure cases, and proposes VisEditAgent, a render-grounded editing framework that iteratively generates, executes, validates, and refines candidate edits.
Turning a natural-language question into a correct, publication-ready data visualization usually takes several rounds of coding, inspection, and debugging. Large language models can draft plotting code, but a single-shot model call has no way to run the code, look at the resulting image, recover from execution errors,...
The new ChartNet training dataset could improve the accuracy of vision-language models that help analyze business trends or interpret scientific figures.