Skip to content
Preprint

Physics as the label for measuring and correcting materials reasoning in multimodal models

Sep 2026 · 0 citations · 57 references
Computer Science Physics

TL;DR

MatPCR is introduced, a label-free benchmark whose programmatic oracles check diffraction geometry through Bragg's law, scale bars, spectral peaks, and Materials Project-grounded checks of near-hull stability, computed band-gap class, and net magnetization.

Abstract

Vision-language and language models increasingly interpret materials data, yet benchmarks report that they hallucinate invalid properties and violate physical law. Evaluation matches final answers to scarce human labels, while discovery agents verify final proposals or density functional theory (DFT) execution. Neither measures the physical consistency of a model's reasoning chain. Materials data carries its own physics, making a large class of materials reasoning verifiable without annotation. We introduce MatPCR, a label-free benchmark whose programmatic oracles check diffraction geometry through Bragg's law, scale bars, spectral peaks, and Materials Project-grounded checks of near-hull stability, computed band-gap class, and net magnetization. We define the Physical-Consistency Rate over image and structure inputs; introduce Constraint-Grounded Self-Verification, an agentic loop whose gain survives self-refinement and equal-compute re-prompting controls; release an open verifier useful in distribution but near chance on all six held-out constraint types; and derive an exact identity for how oracle error displaces the reported rate.

View source

Similar papers

#artificial intelligence Preprint Sep 2026

PhysAlign: A Benchmark for Evidence-Grounded Role Alignment in Multimodal Physics Reasoning

Findings indicate that strong perception alone does not ensure reliable physical interpretation, exposing a distinct grounding bottleneck that is largely hidden by answer-level accuracy and highlighting the need for future models to better align recognized visual evidence with its physical meaning.

Ke-Cheng Liang, Hao Liu, Ze-Xin Chen et al. · 0 citations
Preprint Sep 2026

Separating perception from reasoning in vision-language models: a model-free render ceiling for crystal structures

The render ceiling is introduced, a model-free reference for benchmarks built by rendering known objects: inverting the frozen cameras and re-solving cross-view correspondence recovers exactly the answer the images support, so every point of a model's deficit belongs to the model.

Can Polat, Mustafa Kurban, E. Serpedin et al. · 0 citations
Preprint Sep 2026

GenPuzzle: Benchmarking Visual Reasoning in Image Generation Models

GenPuzzle provides a testbed for measuring progress from image rendering toward visual problem solving, and uses task-specific evaluation protocols that select the automatic judge by measuring agreement with human reference scores.

Chang-Peng Zhao, Yi-Ren Song, Jin-Peng Wang · 3 citations
Preprint Oct 2026

InfiniteWorld: Separating Visual Evidence from Familiar Physics

Physical reasoning requires adapting familiar expectations to the dynamics of the observed world. Existing benchmarks establish broad physical capabilities, yet an overall score leaves unclear whether answers follow visual evidence when familiar rules no longer apply. We introduce InfiniteWorld, a benchmark that pairs...

Tong Zhang, Yuxuan Cheng, Yun Peng · 0 citations
#artificial intelligence Preprint Oct 2026

Do Vision Models Learn Physical Constraints or Rendering Shortcuts? A Counterfactual Benchmark for Grounded Physical Consistency

Modern image editing models can satisfy a text instruction while breaking the physics of the edited scene. A new object may cast no shadow, a mirror may fail to reflect visible geometry, or an object may float above a surface that should support it. We study physical plausibility diagnosis, detecting whether an edited...

M. Esfahani, S. Salem, Mohammed Alser et al. · 0 citations
Preprint Oct 2026

From Reasoning Failures to Composable Video Spatial Intelligence

Spatial reasoning benchmarks evaluate vision-language models across diverse tasks, but task-level scores do not reveal which underlying capabilities account for success or failure. Each task requires recovering spatial evidence, representing geometry, and reasoning over it. We disentangle these capabilities by comparin...

Peng-Zhan Sun, Jun-Bin Xiao, Ramanathan Rajaraman et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.