Skip to content
Conference Open access

Can Vision-Language Models Predict Atomic Structures of 2D Materials?

Oct 2026 · International Conference on Machine Learning and Applications · pp. 144-149 · 0 citations · 25 references

Abstract

Microscopy images contain rich structural information, yet inferring crystallographic properties directly from them remains a difficult inverse problem. In this work, we present a benchmark study of vision-language models for predicting key lattice parameters of 2D materials from simulated scanning transmission electron microscopy (STEM) images. Extending the MicroscopyGPT [1] framework, we fine-tune three open-source multimodal models: Llama3, Qwen2, and Gemma3, on paired STEM-image and text data derived from the JARVIS-DFT 2D (JV2D) and Computational 2D Materials Database (C2DB), and evaluate their ability to predict the lattice parameters a, b, and γ. The benchmark shows that these models achieve strong predictive performance and consistently surpass the previously reported results [1]. On the C2DB dataset, Llama3 delivers the best overall results, achieving MAE values of 0.32 Å for a, 0.39 Å for b, and 4.19° for γ. On JV2D, Qwen2 attains the best performance for a with 0.58 Å MAE, while Llama3 performs best on b and γ with 1.05 Å MAE and 9.66° MAE, respectively. Qwen2 also yields the highest number of correct stoichiometry predictions across both datasets and exhibits strong distributional agreement with the ground truth structural properties. Preliminary tests on a small number of experimental STEM images, including near-exact lattice prediction for MoS2, indicate encouraging real-world generalization. Overall, our results demonstrate that modern vision-language models provide an accurate and scalable route toward automated key lattice parameter predictions for 2D materials.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.