Skip to content
Preprint

ChemReporter: A Framework for Curating and Exporting Large-Scale Chemical Datasets for MLIP Training

Aug 2026 · 0 citations · 41 references
Physics

TL;DR

ChemReporter is a modular, method-agnostic framework that converts arbitrary molecular and materials datasets into a unified, queryable representation and exports the results directly into MLIP-ready training data.

Abstract

Training set quality and diversity are key determinants of the reliability of machine learning interatomic potentials (MLIPs), yet using massive datasets in full is often impractical and redundant, making intelligent data selection essential. A major bottleneck, however, is the lack of infrastructure for uniformly accessing, curating, and subsampling heterogeneous large-scale chemical datasets, which differ widely in structure, metadata, and file format. We address this gap with ChemReporter, a modular, method-agnostic framework that converts arbitrary molecular and materials datasets into a unified, queryable representation and exports the results directly into MLIP-ready training data. ChemReporter operates in three decoupled stages: processing, which parses raw datasets into a partitioned Apache Parquet repository enriched with structural, physical, and chemical metadata; querying, which filters and samples this repository via a CLI or Python API using arbitrary selection criteria, from simple physical constraints to custom, user-defined strategies; and exporting, which streams the selected subset into an HDF5 file ready for direct use in modern MLIP training frameworks. Throughout this process, every exported data point remains traceable to its original source entry, and dataset exports can be reliably reproduced given the same configuration and query database version. Because data is stored in a queryable, disk-backed format, ChemReporter can process datasets far larger than available memory, allowing it to scale to billion-structure datasets on standard compute infrastructure. ChemReporter is available on GitHub and PyPI under the Apache License 2.0.

View source

Similar papers

#artificial intelligence Open access Oct 2026

RxnExBench: Benchmarking Vision-Language Models for Structured Chemical Reaction Extraction from Scientific Literature

Screen Tables and Substrate Scopes are information-dense figures widely used in synthetic chemistry literature, yet automated extraction of structured reaction data from them remains challenging. We introduce RxnExBench, a multimodal benchmark for fine-grained reaction-level evaluation. Built from high-impact chemistry...

Jia-Xing Sun, Jun-Yuan Gao, Rui-Kang Zhang et al. · 0 citations
Conference Open access Oct 2026

Automated Feature Discovery in Large-Scale Databases via Deep Learning

A three-phase deep relational learning framework that combines neural representation learning with a Relational Feature Miner (RFM) to automate feature discovery from structured database schemas is introduced.

N. Prakash, Madhushree Kumari · 0 citations
Review Open access Nov 2026

SABIO-RK in 2027: enzyme kinetics data with modernized framework and enhanced data access.

SABIO-RK (https://sabiork.h-its.org) is a publicly available database providing expert-curated information on biochemical reactions, kinetic parameters, enzymes, and experimental conditions. For over 20 years, it has served as a central resource for experimentally measured enzyme kinetic data, as required for modeling...

Maja Rey, Ulrike Wittig, Xiao-Ming Hu et al. · 0 citations
Preprint Oct 2026

Bridging the Omics Divide: A Modular Relational Approach to Multi-Layer Biological Data Management

Background: Rapid growth of high-throughput molecular data demands systems for efficient retrieval, integration, and scalability across omics layers. Traditional file-based workflows hinder cross-modal analysis and reproducibility because of fragmented storage and ad-hoc querying. Few existing tools for genomic variati...

Alessandro Balestrucci, Donald Friggieri, Andrea Gariboldi et al. · 0 citations
Review Oct 2026

Automated Materials Data and Knowledge Extraction using Machine Learning and Large Language Models

Over the past decade, substantial efforts have been devoted to developing machine learning (ML) methods for extracting materials data and, where possible, materials knowledge directly from the scientific literature. This review traces the evolution of these methods from heuristic, rule-based pipelines through natural l...

A. Sundaram, B. P. Prusty, Rohit Batra · 0 citations
Book Open access Oct 2026

LLM-Based Semantic Metadata Enrichment for Findable Scientific Data Collections

This work generates natural-language descriptions for opaque columns with a language model at ingestion, from their names, statistics, and sample values, and stores them as curator-reviewable catalog metadata for search and a natural-language visual-discovery interface.

Caio N. Maia, A. N. Maia, Pedro L. P. Corrêa · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.