MicrobeQuest: A Context-Aware Multimodal Benchmark for Microbiology Information Extraction
TL;DR
This work introduces MicrobeQuest, the first comprehensive multimodal benchmark for microbial information extraction tasks, comprising 11,877 expert-validated, context-aware query–response pairs, and formalizes microbiology-specific IE as a class of tasks that require structured data extraction, joint multimodal understanding, and context-dependent scientific reasoning.
Abstract
AI for Science (AI4S) is rapidly advancing scientific discovery, yet its progress critically depends on the availability of large-scale, high-quality structured scientific data and reliable evaluation benchmarks. In microbiology, abundant multimodal information—spanning text, images, tables, and charts—is distributed across scientific literature and public databases, making information extraction (IE) essential for transforming raw data into structured scientific knowledge. We formalize microbiology-specific IE as a class of tasks that require structured data extraction, joint multimodal understanding, and context-dependent scientific reasoning. Although numerous IE models have been proposed, their effectiveness in microbiological settings remains difficult to assess due to the lack of a standardized, domain-specific evaluation benchmark. To address this gap, we introduce MicrobeQuest, the first comprehensive multimodal benchmark for microbial information extraction tasks, comprising 11,877 expert-validated, context-aware query–response pairs. Evaluations of 19 state-of-the-art IE methods reveal substantial performance variability and modality-specific challenges, establishing MicrobeQuest as a standardized evaluation framework for advancing AI-driven microbiological research. All benchmark resources are publicly available at https://github.com/yulab-pku/MicrobeQuest.