TabgenQA: A Platform for Non-Relational, Multi-Tabular, Numerical QA Benchmark Generation and LLM Evaluation

Abstract

Large language models (LLMs) are widely used for tabular data analysis, but their performance often does not transfer from standard Table QA benchmarks to real-world settings, where tables are typically non-relational, with hierarchical headers, merged cells, inconsistent units, noise, and implicit cross-table dependencies. We present \name, an interactive system built on top of \gradino for generating verifiable non-relational multi-table QA benchmarks and evaluating LLMs on them. Users configure domain-specific benchmark settings, inspect and refine generated instances, and compare API-based or locally deployed models through live leaderboards and error analysis views. Across procurement, failure diagnosis, and benchmark curation scenarios, \name provides an end-to-end workflow for deployment-oriented assessment of LLMs on real-world tabular data.