Aug 2026· Machine Learning and Knowledge Extraction· 0 citations· 55 references
TL;DR
This review is, to the authors' knowledge, the first organised around the current generation of tabular foundation models, and taxonomises the architectures by pretraining regime, maps the capability space across five axes, isolates the language-model-on-tabular strand for prediction, feature engineering and generation, and summarises openness and deployment.
Abstract
Tabular foundation models (TFMs) have moved tabular machine learning from per-dataset training towards amortised in-context inference, fitting a small-to-medium table in a single forward pass without a training run. The 2024–2026 release train, the TabPFN and TabICL lines and challengers such as Mitra, LimiX and Orion, has produced a generation whose architectures, capabilities and limits are documented mainly in preprints, while existing surveys treat these models as a subsection of tabular deep learning or of language-model table understanding. This review is, to our knowledge, the first organised around the current generation. From a corpus of 961 screened records and 98 retained studies, it taxonomises the architectures by pretraining regime, maps the capability space across five axes, isolates the language-model-on-tabular strand for prediction, feature engineering and generation, and summarises openness and deployment. A dedicated critical synthesis then reads the reported capabilities against independent evidence: on the studies reviewed here, tree-based and deep models retain the lead across 142 curated datasets that go beyond the standard independent and identically distributed setting; on 112 datasets, the models attain the highest accuracy but weaker conditional coverage than gradient-boosted trees; and robustness under feature shift, fairness and generation quality remain open. Amortised in-context prediction is thus a working paradigm whose independent evidence has yet to match its benchmark claims.
This work adopts an alternate approach, sticking with row-based attention while incorporating long context pre-training to eliminate the need for retrieval in TabDPT-Turbo, a model that provides comparable default performance to TabDPT v1.1 on TabArena-Lite, CC18, and CTR23, at orders of magnitude faster.
Rasa Hosseinzadeh, Alex Labach, Zexin Xue et al.· 2 citations
The results show that the LLM's capability dissolves with dimension in a way no noise-corrupted classical learner mimics - which explains why LLMs, so capable elsewhere, keep losing to fifty-year-old baselines on tables, while leaving the mechanism of the prediction as an open question.
A task-centric, retrieval-based perspective is offered for how TFMs generalize: it is believed that tabular in-context generalization is largely retrieval-based, and good models are those that learn to identify relevant examples in the provided context and aggregate them well.
Nour Shaheen, Junwei Ma, Alex Labach et al.· 1 citation
EXAONE Tabular is a compact tabular foundation model family for classification and regression via in-context learning, producing predictions without dataset-specific gradient updates. Pretrained exclusively on a synthetic structural-causal-model (SCM) prior, its central contribution is an architecture-centered redesign of tabular in-context learning. Rather than compressing features into a fixed row embedding before a separate row-level learner, EXAONE Tabular interleaves feature-axis attention within each item with support-conditioned item-axis attention within each feature at every Transformer layer, mediated by item-summary and feature-summary tokens. Across four public benchmarks, EXAONE Tabular combines strong predictive performance with high efficiency. On TabArena, its 20.81M-parameter classification model ranks first overall, surpassing tuned ensembles and 4-hour AutoML pipelines, while regression reaches the performance regime of the 1.64B-parameter TabFM at roughly 1/11 the inference cost. On BCCO and TALENT, EXAONE Tabular ranks second in classification and first in regression. On ScoringBench, it achieves the best mean rank for both point-estimation and predictive-distribution quality, leading the $R^2$, RMSE, and CRPS evaluations. Together, these results establish EXAONE Tabular as a state-of-the-art compact tabular foundation model family, combining strong predictive performance across classification, point regression, and probabilistic regression with an efficient model design.
Moonjung Eo, Min-Kook Suh, Hye-Seung Cho et al.· 0 citations
Existing benchmarks for OOD in tabular data are extended, providing evidence to support their adoption in high-stakes domains characterized by structural distribution shifts and identifying a scalability gap.
Malena Loza, David Chushig-Muzo, Eva Milara et al.· 0 citations
Foundational models for tabular data have made significant progress in recent years, with TabICLv2 reporting state-of-the-art performance on several tabular classification tasks. However, full-context tabular ICL still suffers from attention cost that grows with the training-context size, which limits its ability to handle large datasets efficiently. Localized TabICLv2 introduces a method that reduces the inference cost of TabICLv2 by retrieving only the k nearest training neighbours for each test point, measured by similarity in the model's Stage 2 row-representation space, rather than using the full training context. This requires no architectural changes, and we show that accuracy retention can be improved through additional Stage 2 and Stage 3 fine-tuning. On TabArena classification tasks, the fine-tuned localized model retains 98.64% of Full TabICLv2 accuracy and it achieves a median 2.18$\times$ speedup in batch inference, and reaches approximately 249$\times$ median speedup in the single-query serving setting.