#machine learning #data science Preprint

Causal Foundation Models

Christopher Stith Hossein Rahmani Jesse C. Cresswell

Sep 2026 · 0 citations · 108 references

Computer Science Mathematics

TL;DR

Causal foundation models are pretrained neural networks that estimate causal quantities, such as the average treatment effect, on entirely new datasets using in-context learning without requiring model updates.

Abstract

Causal inference is the practice of estimating the effect of a treatment or intervention from data. It traditionally requires a bespoke pipeline for every new problem: first proposing a causal mechanism, selecting a compatible estimator, and finally training it. Meanwhile, across diverse settings and modalities, much of machine learning has shifted to the paradigm of foundation models: networks pretrained once at scale and applied to new tasks without fine-tuning. Causal foundation models (CFMs) bring this paradigm to causal inference. CFMs are pretrained neural networks that estimate causal quantities, such as the average treatment effect, on entirely new datasets using in-context learning without requiring model updates. This work provides a practical introduction to this emerging area. We summarize the necessary background in causal inference and machine learning before discussing CFMs. Throughout, we include example code and Jupyter notebooks.

DOI arXiv Semantic Scholar

View source

Similar papers

#artificial intelligence Preprint Sep 2026

When and Why LLM Causal Priors Help: Closed-Loop Prior Selection for Amortized Causal Inference

A closed-loop prior selection framework is proposed that casts prior injection as a budget-constrained optimization over a candidate prior pool and shows that the use of LLM causal priors stops being manual trial and error and becomes an empirically verifiable selection problem.

Hao-Hao Zhou · 0 citations

#artificial intelligence Preprint Sep 2026

Demistifying Data and Simulator Assumptions in Supervised Causal Discovery

This paper organizes methods by prediction target, prediction granularity, encoder, structural decoder, and training regime to relate what each method predicts to how it uses data and simulator-based supervision, and distinguishes two questions: whether the target is identifiable under the assumed model class and wheth...

Pingchuan Ma, Rui Ding, Bojun Huang et al. · 0 citations

From Causal Structure to Conditional Generation

Yu-Chen Guan, Chair Pradeep Ravikumar, Bryan Wilder · 0 citations

Preprint Aug 2026

Foundation Models for Partial Causal Identification

This paper investigates the development of causal foundation models for bounding the effect of interventions and counterfactuals from observational data. We show that a canonical prior can be defined with full support over the space of structural causal models with discrete observables. With this canonical prior, we tr...

A. Bellot, Anish Dhir · 2 citations

#machine learning Preprint Sep 2026

CIDER-FM: Foundation Models for Causal Inference from Diverse Experimental Regimes

Causal foundation models (CFMs) amortise causal inference over priors of synthetic structural causal models (SCMs), predicting the effect of an experiment on a specific variable. However, observational data alone may leave multiple causal models compatible with available evidence, while experimental data with intervent...

Yu-Che Gao, Arik Reuter, Si-Yuan Guo et al. · 0 citations

#artificial intelligence Preprint Sep 2026

Beyond Conditional Independence: Root Cause Analysis with Deep Causal Models

RCA-DCM improves top-1 accuracy over the strongest baseline on both Sock Shop and Online Boutique, and when the true root cause in the causal chamber is unobserved and acts as a latent confounder, it recovers the exact root-cause set more often than any competing method.

Md. Musfiqur Rahman, Kenneth Lee, Zi-Wei Jiang et al. · 0 citations