Skip to content
Preprint

Timestep-Conditioned Transformers for Global Weather Forecasting

Aug 2026 · 0 citations · 26 references
Computer Science

TL;DR

GEM-3, a probabilistic global weather model that addresses this trade-off through explicit multi-timestep inference, is presented, a practical forecasting system that couples near-SOTA medium-range probabilistic skill, stable extended-range rollouts, efficient training and inference, and decision-relevant diagnostics.

Abstract

Existing machine-learning weather forecasting models rely on predetermined and fixed autoregressive timesteps. The choice of model timestep involves a fundamental trade-off: shorter timesteps (e.g. 1 to 6 hours) finely resolve atmospheric dynamics within the diurnal cycle but increase error accumulation for a given forecast horizon, while longer timesteps (e.g. 24 hours) reduce error accumulation but limit the usability of short-range forecasts where sub-daily predictability is high. In this work, we present GEM-3, a probabilistic global weather model that addresses this trade-off through explicit multi-timestep inference. With a single set of trained weights, the model timestep can be configured at inference time to balance predictability and usability across a broad forecast horizon. Additionally, we find that mixed-timestep training consistently improves rollout stability relative to timestep-specialist models. Under the hood, GEM-3 is a lightweight neighborhood-attention transformer with ~134M parameters on an equirectangular grid with a number of architectural advancements beyond its predecessor GEM-2. The result is a practical forecasting system that couples near-SOTA medium-range probabilistic skill, stable extended-range rollouts, efficient training and inference, and decision-relevant diagnostics.

View source

Similar papers

#machine learning Preprint Sep 2026

Stochastically Perturbed Weights: Ensembles from Deterministic Machine-Learning Weather Models

Stochastically perturbed weights (SPW) is at present a tuning procedure rather than a plug-and-play recipe, and its main failure mode is a coherent whole-field offset that overdisperses the domain mean, and restricting the noise to coarse scales or perturbing the initial conditions each repair part of it.

Simon Adamov, O. Fuhrer, R. Knutti et al. · 0 citations
Preprint Aug 2026

AICON: An operational global machine learning weather forecasting model

An extensive evaluation, including routine verification against observation, a tropical cyclone case and spectral analysis reveal the strengths and limitations in the representation of atmospheric variability across scales.

Tobias Goecke, M. Jacob, F. Prill et al. · 1 citation
Preprint Sep 2026

TaiJi: State-Conditional Adaptive Combination of Machine-Learning and Physics-Based Global Weather Forecasts

Machine-learning weather models now rival or surpass operational numerical weather prediction (NWP), yet no single model dominates across variables, pressure levels, lead times, or regions, and the returns from ever-larger individual models are diminishing. We present TaiJi (after the yin-yang concept of complementary...

Jia-Le Wang, Yi-Ning Li, Gui-Hua Wang · 0 citations
#machine learning Preprint Oct 2026

Learning joint probabilistic weather forecasts from station observations alone

CLARA (Calibrated Advection-Routing Attention) learns joint Gaussian predictive distributions of five surface variables from station observations alone, without numerical weather prediction or reanalysis, and outperforms persistence in all 60 multi-year region-lead comparisons and a similarly sized learned model in 57...

Chaeyeon Yi, Y. Seo · 0 citations
#machine learning Preprint Oct 2026

SDECast: Probabilistic Weather Forecasting in Continuous Time with Neural SDEs

Existing machine learning weather forecasting models typically generate forecasts through autoregressive rollouts at a fixed temporal resolution. While highly efficient for long-range prediction, this formulation can suffer from severe error accumulation when used with shorter time steps and does not explicitly encode...

Maria Marchenko, Martin Andrae, F. Lindsten et al. · 0 citations
Preprint Oct 2026

Domain-Adaptive Data Assimilation for Global AI Weather Forecasting

AI weather forecasting models are commonly trained on the ERA5 reanalysis, which is unavailable in real time. Operational deployment therefore relies on initial conditions produced by numerical or AI analysis systems that differ from those encountered during training. This mismatch can degrade forecast skill, while ret...

Minseok Seo, Noah D. Brenowitz, Do-Yun Kim et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.