Skip to content

Tabby: An Open Pretraining Recipe for Time Series Foundation Models

Sep 2026 · 1 citation · 47 references
Computer Science

TL;DR

Tabby, a long context probabilistic time series foundation model, is released together with a complete and open recipe of how it was built, which achieves competitive zero-shot forecasting performance on GIFT-Eval and the out-of-distribution TIME benchmark.

Abstract

In this report, we release Tabby, a long context probabilistic time series foundation model, together with a complete and open recipe of how it was built. Tabby adopts an encoder-only patch Transformer architecture and concentrates the contributions on the data and the training procedure. The pretraining corpus combines an extended real-world collection, GIFT-Eval-Pretrain+ and BLAST, with synthetic data from KernelSynth and CauKerV2, an online generator that composes temporal dynamics through randomly sampled structural causal models. Training couples a progressive convergence schedule, which yields reusable intermediate checkpoints, with a deep quantile supervision objective for intermediate layers. The resulting 145M parameter backbone supports contexts of up to 8,192 observations and serves forecasting, classification, and anomaly detection, while a prompt-tuning module further improves in-distribution forecasting performance with the pretrained weights frozen. Tabby achieves competitive zero-shot forecasting performance on GIFT-Eval and the out-of-distribution TIME benchmark, while the same pretrained backbone also supports classification on the UCR Archive and zero-shot anomaly detection on TSB-AD-U. We release training pipeline and model as open source at huawei-noah/trustworthyAI.

View source

Similar papers

Aurora-X: Built for Extreme Time Series Forecasting

A novel pattern-guided mixture-of-experts that expands model capacity through sparse activation and uses shallow patch similarities to constrain deep-layer routing, guiding expert specialization across heterogeneous time series and an implicit quantile network head that predicts arbitrary quantiles to characterize pred...

Xingjian Wu, Chen-Juan Guo, Xiangfei Qiu et al. · 0 citations
Preprint Aug 2026

Into the ORBIT for Time Series: Training Regimes for Foundation Models

This work introduces ORBIT (Omni-Range Bootstrap Incremental Training), a training paradigm that makes this distribution explicit and controllable and introduces Rank-Guided Cross-Depth Alignment, a training objective that uses late-layer representations as stop-gradient teachers for shallow layers without additional i...

Hongjie Xia, Yi-Ding Liu, Yi-Fan Hu et al. · 5 citations
#artificial intelligence Preprint Sep 2026

A JEPA Recipe for Tabular Foundation Models

A recipe under which the latent term of a joint-embedding predictive architecture (JEPA) collapsed in earlier runs and took the encoder with it to a constant map is reported, under which the value head reads the encoder field rather than the predictor, and the target is an exponential moving average difference.

Mingyu Jeon, Suwan Cho, Jae-Young Suh · 0 citations

Loss-Guided Pretraining Data Selection for Time-Series Foundation Models

This work connects forecasting loss to optimization difficulty by showing that normalized squared loss controls the per-sample gradient norm under a local Jacobian condition and defines a reference loss score and applies dataset-stratified selection to preserve the diversity of samples.

Yi-Ke Li, Shao-Xu Song, Jian-Min Wang · 0 citations

Related blog posts

Microsoft Research Blog Nov 16, 2026

TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient

AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.