Skip to content
Preprint

LiveHouse-TS: An Open-world Living Benchmark for Time Series Foundation Models

Aug 2026 · 0 citations · 57 references
Computer Science

TL;DR

The LiveHouse-TS infrastructure serves as a continuous time series infrastructure designed to explore vital, long-term scientific questions: Can model rankings be maintained over the long term?

Abstract

Time Series Foundation Models (TSFMs) have recently emerged as a highly promising paradigm for cross-domain zero-shot forecasting. However, existing evaluation protocols predominantly rely on static benchmarks with fixed historical test windows. While these benchmarks provide a valuable baseline snapshot, they evaluate an average performance on a fixed history, failing to capture how models behave in continuously evolving real-world environments characterized by seasonal variations, distribution shifts, and unexpected events. To bridge this gap, we introduce LiveHouse-TS, the first open-world living benchmark infrastructure for TSFMs. By evaluating models prequentially on real future data in open-world environments, LiveHouse-TS shifts time series benchmarking from snapshot accuracy to continuous temporal validity. Rather than acting as a one-off leaderboard, our infrastructure serves as a continuous time series infrastructure designed to explore vital, long-term scientific questions: Can model rankings be maintained over the long term? Which models remain genuinely robust under distribution shifts? Extensive streaming evaluations across 11 domains with 17 datasets demonstrate that static rankings undergo a dramatic reshuffling under a live protocol.

View source

Similar papers

Preprint Aug 2026

TimeSage-EV: A Live Benchmark for Agentic Time Series Analysis in Evolving Environments

TimeSage-EV is introduced, a live benchmark for agentic time series analysis in evolving environments that evaluates state identification, data summarization, and outlook reasoning, and is released as a research resource with monthly updates, code, a leaderboard, and failure-mode analyses.

Qingren Yao, Ya-Xuan Kong, Yuqi Nie et al. · 0 citations
#machine learning Preprint Oct 2026

Benchmarking Time Series Foundation Models for Load Forecasting Under Covariate Uncertainty

Accurate short-term load forecasting (STLF) is essential for the reliable and efficient operation of modern power systems. While time series foundation models (TSFMs) have recently demonstrated remarkable performance across a wide range of forecasting tasks, their effectiveness for STLF under realistic operational cond...

Tomas Kaljevic, I. Arzola, Yu Zhang · 0 citations
#artificial intelligence Preprint Aug 2026

Beyond MSE: Rethinking the Evaluation Metric and Benchmarking for Irregular Time Series Forecasting

The Continuous-time Squared Error (CSE) is proposed, which employs importance weighting to eliminate the influence of the timestamp sampling distributions and theoretically proves that CSE's asymptotic estimation error with respect to continuous-time risk is no greater than that of MSE.

Rong Li, Hai-Xin Xie, Xiao Wang et al. · 0 citations

When, Not How Much: Evaluating Time-Series Foundation Models on Sparse Events

Pretrained time-series foundation models (TSFMs) are evaluated as forecasters of future values, yet for sparse series many decisions depend only on which future periods contain activity. Standard benchmarks do not assess this. On five sparse datasets, we rank positions within forecast windows that contain both events a...

Daniel Schoess, F. von Wangenheim · 0 citations
#artificial intelligence Preprint Oct 2026

On the Divergence of Accuracy and Mechanism Consistency in Time Series World Models

A time series world model (TSWM) predicts a controlled system's state from its observed history and planned actions and exogenous inputs. Current approaches build forecasters with actions as covariates, trained and evaluated on prediction error under the executed plan. Yet world models compare unexecuted plans, but the...

H. Zhang, Jia-Heng Guo, Zheng Xu et al. · 1 citation · ⚡1

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.