NoDE: noise-decorrelation embedding for robust long-term multivariate time series forecasting
Abstract
In long-term multivariate time–series forecasting, longer input windows are often assumed to help. However, on some standard benchmarks we observe that, for certain model–dataset combinations, performance becomes non-monotonic and unstable as the look-back length grows. Taking a data-mining perspective, we first characterize correlated input noise on the Weather benchmark, a widely used multivariate forecasting dataset. Residual-denoising analyses on this dataset show that, for some variables and time periods, input perturbations exhibit substantial short-lag autocorrelation rather than behaving as white noise. Building on this evidence, we provide a window-conditional second-order interpretation that relates whole-window embeddings to variance inflation (for linear forecasters) and curvature-dependent bias (for nonlinear forecasters) arising from residual lag covariance under temporally correlated perturbations, and discuss how this interpretation connects to both squared and absolute error criteria used in evaluation. To address this issue in real-world datasets, we introduce NoDE (Noise-Decorrelation Embedding), a simple and model-agnostic structured linear embedding that reduces effective lag covariance via downsampling, permutation-based reshaping (shuffle/unshuffle), and shared affine mappings, while preserving a drop-in interface to common backbones. Our protocol automatically selects the downsampling window in a data-driven manner through a two-stage procedure: candidate resolutions are proposed based on ACF-energy statistics, and the final choice is made via a lightweight quick-train validation. An explicit no-op option is included to disable NoDE when downsampling is not beneficial. Comprehensive experimental evaluations validate that NoDE stabilizes performance under long look-back windows across multiple datasets and backbone families (Transformer, CNN, and MLP). In addition, the protocol’s ability to automatically activate or deactivate NoDE via a no-op option allows it to support standard, patch-wise, and multi-scale front-ends through the same selection interface, enabling robust and architecture-agnostic deployment. Code and models are open-source at https://github.com/lmh9507/NoDE.