Variable–Patch Semantic Routing for Multimodal Climate and Environmental Time Series Forecasting

Abstract

Multimodal time series forecasting incorporates external textual information, such as weather reports, environmental bulletins, and warning-related descriptions, to provide additional context for numerical sequence modeling. However, existing multimodal fusion methods mainly rely on global cross-modal interactions or feature concatenation, which often inject textual semantics indiscriminately into all variables and temporal positions, leading to semantic dilution and cross-modal noise. To address this issue, we propose Variable–Patch Semantic Routing (VPSR), a multimodal forecasting framework that aligns numerical temporal patches with textual representations, organizes the multivariate series into a two-dimensional variable–time-segment grid, and introduces variable relationship modeling before routing. A multi-head semantic routing mechanism then dynamically allocates textual semantics to the most relevant variable–time units, enabling more selective cross-modal fusion. Experimental results on real-world climate and environmental benchmark datasets show that VPSR consistently outperforms existing unimodal and multimodal baselines across multiple forecasting horizons, validating the effectiveness of variable relationship modeling and fine-grained semantic routing for multimodal time series forecasting.