Skip to content
Book Open access

When Should Forecasting Models Listen to Text? Counterfactual Utility Routing for Multimodal Time Series Forecasting

Nov 2026 · Proceedings of the 34th ACM International Conference on Multimedia · 0 citations · 12 references

Abstract

Time-series forecasting supports decisions in domains such as finance and healthcare, where forecasts may depend on events not fully represented in numerical histories. Multimodal forecasting can supply this context through text, but the additional information is useful only when it remains relevant to the target and forecast horizon. Existing methods commonly fuse text at every prediction step, even when it is stale, weakly related to the target, or informative only for certain horizons. We therefore frame text use as a horizon-wise decision and introduce CUTS, a Counterfactual Utility Routing framework for selective fusion. For each forecast step, CUTS retrieves horizon-specific evidence and produces matched text-conditioned and no-text forecasts. A utility estimator predicts the benefit of the retrieved evidence, while its uncertainty moderates the final interpolation between the two forecasts. On Time-MMD, CUTS achieves the best nine-domain normalized MSE and MAE among the evaluated methods. It records the lowest MSE in five of nine domains. On FNSPID, it achieves the lowest average MSE and MAE and obtains the lowest MSE at the 3-day and 5-day horizons. Ablations identify horizon-specific evidence selection and staged training as key contributors. Corruption and routing analyses further show that uncertainty-aware routing helps CUTS limit its reliance on unreliable text.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.