This work investigates LLM-based forecasting agents, meaning systems in which a language model contributes to a scored prediction about a future or currently unobserved target, and organizes architectures into three groups.
Abstract
Large language models (LLMs) now support forecasting systems that combine language-based reasoning with temporal data, evidence retrieval, external tools, and iterative prediction. We investigate LLM-based forecasting agents, meaning systems in which a language model contributes to a scored prediction about a future or currently unobserved target. We organize architectures into three groups. Standalone LLM workflows operate on encoded time series or event context. Tool- and retrieval-augmented agents incorporate external evidence. Hybrid systems pair LLMs with statistical or foundation models. We then review training methods and evaluation protocols. We examine negative as well as positive evidence, including sensitivity to small input perturbations, ablations in which the LLM component does not improve accuracy, and benchmark gains that may reflect contamination instead of temporal reasoning. We cover applications in finance, weather, health, energy, and operations, and we summarize the benchmarks and datasets used for evaluation. The evidence indicates that measurement is a central limitation. Future work requires calibration under distribution shift, contamination-resistant live evaluation, explicit reporting of cost and accuracy together, and methods for handling feedback between deployed forecasts and the outcomes being forecast.
Due to the diversity of real-world time series, no single forecasting model consistently dominates across all samples. Ensemble learning addresses this by combining complementary model strengths, yet existing methods rely on fixed rules or black-box models based solely on numerical inputs, failing to leverage LLM reasoning for interpretable weighting decisions. We propose REATS, which leverages LLM reasoning capabilities as an intelligent ensemble router that jointly processes textual temporal pattern descriptions and numerical features to produce interpretable, sample-adaptive ensemble weights through chain-of-thought reasoning. To enable effective LLM-based ensembling, we study its key design choices and propose: (i) a structured input pipeline that transforms raw time series into hybrid textual--numerical representations with fixed token cost, enabling rule-based chain-of-thought construction without API dependency, augmented with retrieved similar-sample priors; (ii) a diverse multi-row weight supervision scheme coupled with a token-efficient percentage-table format that reduces numerical complexity and mitigates LLM hallucinations; and (iii) a two-stage fine-tuning framework combining SFT with GRPO, where a reciprocal reward mapping transforms the continuous unbounded MSE gap into bounded signals with amplified near-oracle sensitivity, addressing the uniform sensitivity and outlier-dominated advantage compression inherent in naive reward designs for regression-based GRPO. Experiments on eight benchmarks demonstrate that REATS outperforms competitive ensemble baselines while providing natural language explanations and demonstrating strong transfer learning and out-of-domain generalization to unseen candidate models.
Financial markets are one of the few machine-learning settings where feedback is fast, mistakes are expensive, and deployment is constrained by execution and regulation. Yet most existing tutorials still treat finance either as an NLP benchmark or as a classical forecasting problem. This tutorial takes a different view. We show how recent progress in generative AI becomes genuinely useful in markets when it helps model participant behavior, ground reasoning in live documents and order-flow data, and support research and execution workflows that can survive contact with production. The material therefore moves in three steps: behavioral and market foundations; grounded LLM systems, including RAG, critique models, and agentic workflows; and responsible deployment, including corrected backtesting, execution, risk controls, and governance. Two live demonstrations and three optional Colab notebooks connect the lecture material to implementation.
Z. Iklassov, Hachem Madmoun, J. Duhot et al.· Proceedings of the 32nd ACM...· 0 citations
A novel alignment framework, Reinforcement Learning for Selection Reward (RLSR), is proposed, which targets the area under the risk-coverage curve (AURC) -- a popular SP performance metric -- as its alignment objective and achieves substantially better risk-coverage trade-off compared to multiple alignment baselines on both in-domain and out-of-domain tasks.
Gaoxiang Luo, Yi-Fan Wu, Sinian Zhang et al.· 0 citations
Text-conditioned time-series forecasting predicts a series from both its numerical history and natural-language context, allowing forecasts to account for events and constraints that the past alone cannot reveal. This requires both reliable numerical forecasting and the ability to interpret contextual information. Time-series foundation models (TSFMs) provide strong numerical forecasts, while large language models (LLMs) can reason over text, but combining their strengths remains challenging because asking an LLM to generate or revise forecast values directly can distort the temporal structure captured by the TSFM. We instead formulate forecasting as a planning problem over TSFM-generated trajectories. The frozen TSFM acts as a simulator that proposes numerical continuations, while the LLM acts as a policy and value function that guides candidate selection and evaluates completed trajectories against the context. We instantiate this as \rc{} (\textbf{L}LM \textbf{A}s \textbf{F}orecasting \textbf{P}lanner), a training-free framework that bridges the modality gap without retraining either model, using Monte Carlo tree search (MCTS) over the forecast horizon with a \emph{Ranker} LLM as policy and a \emph{Judge} LLM as value function. Experiments on Context-is-Key and Time-MMD across two TSFM backbones (Chronos and TimesFM) and four LLMs show that \rc{} delivers consistent improvements across model choices, supporting sequential search as an effective training-free approach to text-conditioned forecasting.
H. Nguyen, Dung Nguyen, M. Nguyen et al.· 0 citations
Large language models and foundation models are increasingly embedded in reasoning systems that plan, invoke tools, use memory, gather evidence, and iteratively refine their outputs. The second KDD Day on AI Reasoning brings together researchers and practitioners from academia and industry to examine how these systems can be made more capable, reliable, interpretable, and efficient. The program spans scientific discovery, human-centered interaction, software engineering, time-series analysis, deep research, computer use, and inference infrastructure. Across these domains, the day highlights shared challenges: grounding decisions in evidence, designing effective feedback and verification mechanisms, evaluating open-ended behavior, managing test-time computation, and preserving meaningful human control. Through keynote and invited presentations, the event provides a forum for connecting advances in models, agents, data, systems, and applications, and for identifying research directions toward trustworthy next-generation reasoning systems.
Jun Huan, James Caverlee, Lei Li et al.· Proceedings of the 32nd ACM...· 0 citations
Event-driven catalyst-outcome forecasting increasingly uses retrieval- and memory-augmented large language model agents for prediction. However, training-data contamination and temporal leakage can create an Evidence Chasm between reported accuracy and true predictive ability. We propose TIEM, a timestamp-gated framework with three coordinated components: an Event-Evidence Hypergraph (EEH) for timestamp-filtered multi-tier retrieval; a Case-based Skill Memory (CSM) for source-tagged temporal skills; and Heterogeneous Evidence-Experience Fusion Reasoning (HEFR) for evidence-experience fusion and prediction. We also introduce FinPURE, a recent-period A-share holdout benchmark, and use a Name-Date Probe to assess per-model name-date sensitivity rather than assuming training cutoffs. Results on five financial forecasting benchmarks show TIEM outperforms current baselines. Our project is available at https://github.com/QwenQKing/Fin_TIEM.
Wenjin Liu, Shengjie Pang, Chen-Xi Wang et al.· 1 citation
