Jul 2026· 2026 IEEE 9th International Conference on Big Data and Artificial Intelligence (BDAI)· pp. 187-192· 0 citations· 26 references
Abstract
Large language models (LLMs) have recently been used as frozen backbones for time-series forecasting, aiming to transfer general sequence priors through prompts and embedding-based interfaces. However, recent evidence suggests that many LLM-based forecasting pipelines do not reliably benefit from the language model itself, raising a fundamental question: under what design constraints does a frozen LLM provide measurable and attributable value for numerical time series prediction. Thus, we present Align-TS, a modular framework that makes the role of the frozen LLM explicit and testable by separating structure learning from semantic conditioning. First, a structure-guided encoder models inter-variable dependencies by operating attention over variable tokens, producing representations that preserve multivariate structure. Second, a languagespace recasting module converts structure-aware patches into embeddings that are compatible with the frozen LLM without requiring backbone updates. Third, an alignment-driven conditioning module produces task-aware prompt representations and aligns them with numerical embeddings to reduce entanglement and stabilize inference. This design yields a controllable pathway in which the LLM functions as a semantic conditioner rather than a monolithic predictor. Experiments on multiple public multivariate time-series forecasting benchmarks demonstrate that Align-TS achieves competitive and often superior forecasting performance across diverse domains while keeping the language model fully frozen.
AdaMTP is proposed, an adaptive training paradigm that dynamically aligns the prediction horizon with the intrinsic predictability of the sequence, and consistently outperforms standard MTP in both task performance and inference speedup.
Ziqiang Cui, Han Shi, Bowei He et al.· 0 citations
Large language model (LLM) agents increasingly operate over long interaction histories, where effective reasoning requires identifying and exploiting task-relevant evidence distributed across past observations and actions. However, useful information encoded in previously computed representations is often underutilized during subsequent generation. We propose \textbf{TransMem}, a lightweight inference-time parametric memory module that transforms sparse historical hidden states from a frozen LLM backbone into reusable memory representations. TransMem uses a lightweight gating network to dynamically apply the latent intervention to the current hidden states, without repeatedly encoding the preceding context. To learn transferable memory utilization rather than task-specific knowledge, we introduce evidence-conditioned self-distillation. A memory-augmented student processes the full context and matches the predictive distribution of an evidence-only teacher that shares the same frozen backbone. Experiments on LoCoMo, HotpotQA, and MemoryAgentBench demonstrate consistent improvements across different model architectures and scales. TransMem yields gains of 11.58--29.25 $F_1$ on LoCoMo and 10.20--13.03 $F_1$ on HotpotQA, while improving the average MemoryAgentBench accuracy from 29.54\% to 40.00\%. These results establish sparse historical hidden states as an effective and efficient memory substrate for long-context LLM agents. Our code is available at https://github.com/Haodong-Lei-Ray/TransMem.
Haodong Lei, Junming Liu, Yirong Chen et al.· 0 citations
Time series forecasting models often face challenges in cross-domain fine-tuning, such as high training costs and limited adaptability. To address these issues, we propose a Cue-driven Feature Fusion Network (CFF-Net), which combines semantic cues from textual prompts with numerical time series features for parameter-efficient adaptation. The main idea is to use the semantic representation ability of large language models to provide auxiliary guidance, while dynamically modulating numerical predictions through scale-and-shift operations. Specifically, CFF-Net includes three main components. First, the Semantic Prompt Encoding Module (SPM) transforms numerical sequences into temporally relevant natural language descriptions, which are processed by GPT-2 to extract semantic representations. Second, the Dynamic Semantic Modulation Module (DSM) maps these semantic representations into learnable scaling (γ) and shifting (β) factors through a multi-layer perceptron, enabling modulation of PatchTST predictions within the Scale-and-Shift Feature (SSF) mechanism. Finally, a warm-start strategy is used to stabilize semantic integration during training. Experimental results on three public datasets and the TCTS dataset show that CFF-Net achieves lower errors than PatchTST in many settings, although the improvements are not uniform across all datasets and metrics. For example, on the Weather dataset, CFF-Net reduces MSE by 12.50% and 11.88% under the 30% and 20% training-sample settings, respectively. On the TCTS dataset, the corresponding MSE reductions are 5.83% and 6.60%. These results suggest that semantic prompt guidance can improve forecasting performance in several limited-data scenarios while keeping most backbone parameters fixed.
Kaibin Wei, Jianqiang Jing, Jiawei Liu et al.· PLoS ONE· 0 citations
Time series forecasting models operate on raw numerical sequences, lacking the semantic knowledge that domain experts implicitly leverage, such as the physical meaning of each variable, its statistical behavior, and its temporal dynamics. Recent efforts to bridge this gap fall into two camps. Some rely on large language models at inference time, which is computationally expensive. Others apply uniform textual prompts at the dataset level, ignoring the heterogeneous semantics across individual variates. We propose SAGE (Seeing and Augmenting with Grounded Encoding), an end-to-end CLIP-based framework that jointly models temporal, cross-variable, textual, and visual information. The CLIP text encoder processes frequency-enhanced patches and variable tokens, while gated residual paths inject variable-specific descriptions and statistical descriptors. In parallel, the frozen CLIP vision encoder aligns rendered series with temporal representations through a training-only contrastive objective. This dual use of CLIP adds complementary semantic and visual supervision without placing an LLM in the forecasting loop. Across eight long-term benchmarks and M4, SAGE achieves state-of-the-art accuracy. Ablations confirm complementary gains from multimodal alignment and variable-level knowledge.
Evidence is provided that cross-scale heterogeneous fusion can succeed without explicit semantic alignment when the donor contribution is sufficiently concentrated and carefully selected, and that activation-guided extraction improves the quality of the transferable donor slice while preserving the small-ratio fusion regime.
Jiahe Fan, Sixiang Chen, Yinghao Hou et al.· 0 citations
Training multimodal models to align time series with language runs into a self-supervision trap. The usual recipe asks an LLM to read a series and write a description, so label quality is capped by the perceptual skill the model is supposed to learn. The data can never teach more than the labeler already knows. A second gap makes this worse: most datasets use a single variable, but the patterns that matter (cross-channel correlation, lead-lag structure, co-occurring anomalies) appear only with several variables, right where the labeling LLM's limits are most exposed. These two problems create a trilemma: existing methods are reliable, realistic, or scalable, but none achieves all three. We resolve this by decoupling perception from description. Deterministic code computes a set of statistics from real, open-source multivariate series; the LLM verbalizes those precomputed facts. Perception, which LLMs do poorly, is handled by computation, while the LLM handles expression. This produces CGTime, our 4B-parameter computation-grounded time-series-language model. CGTime outperforms far larger general-purpose models on multivariate understanding tasks: it attains the best multivariate fact score on our held-out benchmark (0.283 vs. 0.173 for GPT-4o-mini and 0.203 for GPT-5.4-nano), a gap that survives Holm-corrected paired significance tests against every baseline. It also states verifiable numerical facts in generated captions more accurately and covers a broader range of statistical properties.
Xinran Feng, Yi Xie, Chaolin Zhang et al.· 0 citations