It is demonstrated that conventional regression metrics alone are insufficient for evaluating time-series LLMs and the need for reliability-focused diagnostics when deploying LLM-based forecasting systems in high-stakes domains is highlighted.
Abstract
Large Language Models (LLMs) are increasingly being applied to time-series forecasting, giving rise to a class of models referred to as time-series LLMs. While these models achieve competitive predictive accuracy, their reliability and structural consistency remain insufficiently understood. These models may produce forecasts that are numerically accurate on average yet statistically or temporally implausible, a phenomenon referred to as hallucination. Unlike conventional forecasting errors, hallucinations represent deviations from underlying temporal dynamics that exceed expected volatility patterns. This paper presents a systematic investigation of hallucination in time-series LLM forecasting. We introduce a quantitative evaluation framework that complements traditional regression metrics with two reliability-oriented measures: perplexity (PP), which reflects predictive uncertainty, and hallucination rate (HR), which measures statistically significant deviations from ground truth. Experiments on widely used benchmark datasets (Electricity and ETT variants) reveal a critical trade-off between forecasting accuracy and reliability. In several settings, improvements in average error metrics do not correspond to improved reliability; models can maintain low mean absolute error (MAE) while exhibiting high HR. To mitigate this issue, we evaluate two strategies: data-centric preprocessing, whose effectiveness depends on dataset characteristics, and structured tokenization, which consistently reduces hallucination across the evaluated datasets. Sensitivity analysis over quantile thresholds confirms the robustness of hallucination trends and model rankings. These results demonstrate that conventional regression metrics alone are insufficient for evaluating time-series LLMs and highlight the need for reliability-focused diagnostics when deploying LLM-based forecasting systems in high-stakes domains.
A D-LLM hallucination detection framework that formulates the Denoising trajectories as Multivariate Time Series over learnable latent variables (DeMTS for short) that outperforms existing hallucination detection methods while maintaining strong robustness, efficiency, and cross-task transferability.
Time series forecasts are widely used in decision-critical domains, where they are rarely consumed without accompanying explanations. Producing such explanations is usually a manual and costly process, and attempts to automate it using large language models often suffer from hallucination when applied to temporal data. We propose a domain-agnostic framework for grounded natural language explanation generation for time series forecasts, illustrated in Figure 1. The framework consists of three components: (i) extraction of structured explanatory factors from historical analyst-written explanations, (ii) evidence-conditioned explanation generation, and (iii) scalable evaluation for readability, logical consistency, and persuasiveness. The design explicitly constrains generation to verifiable evidence, reducing unsupported claims. We evaluate the framework on a financial forecasting case study involving the NASDAQ-100 index and a freight pricing case study using data from Vortexa. Results show that generated explanations approached analyst-written explanations in terms of readability, consistency and persuasiveness. These findings demonstrate that grounded explanation generation for time series forecasting can be achieved at scale without domain-specific fine-tuning.
Ria Mundhra, Gustavo Sato dos Santos, Michael Benedikt· 0 citations
A unified symbolic, behavioral, and mechanistic framework that connects symbolic triggers with internal failure dynamics in transformer architectures and provides an interpretable basis for diagnosing and stabilizing symbolic reasoning in LLMs is introduced.
This review paper provides a comprehensive overview of hallucinations in GAI and LLMs, and synthesizes a range of correction and mitigation techniques, from proactive measures during training to hybrid approaches that combine detection and intervention.
M. Naser· Language Resources and Evalu...· 0 citations
Large Language Models (LLMs) are temporally overexposed: trained on vast contemporary corpora, they encode present-day concepts that make them unreliable narrators of the past. We present TimeCapsule, a 1.2B-parameter LLaMA-style causal model trained exclusively on Victorian texts (1800–1875) as an epistemologically isolated generative archive. Quantitative evaluation shows a 45.4% perplexity reduction over a GPT-2 baseline on held-out Victorian prose, while larger contemporary causal models achieve lower raw perplexity through broader pretraining but lack temporal isolation. TimeCapsule exhibits computational sensemaking, generating historically plausible analogical explanations for unfamiliar modern concepts (e.g., describing a computer as a “hypertrophied lung”). A qualitative hermeneutic probe with two humanities scholars revealed a crisis of authenticity, as both misclassified approximately 40% of genuine Victorian excerpts as machine-produced. We argue that structural ignorance of the future transforms hallucinations into interpretive probes of nineteenth-century ontologies.
Large Language Models frequently generate responses that appear fluent and coherent but may contain factually incorrect information, a phenomenon known as hallucinations. Hallucinations have emerged as a significant barrier to the effective application of Large Language Models. Some studies suggest that hallucinations can be detected by analyzing structural properties of attention maps. However, such methods primarily focus on attention structures and their spectral features, whereas uncertainty, randomness, and other statistical measures can also be leveraged to detect hallucinations. In this work, we propose a method for detecting hallucinations using spectral and statistical features extracted from attention maps. We compute eigenvalues of the Laplacian matrix from each attention map to capture spectral features and compute statistical measures for every attention head. The spectral and statistical features are used as a feature vector for hallucination detection. We conducted experiments on the HaluEval dataset across multiple language models, and the results show that jointly modeling structures and uncertainty-aware characteristics leads to more robust hallucination detection.
Rln Lokesh, Chirumamilla Mokshagna, K. Dinesh et al.· International Conference on...· 0 citations
