Beyond Language Processing: LLMs Rules-Injected Instruction Tuning for Traffic Prediction
TL;DR
Rules-Injected Instruction Tuning (RuleGPT), a paradigm that injects IF–THEN rules extracted from training data directly into the LLM's instruction sequence, allows the LLM to focus on higher-order, context-dependent interactions.
Abstract
Large Language Models (LLMs) demonstrate strong capabilities in contextual integration and multi-step reasoning, which endow them with the potential to model heterogeneous traffic data. However, the knowledge acquired during LLM pre-training is primarily qualitative and broad, and does not provide the fine-grained, context-specific quantitative dependencies required for accurate traffic prediction. This mismatch between pre-training knowledge and task requirements can reduce the effectiveness of instruction tuning, the standard approach for adapting general-purpose LLMs to downstream tasks. To address this issue, we propose Rules-Injected Instruction Tuning (RuleGPT), a paradigm that injects IF–THEN rules extracted from training data directly into the LLM's instruction sequence. For each sample, relevant rules are retrieved and encoded as learnable rule tokens that are concatenated with the input, providing explicit guidance to the model. By encoding low-order dependencies in this way, RuleGPT allows the LLM to focus on higher-order, context-dependent interactions. Experiments on real-world traffic datasets demonstrate that RuleGPT consistently improves prediction performance over standard instruction tuning and strong forecasting baselines. The code is available at https://github.com/tigerbunny2023/RuleGPT.