A transformer encoder architecture for node-level time series forecasting
TL;DR
Experimental results show that the proposed model outperforms traditional RNN and LSTM models across various tasks, indicating promising potential for practical applications.
Abstract
This paper proposes a time-series data processing model based on the Transformer Encoder, designed to address the limitations of traditional Recurrent Neural Networks (RNNs) and Long Short-Term Memory networks (LSTM) in handling long sequences, such as long-term dependency issues, vanishing or exploding gradients, and low computational efficiency. By employing a self-attention mechanism along with learnable positional encoding and multi-head self-attention, the model effectively captures global dependencies in sequential data and significantly enhances its expressive power. To improve training stability, layer normalization is applied after the Transformer encoder, and the data processing flow is optimized through reorganization across batch and node dimensions. The model demonstrates clear advantages in parallel computation and long-term dependency modeling, enabling more efficient processing of complex time-series data. It exhibits superior computational efficiency and stronger generalization capabilities, particularly in real-time traffic flow prediction tasks based on large-scale datasets and long sequences. Experimental results show that the proposed model outperforms traditional RNN and LSTM models across various tasks, indicating promising potential for practical applications.