Probabilistic short-term load forecasting via quantile regression transformer with learned calendar embeddings

Abstract

Accurate probabilistic short-term load forecasting is essential for uncertainty-aware scheduling and reserve planning in modern power systems. Existing approaches rely on fixed calendar feature encodings — such as one-hot vectors or sinusoidal functions — that fail to capture semantic relationships among temporal variables. This paper proposes a Quantile Regression Transformer with Learned Calendar Embeddings (QRTransformer), which maps discrete calendar attributes to trainable dense vectors and fuses the resulting context with historical load and meteorological inputs through a four-layer Transformer encoder, producing 99 conditional quantile forecasts in a single forward pass. Evaluated on the GEFCom2014 Load benchmark, the proposed method achieves an average Pinball Loss of 0.0357 and a CRPS of 0.0715, reducing Pinball Loss by 77.9%, 45.8%, 44.1%, and 6.8% relative to a Naive baseline, LSTM-QR, Informer-QR, and an ablated Transformer without calendar embeddings, respectively. All improvements are statistically confirmed by Diebold-Mariano tests (p<0.001). Ablation results show that removing the Calendar Embedding Module degrades 80% prediction interval coverage from 0.740 to 0.610. Hyperparameter sensitivity analysis confirms the proposed configuration (d_emb=16, L=4) is near-optimal. Generalisation is demonstrated on the EIA-930 ERCOT 2023-2024 dataset (Pinball Loss 0.1226, CRPS 0.2453), where the model maintains its advantage under COVID-era demand disruptions and elevated renewable energy penetration.