HYPERPARAMETER SENSITIVITY ANALYSIS OF AUTOFORMER FOR SHORT-TERM ELECTRIC LOAD FORECASTING

Abstract

Short-term electric load forecasting is a key task in power system operation, electricity market scheduling, energy management, and demand-side planning. Accurate short-term forecasts help operators anticipate demand variations, reduce reserve requirements, improve economic dispatch, and maintain system reliability. In recent years, Transformer-based forecasting models have attracted increasing attention due to their ability to capture temporal dependencies in time-series data. Autoformer is one of the representative Transformer variants designed for time-series forecasting through series decomposition and an auto-correlation mechanism. However, the performance of Autoformer is highly affected by hyperparameter settings, especially those controlling model representation capacity, attention structure, and optimization behavior. This paper investigates the sensitivity of Autoformer performance to three key hyperparameters: hidden size, number of attention heads, and learning rate. A grid-based experimental framework is developed for short-term load forecasting using the New South Wales electricity demand dataset with 30-minute temporal resolution. The forecasting task is configured for a one-day-ahead prediction, with the model forecasting 48 future time steps using a historical input window of 7 days. Each hyperparameter is evaluated at 5 candidate values, yielding 125 experimental trials. Model performance is assessed using mean absolute error, mean absolute percentage error, mean squared error, and root mean squared error. Among the 125 designed configurations, 100 were completed successfully, while 25 with hidden_size = 512 failed due to GPU memory limitations. The best configuration, consisting of hidden_size = 128, n_head = 4, and learning_rate = 0.0005, achieved an MAE of 372.70 MW, a MAPE of 4.9978%, an MSE of 289,690.53, and an RMSE of 538.23 MW. The sensitivity results indicate that learning_rate has the strongest marginal influence, while the effects of hidden_size and n_head depend substantially on their interactions with the other hyperparameters. The proposed framework provides a systematic basis for selecting Autoformer configurations while considering forecasting accuracy, variability, and computational feasibility.