Station-Based Evaluation of AI Weather Models for Near-Surface Temperature, Pressure, and Wind Forecasts over Eastern Coastal China
TL;DR
The results support variable-specific model selection, but they should not be interpreted as a general ranking of atmospheric forecast systems because the validation is limited to near-surface station data and a six-month period.
Abstract
Accurate prediction of near-surface meteorological variables is important for weather services and coastal risk management. However, the station-level performance of global artificial intelligence (AI) weather models remains insufficiently characterized in complex coastal environments. This study evaluated Pangu-Weather, FengWu, FuXi, and the Global Forecast System (GFS) against observations from 210 stations in eastern coastal China from July to December 2022. The assessment focused specifically on 2 m temperature, surface pressure, 10 m wind speed, and wind direction across forecast lead times, stations, and routine and typhoon conditions. FuXi had the lowest temperature RMSE (1.70 °C), whereas FengWu had the lowest pressure and wind-speed RMSE values (0.89 hPa and 1.17 m/s, respectively). The models showed distinct spatial error patterns, and wind-speed errors were concentrated at several northern coastal and transition-zone stations. During Typhoon Muifa, errors increased for all models, with the largest deterioration occurring for wind speed. FengWu retained the lowest typhoon-period wind-speed RMSE (1.87 m/s), whereas GFS had the largest value (3.03 m/s). Wind-direction distributions remained difficult for all models to reproduce. These results support variable-specific model selection, but they should not be interpreted as a general ranking of atmospheric forecast systems because the validation is limited to near-surface station data and a six-month period.