MAUSAM: An Observations‐Focused Assessment of Global AI Weather Prediction Models During the South Asian Monsoon
Past evaluation of artificial intelligence (AI) weather prediction has primarily relied on reanalyses, which can obscure important deficiencies due to prevailing biases in reanalyses themselves. Here, we present MAUSAM (Measuring AI Uncertainty during South Asian Monsoon), an evaluation of seven leading AI‐based prediction systems—FourCastNet, FourCastNet‐SFNO, Pangu‐Weather, GraphCast, Aurora, AIFS, and GenCast—during the South Asian Monsoon, using ground‐based weather stations, rain gauge networks, and geostationary satellite imagery. The AI models demonstrate considerable forecast skill during the monsoon across a broad range of variables, ranging from large‐scale surface temperature and winds to precipitation and cloud cover. In addition, the models generate realistic zonal means and eddy statistics on subseasonal to seasonal timescales, even capturing key aspects of the abrupt Monsoon pause of 2025. However, the models still exhibit systematic errors in finer‐scale features, including the underprediction of extreme precipitation (“tail” events), divergent cyclone tracks, and disagreements in the mesoscale kinetic energy spectra, highlighting avenues for future improvement. A comparison against a combination of in situ and remote sensing observations reveals forecast errors 15%–45% larger than those relative to reanalysis and traditional HRES forecasts, indicating that reanalysis‐centric benchmarks can overstate forecast skill. Of the models assessed, AIFS exhibits the most consistent representation of atmospheric variables, with GraphCast and GenCast also showing strong skill. The analysis presents a framework for systematically evaluating global AI weather models on regional predictions. The results highlight both the promise and current limitations of AI weather prediction and the continued need for observations‐focused evaluation.