Learning from model failure: insights from an AI/SHAP-based temporal error decomposition of tracer-aided ecohydrological modelling

Abstract

Abstract. Process-based models (PBM) have served as testing grounds for hypotheses, being falsified and refined through model evaluation based on temporally complex PBM errors. Yet, conventional evaluation practices (e.g., performance metrics, visual inspection across time series, etc.) largely depend on the user’s a priori knowledge to trace PBM errors to environmental forcings and are limited in resolving temporal error implications. In this study, temporal characteristics of PBM errors were explored using a data-driven Artificial Intelligence (AI) model focusing on delayed and non-stationary signatures and linking them to the predictors (i.e., PBM input data). First, an ecohydrological isotope-enabled PBM, EcoHydroPlot, was calibrated against high-resolution datasets of water amounts and water stable isotopes (δ2H) in soil of two depth layers and tree xylem, monitored from June to October 2020 in a riparian willow plot in Berlin, Germany. Then, PBM errors were calculated for six calibration targets. Second, an ensemble of LSTMs (i.e., AI error analyser) was trained to reproduce the PBM error, which was then decomposed with SHapley Additive exPlanations (SHAP) across both the retrospective lag and the study period, yielding a two-dimensional (lag × time step) quantification of temporal attribution for each predictor. The analyser reproduced 60–99% of the error variance, showing that these errors were not random noise but carried a systematic, learnable structure. The temporal characteristics of PBM errors depended on both predictors and targets, showing a general tendency that errors contributed by LAI, or targeting xylem δ2H, were attributed to earlier time lags, whereas those related to precipitation or sapflow were to recent lags. Over the study period, error propagations were observed in distinct patterns, i.e., Event-driven, Period-driven, and regime change signal, especially, near the transition period from growing into non-growing season. The results showed that the AI error analyser could support evaluation of a PBM as a complementary tool, and as an example, its diagnoses were translated into hypotheses for a better representation of xylem δ²H in the PBM.