A Delphi–Monte Carlo Framework for Probabilistic KPI Estimation in Public-Sector AI Projects

Abstract

Public-sector organisations must often commit to AI adoption decisions before longitudinal performance data are available to inform them. This study introduces and preliminarily demonstrates a Delphi–PERT–Monte Carlo framework for probabilistic KPI estimation in public-sector AI projects, applied to Kazakhstan’s e-government environment. A three-round modified Delphi study was conducted with 20 domain experts (competence threshold K ≥ 0.70). The Technology–Organisation–Environment framework structured elicitation across three AI scenarios (S0: no AI; S1: partial automation; S2: advanced AI) and four KPIs: timeline, cost, risk detection, and process transparency. Categorical percentage-band responses were translated into PERT distributions via a documented band-to-midpoint mapping rule and simulated through 10,000 Monte Carlo iterations. All outputs reflect structured expert belief, not observed performance data. Under S2, timeline effects were directionally ambiguous (P50 = −0.5%; bootstrap 95% CI: [−5.0%, +13.5%]); cost was consistently positive (P50 = +13.7%), reflecting near-term implementation cost increases expected by 55% of panelists. Risk detection (P50 = +5.8%; CI: [+3.7%, +12.8%]) and transparency (P50 = +8.2%; CI: [+2.5%, +8.8%]) showed modest but stable positive effects. Round 3 returned 80% consensus acceptance. The framework provides a replicable, sensitivity-tested procedure for generating probabilistic planning inputs in expert-data-scarce contexts; the Kazakhstan case is its first Central Asian application.