Automatic Conversion of NICE Guidelines to an Executable Computational Model Using Large Language Models
TL;DR
This work demonstrates that LLMs can be used to automatically transform natural language NICE guidelines into interpretable and executable models that preserve guideline structure, allow transparent inspection and modification, and can be executed to generate patient‐specific recommendations.
Abstract
ABSTRACT Introduction The UK National Institute for Health and Care Excellence (NICE) produce guidelines that provide evidence‐based recommendations to support clinical care across England and Wales, but remain available in unstructured natural language form. Converting these guidelines into computable, logically coherent representations is an active area of research yet existing approaches typically focus on individual diseases, require substantial manual encoding, and do not scale. Recent advances in large language models offer an opportunity to automate much of this translation process. Methods We present an end‐to‐end approach that automatically converts textual clinical guidelines into an executable model capable of generating explainable patient‐specific recommendations. Our approach uses a stepwise LLM‐based transformation with in‐context examples that can be customized to the guideline of your choice. Each step generates human‐inspectable intermediate artifacts, ensuring full transparency and modifiability. We apply the approach to both pancreatic and lung cancer NICE guidelines and use expert human review to assess the alignment of the produced rules as well as evaluating the executable model over 20 pancreatic cancer patient vignettes. Results Human experts review demonstrated strong alignment between the natural language guidelines and the generated executable models, with the majority of guideline recommendations translated correctly. Most discrepancies involved partial omissions of specific details rather than incorrect logic, and instances of hallucinated or fundamentally incorrect rules were rare. When executed on the vignettes, the resulting executable models produced patient‐specific recommendations with an F1 score of 82.5%. Conclusion This work demonstrates that LLMs can be used to automatically transform natural language NICE guidelines into interpretable and executable models. The models preserve guideline structure, allow transparent inspection and modification, and can be executed to generate patient‐specific recommendations. Our findings highlight the feasibility of automated guideline generation, opening the door to scalable computable guidelines.
