Conceptualising Health Economic Models Using a Graph Retrieval-Augmented, Agentic AI Framework with Human-in-the-Loop Validation: Proof-of-Concept Study.
TL;DR
This is the first study to integrate cutting-edge Gen-AI techniques such as human-in-the-loop, chain of thought-self-consistency (CoT-SC), and graph-based RAG for HEM conceptualisation or broader health economics and outcomes research (HEOR) applications.
Abstract
Objective
Health economic models (HEMs) are crucial for assessing the cost-effectiveness of healthcare interventions, informing policy decisions and guiding reimbursement strategies. This study examines the feasibility of employing generative artificial intelligence (Gen-AI) for the conceptualisation of HEMs. Two proof-of-concept studies were conducted, implementing a first-of-its-kind agentic Gen-AI-augmented approach for model conceptualisation.
Methods
An agentic Gen-AI framework was developed, incorporating the chain-of-thought (CoT) reasoning algorithm through advanced prompt engineering, compounded with self-consistency, executed in an external Python programming environment. Within this framework, an agentic execution layer (the 'agent') autonomously orchestrated a multi-step retrieval-reasoning-generation task. Additionally, a graph-based retrieval-augmented generation (RAG) technique was implemented by linking the large language model (LLM) to an external database curated with user-selected data specific to HEM. Two case studies were explored: breast cancer, representing a data-rich disease area, and transthyretin amyloid polyneuropathy (ATTR-PN), exemplifying a data-sparse rare disease.
Results
The agentic Gen-AI framework successfully generated two key outputs that were subsequently validated against published literature and expert review from health economic (HE) modelers and clinicians: (1) a disease concept diagram illustrating the progression of the disease and its associated health states, and (2) a model structure diagram depicting patient transitions between various health states.
Conclusions
The findings suggest that Gen-AI-augmented methodologies can enhance multiple aspects of HEM conceptualisation, including problem definition, disease and treatment effect pathway development, model structure recommendations, and drafting of model specification plans. To the best of the authors' knowledge, this is the first study to integrate cutting-edge Gen-AI techniques such as human-in-the-loop (HITL), chain of thought-self-consistency (CoT-SC), and graph-based RAG for HEM conceptualisation or broader health economics and outcomes research (HEOR) applications.