Ethics of Autonomous AI Clinical Trials: Delphi Study

Abstract

Abstract Background Safe implementation of autonomous AI in medicine requires rigorous evaluation through clinical trials. The 7 guiding principles for ethical clinical research endorsed by the National Institutes of Health (NIH) provide an established framework for promoting scientific rigor and protecting the safety of human participants. However, clinical trials of autonomous AI raise novel ethical issues that require adaptation of these principles to account for effects that can vary across stakeholders and implementation contexts, including model performance across clinical settings. Incorporating expert perspectives on such challenges is critical to developing effective and ethically robust guidelines for autonomous AI clinical trials. Objective This Delphi study aimed to generate expert consensus on how the National Institutes of Health’s 7 principles of ethical clinical research should be applied to clinical trials of autonomous AI. Methods We conducted 2 rounds of surveys followed by a final virtual meeting using a modified Delphi approach with a multidisciplinary expert panel. Participants were purposively identified through PubMed literature searches and selected for expertise in AI, data science, ophthalmology, public policy, law, bioethics, and patient advocacy. Round 1 used open-ended survey questions based on a vignette describing an autonomous AI tool. Round 1 survey responses were analyzed qualitatively using thematic coding, and were used to generate representative statements for Round 2. In round 2, panelists rated statements on a 5-point Likert scale. In accordance with Delphi methodology, statements of consensus within the surveys (at least 80% rating agreement) and moderate agreement (60%‐80% rating agreement) were identified for further discussion and iteration. Findings from a final virtual meeting were then analyzed thematically and synthesized into actionable recommendations. Results Fourteen expert panelists participated in the Delphi study over a 6-month period. Participation was 12 (85.7%) of 14 experts in round 1, 10 (71.4%) of 14 experts in round 2, and 13 (92.9%) of 14 experts in the final virtual meeting. Round 2 survey results yielded 9 strong agreement statements, 2 moderate agreement statements, and 4 divisive statements for participants to explore further in developing recommendations. Final recommendations from the virtual meeting addressed transparency regarding training and validation data, bias assessment before deployment, performance across clinical settings, health inequities before implementation, stakeholder engagement, informed consent, comparison of AI tools with existing standards of care, downstream access to care after AI-generated recommendations, and cost and access implications. Conclusions Ethical evaluation of autonomous AI clinical trials should extend beyond technical accuracy to help mitigate potential harms to patients. This study highlights key ethical considerations for clinical trials of autonomous AI and provides consensus recommendations from a multidisciplinary Delphi panel. These recommendations can inform future research, policy, and guidance for the ethical development and implementation of autonomous AI clinical trials.