Review Open access

GENERATIVE ARTIFICIAL INTELLIGENCE AND THE VALIDITY CRISIS OF ASSESSMENT IN MEDICAL EDUCATION: AN INTEGRATIVE REVIEW

Viviane Canhizares Evangelista de Araújo Marcia Abusio P. P. Herculiani Fábio Augusto Freiria Barbosa Adriana Porto Nunes Gazetta Josiane Ramos Garcia Rodrigues Eleny Rosa Guimarães Gonçalves João Paulo Galletti Pilon Gustavo Pereira Mais Ana Gláucia Paulino Lima Mara Flávia Mamedio Avallone Igor Soares Gianini Grecca S. H. S. Gianini C. Galhardi Cíntia Gisele de Andrade Pozenato Wilson Bernardo Silva Lisete Horn Fabiane Lopes Toledo Marcus Vinícius Freire Uchôa Araujo Tawana Pascon Lopes Samantha Santos da Silva Thaís dos Santos Rodolfo de Oliveira Medeiros

Oct 2026 · Veredas do Direito · 0 citations

Abstract

Introduction: The rapid spread of generative artificial intelligence tools based on large language models has challenged the assumptions that support assessment in medical education. When a chatbot can answer licensing-style questions, write reflective narratives, generate test items, and score clinical performance, the interpretations drawn from assessment results may no longer correspond to the competencies they are intended to represent. Objective: To examine the scientific evidence on the effects of generative artificial intelligence on the validity of assessment in medical education, considering its roles as examinee, item writer, and assessor. Method: An integrative literature review was conducted, grounded in the principles of evidence-based practice. The review question was structured using the PICo strategy. The search was carried out in the PubMed/MEDLINE, Scopus, Web of Science, and ERIC databases. Included studies were characterized in terms of their methodological features and levels of evidence, and the findings were subjected to interpretive thematic synthesis. Results: Eleven studies were included in this review. The literature indicates that generative artificial intelligence performs at or above the level of students in knowledge-based and written tasks, that educators show limited ability to identify texts produced by these tools, that automatically generated items require expert review to preserve content validity and cognitive level, and that AI-based scoring shows limited agreement with human judgment and may introduce linguistic bias against specific groups of students. Conclusion: The validity crisis associated with generative artificial intelligence does not stem from the technology alone, but from assessment formats that depend on unsupervised written products and isolated measurements. Programmatic assessment, supervised performance-based methods, transparent institutional policies, and qualified human oversight of AI-supported processes are needed to preserve the credibility of assessment decisions.

DOI Semantic Scholar

Read PDF