Assessment in competency-based medical education for anesthesiology: validity evidence, equity, and faculty readiness

Abstract

Competency-based medical education (CBME) is driving a paradigm shift in anesthesiology graduate medical education, moving from fixed-duration training to assessment-centered models. This narrative review aims to synthesize the sources of validity evidence for existing assessment tools (milestones, entrustable professional activities, and workplace-based assessments), equity threats during evaluation and their intervention strategies, and the impact of faculty readiness on assessment quality. Informed by the Scale for the Assessment of Narrative Review Articles as a quality-appraisal instrument, we searched PubMed, Web of Science, CNKI, and Wanfang databases through June 2026. Two authors screened sources and extracted data through iterative team discussions, using a framework approach for thematic organization. Current evidence indicates that workplace-based assessment tools have been validated primarily through single-center, small-sample designs with limited longitudinal outcome data. Generalizability theory analyses demonstrate that acceptable reliability can be achieved with a rational design; however, rater variation remains the predominant source of measurement error. With respect to equity, national studies have identified differential rating growth trajectories among underrepresented-in-medicine residents and female residents, although tool structuration may moderate bias effects. Faculty readiness appears to be a major implementation bottleneck, as most clinical teachers lack formal CBME training. However, anesthesiology-specific evidence is limited and partly relies on cross-specialty surveys. The Chinese elite teaching hospital consortium has proposed a localized milestone evaluation framework, but it remains in early pilot stages. The success of CBME assessment in anesthesiology relies on dynamic balance among tool validity, assessment equity, and faculty readiness, with the interaction of these three domains producing emergent system-level behaviors. Future research should prioritize tracking longitudinal predictive evidence and conducting specialty-specific randomized trials of equity interventions.