Association between on-demand video lectures and Common Achievement Tests scores in pre-clinical education

Abstract

Since the COVID-19 pandemic, on-demand lectures have been widely adopted in university education. However, the association between on-demand lectures and learning has not been objectively and comprehensively investigated. This retrospective cohort study compared pre-pandemic and post-pandemic student cohorts and examined the association with performance on the Pre-Clinical Clerkship Common Achievement Tests. A total of 643 test records from fourth-year medical students at a Japanese national university were used for this retrospective cohort study. The cohort before the implementation of on-demand video lectures was classified as the no-video lectures group, and the cohort after the implementation as the video lectures group. Learning outcomes were assessed using the Pre-Clinical Clerkship Common Achievement Tests, including the Computer Based Testing (CBT) and Objective Structured Clinical Examination (OSCE), and a generalized linear model (GLM) was used to examine group, grade point average (GPA), and their interaction. After applying the exclusion criteria, 629 records were analyzed. The video lectures group had a significantly higher GPA than the no-video lectures group. CBT scores were significantly higher in the video lectures group, although the effect size was small ( d  = 0.411). No significant difference in OSCE scores was observed between the two groups. In the GLM, a significant interaction between GPA and group was observed, and the association between GPA and CBT performance was stronger in the cohort after the implementation of on-demand video lectures. These findings suggest that the association between academic performance and knowledge assessment outcomes may differ between cohorts. However, this study was a retrospective cohort study, and GPA differed between the groups. Therefore, the observed associations cannot be attributed solely to on-demand video lectures. Further prospective studies with adequate control of confounding factors are warranted.