Comparative analysis of short answer questions and multiple-choice questions in formative assessment of first-year medical students
TL;DR
MCQs were easier and yielded higher scores, whereas SAQs provided a more challenging assessment with comparable discrimination, and using both formats together may enhance assessment quality in undergraduate medical education.
Abstract
Assessment methods are essential for evaluating medical students’ knowledge and cognitive skills. Multiple-choice questions (MCQs) are widely used because of their objectivity and efficiency, whereas short answer questions (SAQs) are considered better suited for assessing higher-order cognitive skills. However, evidence comparing the psychometric performance of these formats in formative undergraduate medical assessments remains limited. This study aimed to compare the difficulty index (P), discrimination index (DI), and overall test performance of SAQs and MCQs in a formative assessment of first-year medical students. A descriptive analytical cross-sectional study was conducted among 178 first-year medical students enrolled in the Medical Parasitology course within the Locomotor and Skin module at Ain Shams University during the 2024–2025 academic year. Students completed content-matched SAQ and MCQ examinations, each comprising 20 items aligned with the same intended learning outcomes. Psychometric analyses included item difficulty, discrimination, reliability, and comparisons between formats. Students achieved significantly higher scores on MCQs than SAQs (13.5 vs. 11.1/20, p < 0.001). MCQ items were significantly easier (mean p: 0.67 vs. 0.55, p < 0.001), while discrimination indices were comparable (0.44 vs. 0.41, p = 0.398). Both formats demonstrated excellent internal consistency (Cronbach’s α = 0.85 for MCQs and 0.83 for SAQs) and strong discrimination between high- and low-achieving students. Pass/fail concordance was 68.0%, with more students passing MCQs but failing SAQs (p = 0.012). Item difficulty correlated strongly across formats (r = 0.71), whereas discrimination showed a weak, non-significant correlation (r = 0.01). Clinical SAQs were associated with lower performance among low-achieving students. Both MCQs and SAQs demonstrated sound psychometric properties. MCQs were easier and yielded higher scores, whereas SAQs provided a more challenging assessment with comparable discrimination. Using both formats together may enhance assessment quality in undergraduate medical education.