Less Marking, More Meaning: Assessing Higher-Level Thinking in Engineering with Auto-Graded Exams

Abstract

This study examines measurement quality following a shift from constructed-response (CR) exams to fully auto-graded selected-response (SR) exams in large engineering courses. Using Classical Test Theory, we analyzed ten final exams across three engineering courses spanning a transition from mixed CR/SR to fully auto-graded SR formats. Internal consistency improved in two first-year courses (Cronbach's α: 0.666–0.758 to 0.781–0.851), with lower relative measurement error (SEM/SD), meeting or exceeding published university MCQ reliability norms. These gains occurred alongside a deliberate upward shift in Bloom's Taxonomy level, suggesting increased cognitive demand did not reduce psychometric quality. Discrimination evidence strengthened, with Bloom’s Apply and Analyze subscores retaining particularly strong alignment with overall proficiency. The same metrics remained acceptable and stable in a third-year design course.  Overall results support that deliberate SR design is a scalable and psychometrically credible approach for high-stakes assessment across Bloom's levels in the studied contexts.