Development of the Assessment of Engineering Literacy for All Majors: An Exploratory Validation of the Internal Structure of a Self-Perception Scale

Abstract

This study aimed to develop a self-perception-based scale, the Assessment of Engineering Literacy for All Majors (AELAM), applicable to Korean university students across all disciplines, and to examine the initial psychometric evidence for its internal structure. Based on a literature review and theoretical integration, a five-factor, 28-item pool was constructed and refined to 26 items through two rounds of expert content validity review (N = 14; N = 8), cognitive interviews (N = 15), and a pilot analysis (N = 248). The main survey collected data from 500 university students via the Macromill Embrain online panel. Following an analysis plan preregistered on the Open Science Framework, parallel analysis, competing confirmatory factor analyses of one- to five-factor models, and reliability and validity tests were conducted. Parallel analysis recommended a single factor, and the five-factor model was statistically equivalent to the three-factor model. Through by the principle of parsimony, a three-factor structure (engineering thinking and performance, sociotechnical responsibility and practice, sociocultural critical analysis) was adopted. A supplementary bifactor analysis yielded an explained common variance of .853 and a general-factor reliability (omega hierarchical) of .917, indicating that the scale is essentially unidimensional. Accordingly, the total score serves as the primary unit of interpretation, with the three factors functioning as auxiliary content domains for curricular feedback. The scale is designed as a learner-centered diagnostic tool that allows students to assess their own engineering-literacy self-perception and supports individualized feedback in liberal education. This study contributes a psychometric basis for redefining engineering literacy as a liberal-education competency required of all citizens and for diagnosing students' self-perception of it.