A Comprehensive Psychometric Validation of Student Online Readiness

Abstract

As online and hybrid education continue to expand, institutions require valid and equitable tools to identify students who may need additional support before beginning online coursework. Although numerous online readiness assessments have been developed, relatively few have undergone comprehensive psychometric validation, and most have not established measurement invariance across demographic groups, limiting confidence in subgroup comparisons and the use of readiness assessments for equity-informed decision-making. The purpose of this study was to provide a comprehensive psychometric validation of the Online Readiness and Student Success Assessment (ORSSA), a five-factor instrument designed to measure Learning Organization, Learning Preferences, Learning Skills, Technical Readiness Access, and Technical Readiness Skills among incoming online university students. Data were collected from 2,669 undergraduate students enrolled in online degree and certificate programs at a large public R1 university in the southern United States. Confirmatory factor analysis (CFA) using weighted least squares mean and variance adjusted (WLSMV) estimation was conducted to evaluate the proposed factor structure. Additional analyses examined model-based reliability (McDonald's omega), convergent and discriminant validity, measurement invariance across gender, ethnicity, and first-generation status, and Multiple Indicators Multiple Causes (MIMIC) modeling to investigate relationships between demographic characteristics and online readiness factors. Results supported the proposed five-factor model, which demonstrated superior fit relative to alternative models, strong standardized factor loadings, excellent model-based reliability, and satisfactory evidence of convergent and discriminant validity. Measurement invariance analyses demonstrated that the ORSSA functions equivalently across gender, ethnicity, and first-generation status, supporting meaningful and fair comparisons among demographic groups. MIMIC analyses further identified significant demographic differences across several readiness dimensions, particularly in Learning Organization, Learning Skills, and Technical Readiness, while also revealing higher online learning preferences among some historically underrepresented student groups. These findings establish the ORSSA as a psychometrically robust and equitable measure of online readiness that extends beyond traditional readiness assessments by integrating comprehensive validation procedures with demographic fairness testing. The instrument provides higher education institutions with actionable information that can be used during student onboarding to identify readiness strengths and potential barriers, enabling targeted interventions, personalized student support, and data-informed strategies to improve retention and success in online learning environments.