Development and validation of a scale on the difficulties of AI technology integration in teaching for primary and secondary school teachers

Abstract

The lack of a comprehensive, theoretically grounded measurement instrument has constrained the systematic diagnosis of multidimensional barriers faced by K−12 teachers in integrating artificial intelligence (AI) into instruction. To address this gap, this study developed and validated a multidimensional scale based on the Cognitive–Emotional–Volitional–Behavioral (CEVB) heuristic framework. Through semi-structured interviews, expert consultation, and pilot testing, an initial 20-item pool was formed and administered to 546 teachers from central and western Anhui Province, China. Psychometric evaluation encompassed item analysis, exploratory factor analysis (EFA), confirmatory factor analysis (CFA), reliability testing, and validity and measurement invariance assessments. Item analysis confirmed satisfactory discrimination and differentiating power. EFA extracted a clear four-factor structure—cognitive understanding difficulties, affective attitudinal difficulties, conative motivational difficulties, and behavioral practice difficulties—accounting for 66.20% of the total variance. CFA supported both a first-order four-factor model (χ2/df = 2.014, CFI = 0.944, RMSEA = 0.061, SRMR = 0.056) and a higher-order model (χ2/df = 2.162, CFI = 0.935, RMSEA = 0.065, SRMR = 0.066). The scale demonstrated strong internal consistency (Cronbach's α = 0.915, McDonald's ω = 0.917). Convergent and discriminant validity were satisfactory, and strict measurement invariance was established across gender and school levels (primary, middle, and high school). Criterion-related validity was supported by significant positive correlations between scale dimensions and teacher AI literacy dimensions. Furthermore, structural equation modeling confirmed a significant sequential mediation pathway (F1 → F2 → F3 → F4), empirically validating the theoretical cascading process from cognitive deficits to affective resistance, then to motivational decline, and ultimately to behavioral impediments. This study provides a psychometrically robust, theory-driven diagnostic tool that enables phased and targeted interventions, moving beyond one-size-fits-all approaches to effectively support teachers' AI-integrated teaching.