Development and initial validation of a survey examining critical GenAI literacy in higher education

Abstract

As generative artificial intelligence (GenAI) becomes increasingly integrated into university contexts, it is essential to understand students' perspectives of the competencies needed to apply its tools for academic writing. This study conceptualized critical GenAI literacy and developed a quantitative survey from theoretical and expert sources. The survey was then administered to undergraduate students, whose responses were randomly divided into two independent subsamples for exploratory ( n  = 214) and confirmatory ( n  = 273) factor analyses. Based on the initial validation results, the final self-report instrument comprised four dimensions: Functional Literacy, Agency and Autonomy, Ethical and Responsible Use, and Reflective Use. The final 23-item instrument captured both applied and critical dimensions of GenAI use in academic writing and demonstrated preliminary structural validity and acceptable internal consistency within a single institutional context. This study provides a foundation for future research on GenAI literacy and supports the development of pedagogical interventions that align with students' critical GenAI literacy levels.