Assessing Learning after Generative AI Support: A Framework for Arabic Language Education

Abstract

With support, generative artificial intelligence (GenAI) can make a language-learning task appear more successful. However, this visible improvement does not necessarily reflect what the learner can produce independently. This conceptual paper thus poses a more specific assessment question: When can the gains be considered as learning in Arabic language education in the context of GenAI-supported activity? To answer that question, the framework is based on research on the performance after support, cognitive offloading and engagement with AI, performance-learning differences, retention, transfer, and Arabic or second-language assessment. It identifies 5 categories of evidence: Supported Performance, Immediate Independent Performance, Delayed Retention, Near Transfer, and Far Transfer. These are viewed from three related perspectives: independence from AI assistance, temporal persistence, and task/context generalization. The framework in practice distinguishes between what a learner can do with GenAI, what remains after the support is withdrawn, what remains after a justified delay, and what can be used when the task changes. The five design considerations that are added to the Assessment-Validity Matrix are: AI availability, assessment timing, task similarity or distance, language output, and baseline or comparison conditions. The Arabic language target remains evident, including the lexical, grammatical, morphological, orthographic, sentence-level, and connected-production outcomes. Five propositions are presented for further empirical testing. The categories are not required to be presented in a specific order: time and task distance are independent of each other, and a study might only include the conditions necessary for the claim. The framework is not an empirically established model, but rather a conceptual specification that helps to ensure that the evidence generated by the assessment design remains close to the learning claims.