The effect of LLM-powered value-integrated story cards on English vocabulary acquisition: A generative AI-supported convergent mixed methods study in middle school students
TL;DR
This study investigates the effectiveness of large language model (LLM)-powered value-integrated story cards in English vocabulary instruction using a convergent mixed methods design with 30 sixth-grade middle school students over a four-week intervention and offers pedagogical implications for GenAI-supported, holistic, and value-integrated material design in foreign language education.
Abstract
This study investigates the effectiveness of large language model (LLM)-powered value-integrated story cards in English vocabulary instruction using a convergent mixed methods design with 30 sixth-grade middle school students over a four-week intervention. Story cards embedding universal values (empathy, responsibility, honesty, and cooperation) were developed via systematic prompt engineering with GPT-4, a generative AI (GenAI) tool, reviewed by domain experts, and applied within a quasi-experimental pre-test/post-test control group design. Generative AI was additionally employed in the preliminary coding phase of qualitative data analysis and in generating the item pool for the Vocabulary Achievement Test. Data were collected via a Vocabulary Achievement Test (KR-20 = .83), structured classroom observation forms, and semi-structured individual interviews. Quantitative findings revealed a statistically significant and practically large difference in favour of the experimental group [t(28) = 4.15, p = .001, Cohen's d = 1.52]. Qualitative content analysis generated four main themes: contextual recall, motivation-enjoyment, value awareness, and material usability. Findings offer pedagogical implications for GenAI-supported, holistic, and value-integrated material design in foreign language education.