Development and Usability Evaluation of an AI-Powered English Vocabulary Learning Application

Abstract

Learning vocabulary is a major challenge for beginner foreign-language learners, as repetition-based methods can be tiring and often lead to rapid forgetting. Generative AI, particularly Large Language Models (LLMs), enables the automatic creation of large amounts of learning content. However, without appropriate pedagogical guidance, AI-generated materials may be inconsistent and provide limited support for effective learning progress. This study therefore developed and evaluated an adaptive application for beginner-level English vocabulary practice. The application uses an LLM to provide contextual assistance and combines it with a modified spaced repetition mechanism. Its vocabulary and learning activities follow the appropriate CEFR level, while the difficulty is adjusted based on the accuracy of each learner’s answers. The application was evaluated through a mixed-methods usability study involving 72 learners with beginner-level English proficiency. User experience and acceptance were measured using the System Usability Scale (SUS) and selected measures from the Technology Acceptance Model (TAM). Usability was rated highly. Although the participants were beginners with limited English, they generally found the interface comfortable and manageable. Their SUS responses produced a mean of 81.46 (SD = 7.18), a result classified as “Excellent.” They also viewed the application as useful and relatively easy to operate. The respective scores were M = 4.38 (SD = 0.42) for Perceived Usefulness and M = 4.29 (SD = 0.45) for Perceived Ease of Use.