AI-Based Language Models (LLMs) in Assessing ESL Learners’ Speaking Skills: Opportunities and Limitations

Abstract

Speaking is widely regarded as the most difficult skill to teach, practise, and assess in English as a Second Language (ESL) classrooms, largely because valid assessment of oral proficiency has traditionally depended on scarce resources: trained human raters, controlled testing conditions, and considerable time. The recent emergence of large language models (LLMs) capable of real-time speech-to-text conversion, conversational interaction, and rubric-referenced feedback generation has renewed interest in automating parts of this process. This article critically reviews the opportunities and limitations of LLM-based tools for assessing and supporting the development of ESL learners’ speaking skills. It traces the evolution from earlier automatic speech-scoring systems grounded in automatic speech recognition (ASR) and statistical feature extraction to the current generation of generative, instruction-following language models capable of producing holistic and analytic feedback in natural language. Five categories of opportunity are identified — scalable formative feedback, on-demand conversational practice, rubric-referenced scoring support, fine-grained pronunciation and fluency diagnostics, and reduced speaking anxiety through low-stakes practice — alongside five categories of limitation: construct underrepresentation, accent- and dialect-related bias, hallucination and inconsistent reliability, washback effects and learner overreliance, and unresolved data-privacy and ethical concerns. Building on this synthesis, the article proposes a five-criterion framework — validity, reliability, fairness, transparency, and pedagogical alignment — intended to guide the responsible integration of LLM-based tools into ESL speaking instruction and assessment, and discusses implications for English-language teaching contexts such as Azerbaijan, where access to trained oral examiners is often constrained. The article concludes that LLMs are best positioned as a complement to, rather than a replacement for, human assessment of spoken English, and it outlines a research agenda for validating LLM-based speaking assessment against established testing standards