Design and Implementation of a Sentence-BERT-Driven Semantic Matching Model for Automated Evaluation of Theory Answers
TL;DR
EvalAI grades theory answers on meaning rather than wording, using Sentence-BERT to measure how semantically similar a student's response is to a lecturer-provided reference answer, showing that semantic grading of this quality can be delivered within a deployable web application without specialised machine learning infrastructure, while keeping the lecturer firmly in the grading loop.
Abstract
Theory-based assessments remain one of the hardest grading challenges to automate. Students rarely express correct answers the way a model answer expects, and the keyword-matching tools built into most learning management systems penalise them for it, not because they are wrong, but because they phrased things differently. EvalAI addresses this by grading theory answers on meaning rather than wording, using Sentence-BERT to measure how semantically similar a student's response is to a lecturer-provided reference answer. Grading suggestions are surfaced to the lecturer, who retains full control over final marks. Evaluation against Mohler's short-answer grading benchmark yielded a Pearson correlation of 0.81 and a mean absolute error of 0.18, indicating that the system's suggestions align closely with human judgement. All functional test cases passed during system testing, and evaluators from both lecturer and student roles completed their tasks without assistance. The results show that semantic grading of this quality can be delivered within a deployable web application without specialised machine learning infrastructure, while keeping the lecturer firmly in the grading loop.