Design and Implementation of a Scalable AI-Based Semantic Evaluation System for Hindi Text Using Transformer Models
TL;DR
This study proposes an automated evaluation framework with intelligent rule-based linguistic preprocessing and transformer-based deep learning that will enable scalable, consistent and repeatable assessment in Hindi.
Abstract
Evaluating linguistically diverse descriptive answers in a consistent and accurate manner in modern digital education systems is a growing challenge, especially in low-resource languages like Hindi. Traditional lexical and rule-based grading systems cannot adequately reflect the meaning behind the words, negation, paraphrasing, and so on, which leads to low grading reliability. To overcome these limitations, this study proposes an automated evaluation framework with intelligent rule-based linguistic preprocessing and transformer-based deep learning. The framework uses a fine-tuned multilingual BERT (mBERT) model bhavikardeshna/multilingual-bert-base-cased-hindi to provide contextual embeddings, and cosine similarity-based semantic alignment with the model l3cube-pune/hindi-sentence-similarity-sbert is used to provide automated scores. With optimal setting of learning rate = 5×10⁻⁴, batch size = 24 and epochs = 40, accuracy, precision, recall and F1 score of 78.9%, 80.6%, 77.4% and 79.0% respectively is achieved on HindiRC-Data-master dataset (24 passages, 127 question-answer pairs, grades 2-5) which is more than 14% higher than lexical similarity baselines and is better than previous Hindi QA architectures without domain-specific preprocessing pipelines. The suggested system will save about 40% manual grading, and will enable scalable, consistent and repeatable assessment.