PEMUTA: Pedagogically-Enriched Multi-Granular Undergraduate Thesis Assessment

Abstract

Undergraduate thesis (UGTE) serves as a critical indicator of a student’s cumulative academic development throughout university education. Although large language models (LLMs) have advanced educational intelligence, existing approaches typically focus on holistic assessment with only a single evaluation score, overlooking nuanced variations across multifaceted criteria. This limits their ability to reflect structural criteria, pedagogical objectives, and multi-dimensional academic competencies. Meanwhile, pedagogical theories have long guided manual UGTE evaluation through multi-dimensional assessment of cognitive development, disciplinary thinking, and academic performance, yet such guidance remains underexplored in automated assessment. Motivated by the research gap, we pioneer the Pedagogically-Enriched Multi-granular Undergraduate Thesis Assessment (PEMUTA) framework, which effectively activates domain-specific knowledge in pretrained LLMs for UGTE assessment in engineering disciplines. Grounded in Vygotsky’s theory and Bloom’s taxonomy, PEMUTA evaluates UGTEs across six fine-grained dimensions: Structure, Logic, Originality, Writing, Proficiency, and Rigor (SLOWPR), alongside an overall holistic assessment. A hierarchical two-stage pipeline is designed to first conduct fine-grained evaluation, then synthesize a holistic judgment, enabling more accurate elicitation of domain-relevant assessment knowledge in pretrained LLMs. Few-shot and role-based prompting are further incorporated to improve alignment with expert evaluation practices without parameter tuning. A benchmark consisting of authentic UGTEs collected from multiple engineering disciplines with expert SLOWPR-aligned annotations is curated to support multi-granular UGTE evaluation. Extensive experiments demonstrate that PEMUTA achieves strong agreement with expert assessments under diverse engineering UGTE evaluation scenarios, showing its substantial potential for fine-grained, pedagogically-grounded UGTE assessment within engineering education.