Semantic-Driven Knowledge Assessment: Ontologies and Large Language Models for Automatic Test Generation

Abstract

This paper proposes a semantic-driven approach to automated knowledge assessment that integrates domain ontology modeling with the capabilities of large language models (LLMs) for generating test items with controlled difficulty. The core idea is to utilize formally structured knowledge, represented as an ontology, as a structural and semantic foundation for constructing valid, consistent, and interpretable test questions. An architecture of the information technology is developed, comprising modules for ontology construction, semantic reasoning, test generation, and quality evaluation. A method for test item generation is proposed, based on templates, ontological axioms, and context-aware queries to LLMs. Evaluation criteria for test quality are defined, including semantic consistency, difficulty level, discriminative power, and alignment with learning objectives. An experimental study is conducted to validate the effectiveness of the proposed approach in comparison with traditional test generation methods. The obtained results demonstrate an improvement in the quality of test items and a reduction in the time required for their development.