An Explainable and Adaptive Multimodal Framework for Real-Time Digital Education Quality Evaluation

Abstract

Digital education quality evaluation increasingly relies on multimodal, AI-assisted systems, yet the literature remains fragmented: models that are explainable are rarely real-time, models that are real-time are rarely multimodal or adaptive, and models that fuse multiple modalities typically trade away transparency for accuracy. This study addresses that gap by extending the MAC-HASA model into EAM-HASA (Explainable Adaptive Multimodal-HASA), a neurosymbolic, multi-objective framework designed to unify explainable reasoning, cross-modal fusion, and continuous real-time adaptation within a single architecture for evaluating digital education quality. The framework couples cross-modal attention fusion of motion, audio/text, and interaction data with a post-hoc SHAP-based explanation layer and a hierarchical reinforcement-learning policy agent that retunes evaluation weights as instructional conditions change. As a proof-of-concept, the predictive component was trained and validated on a public multimodal physical-education teaching dataset (1,000 session records from 200 students) using an XGBoost classifier benchmarked against MAC-HASA and a Transformer baseline. The classifier achieved 92.0% accuracy and an 89.5% macro F1-score in predicting teaching-effectiveness category, and SHAP attribution showed that engagement and instructional-clarity indicators, rather than raw motion-sensor signals, were the dominant drivers of the evaluation outcome. These results support the feasibility of the predictive core and the pedagogical relevance of its explanations, while the full explainable, adaptive, governance-aware architecture is formalized algorithmically and proposed for empirical validation on multimodal, multi-institutional data in future work.