MULTIMODAL GENERATIVE AI FOR ENHANCING STUDENT COMPREHENSION IN STEM LEARNING

Abstract

Student comprehension remains a major challenge in Science, Technology, Engineering, and Mathematics (STEM) education, particularly in online learning environments where uniform instructional approaches may not adequately accommodate differences in learner ability, prior knowledge, engagement, and learning progress. This paper proposes an adaptive multimodal Generative Artificial Intelligence (GAI) tutoring framework that personalizes instructional support by integrating textual explanations, visual illustrations, and interactive learning activities according to the learner's current state. The framework continuously estimates comprehension, engagement, previous performance, and instructional difficulty and uses this information to adapt both content and instructional modality. A simulation-based evaluation using synthetic learner profiles compares the proposed framework with Static Learning, Non-Adaptive GAI, and Rule-Based Adaptive Learning. The evaluation considers learning effectiveness, comprehension gain, engagement, cognitive load, modality contribution, and scalability. Under the considered simulation conditions, the proposed framework achieves higher learning effectiveness, comprehension gain, and engagement while maintaining a lower cognitive-load proxy than the baseline approaches. The results indicate the potential of learner-state-aware multimodal adaptation for personalized STEM tutoring and motivate further validation using real learner interaction data.