Edu-Eval: A Large-Scale Multimodal Benchmark for MLLMs in Authentic Educational Scenarios

Abstract

The modern classroom is an inherently multimodal environment, rich with the teacher's speech, student expressions, and interactive instructional resources. Effective AI assistants should therefore perceive this complex pedagogical process, not just answer text-based questions. However, existing evaluation methods are fundamentally misaligned. Current educational benchmarks primarily focus on unimodal and single-task evaluation, while existing multimodal benchmarks focus on content evaluation by testing MLLMs as students rather than the critical process evaluation by testing them as assistants. To fill this critical gap, we introduce Edu-Eval, the first large-scale benchmark focused on educational scenario-centric evaluation. Inspired by educational theories, we developed our Teacher-Student-Resource framework, which computationally operationalizes abstract pedagogical concepts into nine concrete evaluation tasks. Edu-Eval is constructed from over 3,000 real-world scenarios, comprising 75,900 annotated samples. Our evaluation of eight major MLLM series reveals a critical gap between current MLLM capabilities and the demands of real-world applications. Edu-Eval is more than a dataset. It is a diagnostic tool and a clear roadmap for future research, highlighting the urgent need for the community to shift its focus from isolated content evaluation to a holistic and scenario-centric understanding of the entire classroom ecosystem.