An Experimental Deep Learning Framework Integrating Contextual and Visual Features for Detecting Student Engagement in Virtual Classrooms

Abstract

Detection of student engagement in a virtual learning environment has become paramount. The current systems are mostly based on visual facial features, ignoring contextual demographic features, which can undermine generalization and fairness across different groups of learners. The study introduces a contextually sensible deep learning system that combines visual facial features with demographic embeddings (age, gender) to enhance the classification rate of engagement and achieve fairness and real-time capabilities. A training set of 16,000 labeled images was formed of 17 institutional ethics-approved volunteers aged 5-21+ years. Baseline performance was set by fine-tuning of 4 pretrained CNNs (MobileNet, Xception, DenseNet201, NasNet Large). A custom model, OnEduNet, was developed, which integrated visual and contextual representations. Performance of models was measured using macro F1-score, confidence intervals, significance testing (McNemar), fairness metrics (Demographic Parity, Equalized Odds), ablation testing, robustness testing, and Grad-CAM interpretability. Results: OnEduNet achieved a statistically significant validation accuracy of 92%, outperforming the best baseline, MobileNet (89%). The model demonstrated a minor lack of demographic difference (EOD<0.05) and maintained a real-time inference speed (18ms/image). Combining contextual demographic features with visual representations could contribute greatly to engagement detection accuracy, fairness, interpretability, and deployment feasibility, which can be used to support scalable and patent-oriented educational AI systems.