Which CS1 Students Will Fail? Identifying Digital Markers from Learning Analytics in Computer Systems and Architecture Using Weighted Academic Momentum and Interaction Logs
Results demonstrate that simple digital markers can power a practical early-warning system by the fifth week of the semester, and confirm that weighted academic momentum is the strongest predictor, followed by its interaction with LMS engagement.
Abstract
Digital learning platforms generate rich behavioural traces (digital markers) that offer the potential to identify struggling students early. This paper investigates whether a combination of traditional and digital markers can predict failure in a first-year CS1 course (Computer Systems and Architecture) with sufficient recall to enable timely intervention. Using data from four cohorts (2017-2021, N=284) at a large public university in sub-Saharan Africa, we conducted a mixed-methods stakeholder elicitation to identify ten candidate factors. These were operationalised into a comprehensive feature set spanning demographics, self-reported surveys, Moodle interaction logs, and continuous assessment scores. A systematic ablation study using logistic regression with 5-fold cross-validation and SMOTE+ENN resampling revealed that the most predictive feature subset was Base + Demo + LMS: weighted academic momentum (M = 0.1Q1 + 0.15Q2 + 0.2Q3 + 0.55T1), basic demographics (gender, sponsorship, COVID-19 cohort), and a binary indicator of any LMS activity. On a held-out test set, logistic regression achieved 74.7% accuracy, 0.742 macro F1, and an AUC of 0.800. At the default threshold of 0.5, the model identified 87% of failing students (recall = 0.87) with a 41% false positive rate. SHAP analysis confirmed that weighted academic momentum is the strongest predictor, followed by its interaction with LMS engagement. These results demonstrate that simple digital markers can power a practical early-warning system by the fifth week of the semester. Our main contributions are: (1) a multi-source dataset and a stakeholder-guided methodology; (2) an ablation study quantifying feature group contributions; and (3) an interpretable, high-recall model ready for deployment.
Background: The widespread adoption of Learning Management Systems (LMS) and digital learning platforms in higher education has produced vast repositories of student behavioural and academic data, creating unprecedented opportunities for predictive Learning Analytics (LA). However, academic failure, disengagement, and dropout remain persistent challenges that undermine institutional effectiveness and student wellbeing. Research problem: Existing Machine Learning (ML)-based student-risk prediction systems are predominantly optimised for predictive accuracy and operate as opaque "black boxes," offering limited insight into the reasons underlying a given risk classification. This lack of interpretability constrains their adoption by faculty, academic advisors, and administrators, who require transparent, actionable, and trustworthy evidence before intervening in a student's academic trajectory. Objective: This paper proposes and conceptually validates an Explainable Artificial Intelligence (XAI)-based Learning Analytics Framework designed to identify at-risk students at an early stage of a programme or semester while providing interpretable, human-understandable justifications for each prediction. Methodology: The framework integrates an eight-stage pipeline spanning data collection, preprocessing, feature engineering, comparative machine learning modelling (Logistic Regression, Random Forest, XGBoost, LightGBM, and a Multilayer Perceptron), rigorous evaluation emphasising recall and F1-score under class imbalance, post-hoc explainability using SHapley Additive exPlanations (SHAP) with a supplementary comparison to Local Interpretable Model-Agnostic Explanations (LIME), tiered risk classification, and structured educational intervention pathways. Contribution: The study synthesises 2020-2026 literature on learning analytics, educational data mining, dropout prediction, and explainable AI to identify a converging research gap concerning interpretability, fairness, and pedagogical actionability, and proposes a conceptual and methodological blueprint - including mathematical formulations, an algorithmic specification, and a proposed experimental protocol using publicly available higher-education datasets - that addresses this gap without prematurely claiming unverified empirical results. Educational significance: By coupling predictive analytics with transparent, instance-level and cohort-level explanations, the proposed framework is intended to support - rather than replace - the professional judgement of educators, enabling earlier, fairer, and more defensible academic interventions.
Ankit Kumar Singh, Rubi Singh, Mohd Nadeem· International Journal of Sci...· 0 citations
Competitive programming (CP) offers computer science students an environment for developing algorithmic reasoning skills. However, sustained participation remains a challenge, as many students disengage after encountering skill plateaus or performance anxiety. While educational data mining (EDM) has studied dropout in MOOCs and academic courses, CP attrition remains understudied. This paper presents a dual-layer framework combining large-scale Codeforces activity logs (n=1,816) with a multi-institutional psychographic survey across 10 universities in Bangladesh (n=64). Analysis reveals that true attrition is preceded by an 83.71% reduction in contest participation and a 15.6% increase in struggle time. We identify a"Skill-Application Paradox": stopped students self-report higher mathematical confidence (3.88 vs. 3.41) and data structure understanding (3.57 vs. 3.09) than active peers, yet their independent practice and upsolving habits are significantly weaker (p<0.001). Benchmark evaluations show that a Soft-Voting Ensemble achieves a 0.737 CV F1-score on behavioral logs, while Random Forest achieves 0.924 CV F1-score on psychographic data (an exploratory pilot). We deploy the survey-trained model as a proof-of-concept Early Warning System over 22 active students, identifying 4 at high risk. These findings show that behavioral and psychographic signals can support data-driven mentoring in computing programs.
A. Ruhan, Golam Mostofa Naeem, Rakibul Islam Rafi et al.· 0 citations
Online learning environments generate temporally ordered traces that can support early academic-risk screening, but credible claims require an explicit prediction point and strict control of post-outcome information. This study evaluates whether information available by day 28 of an Open University module presentation can identify students who subsequently fail or withdraw. Raw tables from the Open University Learning Analytics Dataset (OULAD) were reconstructed at student–module–presentation level after timestamped Virtual Learning Environment (VLE) and assessment records had been truncated at the prediction point. The day-28 risk set comprised 27,522 enrolments from 24,832 unique students; 44.1% subsequently failed or withdrew. Seven classifiers were compared using student-grouped five-fold validation, randomized hyperparameter optimization, and a student-disjoint held-out test set. Gradient Boosting achieved the highest grouped-validation ROC-AUC and yielded test ROC-AUC=0.790 (95% CI: 0.777–0.801), PR-AUC=0.770 (95% CI: 0.753–0.786), accuracy=0.724, precision=0.728, recall=0.597, and F1=0.656. Performance increased from day 14 (ROC-AUC=0.747) to day 56 (0.835), demonstrating an earliness–performance trade-off rather than uniformly high early accuracy. SHAP analysis identified completion of assessments due by the cutoff, assessment performance, prior education, and VLE engagement as influential predictors. A score-excluded specification retained meaningful discrimination (ROC-AUC=0.762). Three-class sensitivity analysis showed that failure and withdrawal were considerably harder to distinguish than success. The findings support cautious use of day-28 risk scores for prioritizing supportive outreach, while emphasizing calibration, subgroup monitoring, human oversight, and local validation.
Verda Gizem Oğul, Y. S. Balcıoğlu· OPUS Journal of Society Rese...· 0 citations
Learning Management Systems (LMS) generate rich behavioral data from student interactions, yet many institutions utilize these data only for administrative purposes. This study develops a Machine Learning model for predicting student academic performance based on LMS activity logs and visualizes the predictions through an interactive dashboard. Following the Knowledge Discovery in Databases (KDD) process, data from LMS logs were selected, preprocessed, transformed with feature engineering, and modeled using Logistic Regression, Random Forest, and XGBoost. The Random Forest model achieved the highest performance with 87.0% accuracy and 0.92 F1-score. The interactive dashboard, implemented using Tableau, provides real-time visualizations, risk alerts and actionable recommendations for lecturers and academic administrators. This work contributes to Educational Data Mining and Learning Analytics by integrating predictive modeling with user-friendly visualization, supporting data-driven interventions in higher education.
Pratiwi Rachmadi· Indonesian Journal of Data a...· 0 citations
This study intentionally delayed the introduction of an LMS-integrated LLM tool, CodeHelp, until after early-semester assessments and substantial measures of student effort had already been collected, suggesting that engagement with LLM-based tools reflects underlying self-regulatory behaviors and that the tool functions as a form of technological scaffolding primarily activated by already-engaged learners.
Laura M. Cruz Castro, Maryam K. Multani, Gabriel Castelblanco et al.· IEEE Access· 0 citations
The rapid development of online learning systems has made learner engagement a top priority. While the effectiveness of online education continues to be stifled by low engagement and high rates of dropout, gamification has gained traction as a solution. This work assesses the use of gamification in online learning systems and the correlating roles of learning motivation and student engagement. A quantitative, descriptive-analytical methodology was employed to capture primary data from 250 people, determined through purposive sampling, and who were active users of various online learning systems (i.e., Coursera, Udemy, Unacademy, BYJU’S, and upGrad). The analysis was performed via IBM SPSS Statistics and JASP and was comprised of Reliability Analysis, Confirmatory Factor Analysis (CFA), Pearson Correlation Analysis, and Simple Linear Regression. The analysis indicated high reliability for the three constructs (Cronbach's Alpha > 0.90), and the CFA indicated satisfactory construct validity for Gamification, Student Engagement, and Learning Motivation. Strong, positive, and statistically significant relationships among the three variables were confirmed (p < 0.001) and were reinforced by Learning Motivation and Student Engagement (r = 0.804). Gamification was positively and significantly (p < 0.001) associated with Student Engagement (β = 0.807, R² = 0.652). The results of the analysis indicated that Student Engagement is positively and significantly influenced by gamified features of online learning systems. This study provides practical recommendations for instructional designers and online learning system providers.
P. Magesh, K. Devi· International Journal of Adv...· 0 citations
A new machine-learning framework aims to improve the success rate of computational protein design while moving away from results that reproduce sequences found in nature.
MIT News · Artificial Intelligence· news.mit.eduAug 24, 2026
