Beyond Prediction: Cost-Effective Early Intervention Targeting for Student Attrition in Online Higher Education

Abstract

Early-warning systems for identifying at-risk students in online higher education have proliferated in recent years, with most work optimizing predictive accuracy alone. Few studies translate model performance into an actionable, financially grounded intervention policy. This paper develops a 28-day early-warning model using engagement, assessment, and demographic data from the Open University Learning Analytics Dataset (OULAD; 29,915 registrations after excluding pre-course withdrawals), with a temporal train/test split (trained on 2013 presentations, tested on 2014 presentations, n=17,360) to avoid student-level leakage. We compare an interpretable logistic regression baseline (test ROC-AUC = 0.764, accuracy = 0.705) against a random forest model (test ROC-AUC = 0.794, accuracy = 0.716), and extend both with a cost-benefit framework using published institutional figures: £1,336 net revenue per retained student and £37.26 per flagged-student intervention, the latter derived transparently from Georgia State University’s published advising-program costs. Rather than selecting a decision threshold by maximizing accuracy or AUC, we identify the threshold that maximizes net institutional value under varying assumptions of intervention effectiveness. At a conservative 30% effectiveness anchor, the optimal threshold for both models is 0.10, yielding a net value of £2.72 million (logistic regression) and £2.78 million (random forest) on the test cohort. Break-even analysis shows an intervention need only be 3.7–3.8% effective at a conventional 0.50 threshold to pay for itself, indicating that threshold choice, not intervention effectiveness, is the binding constraint on institutional value. This cost-aware framing offers a template for translating learning-analytics predictions into deployable, budget-justified retention interventions.