Applying Explainable Renewal-Risk Models to a Libyan ISP: A Case Study Using Operational Billing Logs

Abstract

This study examines subscriber renewal behavior at a Libyan Internet Service Provider (ISP) and frames the problem as renewal risk prediction rather than conventional churn prediction. The operational question concerns the probability that a customer whose package expires on a given day will fail to activate a new commercial package within the following 15 days. The analysis draws on raw billing and operational subscription logs collected between January 2024 and April 2026, yielding 33,308 valid subscription renewal events. Renewal failure defined operationally as the absence of any subsequent commercial package activation within 15 days after the expiry date. A rigorous temporal validation design adopted using train, validation, and test splits, supported by paired bootstrap testing to assess the reliability of model comparisons. The observational evaluation compared a rule-based baseline, Logistic Regression, Random Forest, and CatBoost, while SHAP used to support model interpretability. CatBoost performed best in operational terms after Platt scaling, which improved probability calibration while preserving ranking quality. The calibrated CatBoost model achieved a ROC AUC of 0.7091, PR AUC of 0.4268, and a Brier score of 0.1535. Operationally, targeting the highest-risk 20% of subscribers captured 41.84% of renewal failures before they occurred. These findings suggest that explainable renewal risk modeling can support more focused customer retention actions, such as targeted service follow-up, promotional renewal offers, and advertising campaigns, rather than broad messaging to all subscribers. Keywords: Renewal risk prediction; Internet service provider (ISP); Operational billing logs; Event-level modeling; Customer retention; CatBoost; Random Forest; Probability calibration; SHAP; Libyan telecommunications.