Performance Analysis of a Scalable Machine Learning Model for Financial Fraud Detection

Abstract

In this paper, a forensic end-to-end machine learning pipeline to detect financial fraud in PaySim synthetic mobile money dataset a corpus of 6,362,620 transactions with an extreme class imbalance ratio of 0.129% is presented. Proposed system is based on the distributed MLlib framework of Apache Spark in order to overcome the scalability issues associated with processing millions of financial records. Multicollinear raw ledger features are substituted with seven theoretically-based feature engineering transformations such as accounting discrepancy indices and account drain ratios as well as high-risk categorical flags. Ten classifiers such as Logistic Regression, Decision Tree variants, Random Forest ensembles, Gradient Boosted Trees, Linear SVC and a Factorization Machine are considered in a forensic model tournament. The imbalance between classes is alleviated through random undersampling to a balanced 50:50 training sample. The Matthews Correlation Coefficient (MCC) is the statistically rigorous evaluation index of the imbalanced binary classification that allows model selection. The winning ensemble model has a Recall of 99.39%, a Precision of 100.00%, an MCC of 0.9941 and a Financial Protection Index of over 96% which shows the operational reliability and good business value in preserving capital. Limitations regarding the synthetic nature of the data, the absence of sequence modeling, the schema dependence of the engineered features and the batch nature of the deployment are also discussed.