A multi‐criteria ranking framework for machine learning algorithms: credit card fraud detection

Abstract

Credit card fraud detection presents unique challenges for machine learning due to extreme class imbalance, evolving fraud patterns, and asymmetric misclassification costs. While numerous algorithms have been proposed for this domain, their evaluation typically relies on simplistic metrics that fail to capture the multifaceted requirements of operational fraud detection systems. This study introduces a sophisticated multi‐criteria ranking framework that extends beyond traditional performance measures to incorporate temporal stability, computational efficiency, and robustness considerations. We develop an anisometric penalty structure that quantifies deficiencies across multiple dimensions with differential weighting, and we apply this framework to evaluate several leading machine learning approaches with various sampling strategies, XGBoost, Random Forest, and logistic regression—using a large dataset of European credit card transactions. Our findings reveal that an active learning approach with combined uncertainty–diversity sampling achieves superior performance across multiple evaluation criteria, outperforming both XGBoost and traditional classification algorithms. Statistical significance testing and sensitivity analysis confirm the robustness of these results across different weight configurations and operational scenarios. This study advances both the theoretical understanding of machine learning evaluation in fraud detection and provides practical guidance for financial institutions seeking to implement or enhance their fraud detection systems.