Explainable Bad Check Risk Using Hybrid Machine Learning and Large Language Models in Natural Language
An explainable hybrid system model is proposed for detecting non-sufficient funds (NSF) check risks. Data leakage cleaning and feature engineering were applied to create a derived feature set. Multiple ensemble machine learning models were compared using stratified k-fold cross-validation, with LightGBM achieving the highest performance (F1-Makro: 0.7333; ROC-EAA: 0.9212). The best model’s predictions were presented to open-source large language models using zero-shot, few-shot, and chain-of-thought prompting strategies to generate natural language risk explanations. Asymmetric score ranges and strategy-specific thresholds were designed to mitigate central tendency bias. A rule-based scoring system was developed as a baseline. With appropriate prompting strategies, large language models provided competitive results compared to both rule-based systems and direct classification; the best performance was achieved by Qwen 32B with chain-of-thought prompting (F1-Makro: 0.8670; Dengeli Doğruluk: 0.8776).