Colloquial Parsing: Evaluating Deep Learning Approaches for Indonesian Fraud Detection

Abstract

Fraud messages in Indonesia often get past normal spam filters because they use casual language and slang. To study this, we test the Colloquial Parsing framework across six machine learning models, comparing four traditional classifiers against IndoBERT and a parameter-efficient large language model (Llama-3.2 with LoRA). When a combined dataset of Indonesian chat messages is tested, the traditional models, especially SVM and Logistic Regression, perform exceptionally well, with F1-Macro scores ranging narrowly from 96.8 to 98.7 percent. The traditional models, especially SVM and Logistic Regression, edge out the rest at 98.7 percent, largely by learning the common, repeated keywords found in short fraud messages. This small difference indicates that the complexity of the models does not significantly affect the detection performance in this keyword-rich, low-resource scenario, and that simple classifiers continue to be a robust and cost-efficient approach; with the vast variety of models available, it is likely that even the large language models would require a much wider spectrum of conversation data that is not as template-driven as the current dataset used here before their advantages in architecture become evident.