A Comparative Study of Pre-trained Language Models for Cognitive Distortion Detection in Indonesian Text

Abstract

Cognitive distortions are maladaptive thinking patterns that are closely associated with anxiety, depression, and other mental health disorders. Early detection of these distortive thoughts through textual expressions has the potential to support scalable mental health preventive actions. However, the comparative study of pretrained language models (PLMs) for Indonesian cognitive distortion detection remained limited to evaluate effectiveness and efficiency of potential models. This study presents a comparative evaluation of 17 available pretrained language models, including Indonesian language models, multilingual models and English models related to mental health domain, using a publicly available Cognitive Distortion Dataset in Indonesian Language. Three experimental approaches were employed including feature-based learning, full fine-tuning, and multi-seed evaluation. Full fine-tuning approach consistently outperformed feature-based learning across all evaluated models, highlighting the importance of model optimization for task specific implementation. NusaBERT-large achieved the highest overall performance of Macro-F1 score, Weighted-F1, and Accuracy, whereas IndoRoBERTa obtained the best feature-based performance. Multi-seed experiments further showed that IndoLEM/IndoBERT exhibited the most stable optimization behaviour and shortest average time per epochs. Indonesian PLMs consistently outperformed multilingual and English domain adaptive models after full fine-tuning applications. These findings provide the comprehensive comparative evalutions of diverse pretrained language models and offer insights of selecting pretrained language models for cognitive distortion detection in Indonesian text.