CVI-Validated Indo-Transformer Framework for Intelligent Cooperative Supervision
TL;DR
It is demonstrated that CVI-validated ensemble GenAI can construct consistent labels for low-resource administrative texts and that IndoBERT provides the strongest and most stable generalization for cooperative supervision classification.
Abstract
Cooperative supervision reports contain complex narrative structures and overlapping administrative terminology, complicating automatic classification into governance, risk profile, financial performance, and capital adequacy. Reliable automation is particularly important for accelerating the analysis of supervisory findings while addressing limited labeled data and imbalanced categories. This study aimed to develop and externally evaluate a text-classification framework combining quantitatively validated Generative Artificial Intelligence (GenAI) labeling with conventional and Transformer-based models. Data comprised 294 preprocessed sentences collected from the Department of Cooperatives, Small and Medium Enterprises, Industry, and Trade of Semarang Regency during 2023–2025. Few-shot annotations were generated using ChatGPT, Gemini, Perplexity, and DeepSeek, and three-model combinations were evaluated using the Content Validity Index (CVI); majority voting from the best combination established ground truth. TF-IDF with Logistic Regression and Support Vector Machine served as baselines, whereas IndoBERT and IndoRoBERTa represented contextual models. Performance was assessed through stratified five-fold cross-validation and external testing on 58 unseen sentences. ChatGPT–Gemini–Perplexity achieved the highest Scale-Level CVI of 0.898. IndoBERT obtained the best cross-validated F1-score of 0.9099, exceeding IndoRoBERTa (0.8217), Logistic Regression (0.8004), and SVM (0.7863). On unseen data, IndoBERT retained an F1-score of 0.862, compared with 0.759 for IndoRoBERTa. These findings demonstrate that CVI-validated ensemble GenAI can construct consistent labels for low-resource administrative texts and that IndoBERT provides the strongest and most stable generalization for cooperative supervision classification. The framework offers a practical basis for scalable annotation and reliable automated support for evidence-based supervisory decision-making.