Evaluating SLMs for Predicting Tabular Data: An Essay on Higher Education Dropout

Abstract

Student dropout in higher education represents a structural challenge for educational institutions, resulting in both academic and financial losses. Although classical machine learning models are successfully applied to this task, the consolidation of Large Language Models (LLMs) has enabled new predictive approaches. This study investigates the effectiveness of small-scale LLMs (7-14B parameters) adapted via Parameter-Efficient Fine-Tuning (PEFT) compared to specialized tabular-data models. Utilizing a UFPel dataset with 6,011 records, we evaluated three representation strategies: scaled (traditional), labeled (discretized), and storytelling (narrative), inferred by the Phi-4, Qwen2.5-7B, and Mitra models. The results indicate that the specialized tabular model (Mitra) outperformed the LLMs, achieving 80.30% accuracy and an F1-Score of 0.8203, compared to 72.07% and 0.7176 for the best-performing LLM (Phi-4 scaled). The labeled and storytelling linguistic formats yielded performance levels comparable to simple numerical formatting, invalidating the narrative advantage hypothesis in static prediction. Evidence shows that 4-bit-quantized small-scale LLMs perform worse on isolated tabular tasks. We conclude that tree-based and foundationally tabular classifiers remain the most effective architecture for dropout prediction under resource-constrained conditions.