A comparative study of RL-finetuned and base models in data science tasks

Abstract

The present research carries out an expansion on the work of Yue carries out the analysis on Reinforcement Learning (RL) fine-tuning which is from mathematical reasoning to data science classification tasks. Utilizing Qwen2.5-1.5B base model and RL finetuned models upon Diabetes (low-quality) and Breast Cancer (high-quality) data sets, the pass@k metric (k=1,3,5) has been evaluated by us. Our outcomes have brought about three main discoveries. Firstly, data quality is the factor that critically decides RL fine-tuning's value: the RL model has obviously better performance than the base model on low-quality data (80% vs 32%) but gets the same accuracy on high-quality data (87.0%). Second, pass@k provides restricted worth in classification tasks, because multiple sampling cannot enhance accuracy—its fundamental difference lies from mathematical reasoning. Third, when we carry out the validation work on the single-sampling superiority of RL[sZ2.1] models, therefore, we have not observed the reversal of base models in the situation of multiple sampling. This research makes clear that data quality and task category are key adjusting factors in RL assessment, thus offering practical direction: RL fine adjusting