Learning Style, Forgetting Semantics: A Case Study of SFT and RFT on Classification Tasks
Why does supervised fine-tuning (SFT) lead to more forgetting than reinforcement fine-tuning (RFT), even when all teacher demonstrations are semantically correct? We study this question on classification tasks where tokens within each semantic class express the same semantic answer in different styles. The tasks share...
Hao-Dong Liang, Yan-Hao Jin, K. Balasubramanian et al.
· 0 citations