Distillation Defenses Easily Break After Reinforcement Learning
It is argued that a more realistic threat model includes further training with reinforcement learning after distillation, and it is shown that simple attacks can steal reasoning capabilities from existing closed-source language models using data easily obtainable from current APIs, yielding reasoning improvements equiv...
Shidan Javaheri, Alexander Panfilov, O. Britton et al.
· 0 citations