Author
Max Planck Institute for Intelligent Systems
We have 3 of 16 papers
We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.
Not the right person? Other researchers publish under this name.
Models are usually aligned based on their observed outputs, using demonstrations, preference data, or reward signals. These objectives reward responses that look aligned. More capable models may learn to satisfy them without internalizing the intended behavior, for example by faking compliance during training. Such sup...
It is argued that a more realistic threat model includes further training with reinforcement learning after distillation, and it is shown that simple attacks can steal reasoning capabilities from existing closed-source language models using data easily obtainable from current APIs, yielding reasoning improvements equiv...
An architectural vulnerability is identified that circumvents anti-distillation mechanisms, allowing adversaries to extract a proprietary model's reasoning, as well as proposing concrete cryptographic and system-level mitigations to secure client-side reasoning.
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
