Climbing the Hill: Prompt Injection Red-Teaming Against Frontier Models with Curriculum Reinforcement Learning
This work proposes a curriculum learning-based method to address the cold-start problem and can effectively red-team frontier LLMs, and shows that the attacker LLM transfers across targets, e.g., an attacker LLM trained to defeat one strong LLM also succeeds against six other frontier LLMs it was never trained on.
Chen-Long Yin, Xiao-Long Jin, Wei Zou et al.
· 0 citations