SecPO: Principled Adversarial Training for Prompt Injection Security
An idealized reference model---an injection-invariant, utility-preserving oracle---can nevertheless be approximated, as its output distribution in training is interestingly accessible, which motivates a theoretically principled SecPO loss.
Si-Zhe Chen, Yi-Bo Peng, Jaewon Chang et al.
· 1 citation