Scalable Weak-to-Strong Verifier Hardening for Agent Environments
Agent environments are essential for large language model (LLM) development. They use programmatic verifiers to turn complex outcomes into rewards. Verifiers can be attacked into awarding full credit to cheating agents without completing the intended task, causing inaccurate evaluation and misalignment in training. As...
Chong-Fang Zeng, Zi-Qian Zhong, Ke-Xun Zhang et al.
· 2 citations