Large language models can recognize when they are being evaluated and change their behavior accordingly, undermining the reliability of safety benchmarks. Evaluation awareness (EA) is usually measured by having an LLM judge scan reasoning traces for explicit statements of recognition, but this signal is increasingly un...
Daniel Donnelly, E. Chapple, Haritz Puerto et al.· 0 citations
It is found that agents spontaneously develop informal institutions: persistent rules and commitments grounded in reputation, reciprocity, and precedent that carry across tasks and influence resource allocation, with agents sometimes forgoing immediate advantages to honor earlier agreements.
Abhinav Kumar, Rediet Abebe, E. Bagdasarian et al.· 1 citation
As autonomous agents become widely deployed, coordinating access to shared computing resources and services will become increasingly important. Recent incidents have already revealed unplanned coordination between agents, highlighting the safety implications of these interactions. Yet prior studies of agent negotiation...
Abhinav Kumar, Rediet Abebe, E. Bagdasarian et al.· 0 citations
Evaluation of leading models reveals a consistent trade-off: over the course of a sustained interaction, current systems either slowly drift to sycophancy or over-correct into robotic detachment, demonstrating that balancing honesty with appropriate support throughout a natural conversation remains a critical, unsolved...
Sidharth Pulipaka, R. Binkytė, Ivaxi Sheth et al.· 0 citations
The framework and EvalAwareBench provide the tools to measure, attribute, and mitigate evaluation awareness, building the foundation for future solutions to ground evaluation awareness in social psychology.
Changling Li, Terry Jingchen Zhang, Jie M. Zhang et al.· arXiv.org· 1 citation
Verbalization training is introduced, a method for making LLMs less reticent about verbalizing evaluation awareness while avoiding to supervise the latent belief itself, and it is shown that VT-induced verbalizations reflect the richer knowledge acquired by the model.
Usman Anwar, Sahar Abdelnabi, David Krueger· 0 citations
This work introduces SEABench, a benchmark for studying endogenous misalignment arising from agent self-evolution, with 48 longitudinal task sequences that span multiple evolution surfaces, task domains, and harm types in a rich personal-assistant environment and shows that qualitatively different safety behaviors emer...
Saswat Das, Parvati Viswanathan, Daniel Donnelly et al.· arXiv.org· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.