This work evaluates compaction on later questions of language-model agents, using four corpora that annotate the evidence of several questions per text and recorded agent trajectories, and compares every method with random deletion at the same budget.
Jun-Wei Yao, Yong-Xin Zhou, Zi-Ke Yang et al.· 0 citations
Automatic safety judges -- for example Llama Guard, or GPT-4o given a grading prompt -- produce the numbers behind reported jailbreak success rates, defense evaluations, and safety leaderboards. We ask whether these judges grade what a reply contains or how it sounds. We place a fixed piece of text before or after a mo...
Yong-Xin Zhou, Wen-Bo Ye, Yuan-Zhe Liu et al.· 0 citations
A benchmark score is a measurement instrument, yet most benchmarks read each item at a single canonical surface form. We ask whether that reading is faithful: when an item's intent is held fixed and only its meaning-preserving surface form varies, does the canonical-form score estimate model behavior well, and how much...
Yongxin Zhou, Jun-Wei Yao, Yuanzhe Liu et al.· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.