Carolyn Rose

Useless but Safe? Benchmarking Utility Recovery with User Intent Clarification in Multi-Turn Conversations

CarryOnBench is introduced, the first interactive benchmark that measures whether LLMs can revise their interpretation of user intent and recover utility, while remaining safe through multi-turn conversations, and Ben-Util, a checklist-based metric that evaluates how well each model response fulfills the user's benign...