Large language models are increasingly used to simulate how individuals respond to new situations, yet the behavioral reasoning behind these responses is either inherited from pretraining or learned from individual-level annotations, which offer limited behavioral diversity and little supervision of the reasoning itsel...
Yi-Ning Zhao, Bu-Shi Liu, Hao-Fei Yu et al.· 0 citations
The RubrIc-Failure Taxonomy (RIFT), of nine possible ways a rubric fails, organized under reliability and content validity, is introduced, and quality signals based on the rubric's content are imported to show the signals track failure causally.
Ankit Aich, Zheng-Yang Qi, Charles Dickens et al.· 2 citations
These results show that current agents are still far from professional-level computer use: rather than stumbling on basic GUI control or coding, they lose track of constraints, miss information that arrives mid-task, guess rather than ask the user, and skip verification, struggling most when a task hinges on hidden sta...