This work introduces DecisionPivot, a diagnostic benchmark comprising 48 tasks and 40,033 items across seven decision roles that provides fine-grained diagnostics of decision failures that aggregate evaluation alone cannot reveal, and evaluates 40 decision models and 14 general models using documented, model-specific p...
Xia-Lie Zhuang, Ming-Cong Lei, Zun-Hai Su et al.· 1 citation
Terminal agents are moving beyond local operations toward completing real-world professional tasks. Short trajectories can demonstrate commands and tool calls, but they do not fully show how agents accumulate evidence through exploration, use it to inform judgments, continually adjust strategies based on feedback, and...
This synthesis highlights three recurring lessons: output checks, process checks, and human studies provide complementary information; evaluator calibration is specific to the property being assessed; and resource budgets and attempt selection are integral to interpreting performance comparisons.
Liu-Lei Zhang, De-Jing Zhou, Chu-Yue Huang et al.· 0 citations
An embodied kitchen assistant must do more than recognize food in isolated frames. It must track ingredient states over time and integrate visual observations with recipe and nutritional knowledge to support constraint-aware decision-making. We formalize this capability as \emph{Embodied Nutrition Management}: perceivi...
Yue-Ling Wei, Xiang-Chen Wang, Jian-Hui Pan et al.· 0 citations
The Profiling, Investigation, and Judgment (PIJ), comprising 2,500 real homicide cases from five countries, is introduced and it is found that performance degrades systematically as tasks shift from explicit fact extraction to implicit reasoning over unknown suspect profiles.
Yu-Tong Yao, Yan-Jie Cao, Guan-Hua Chen et al.· 0 citations
JustMem is introduced, which stores conversation history as compact atomic memories and adapts memory access along two dimensions to each query and achieves the highest mean accuracy and retrieval recall among the compared memory systems while using substantially fewer generative-model tokens for memory construction an...
Guan-Hua Chen, Yan-Ting Wang, Wen-Jing Zhi et al.· 1 citation
SAFT (Safety-preserving Adaptation via Fine-tuning Transfer), a safety-preserving adaptation framework that decouples task learning from alignment preservation by learning a safety-guided task update on the paired pretrained base model, rectifying task gradients to avoid conflicting directions with respect to a safety...
Zhiwen Ruan, Yan Yang, Zhuocheng Liang et al.· Proceedings of the 32nd ACM...· 0 citations
Clinical education must prepare medical students to conduct safe and coherent patient interviews under conditions of uncertainty. Traditional standardized patient (SP) training is resource-intensive and difficult to scale. We developed a scaffolding-oriented multi-agent Large Language Model (LLM) AI Standardized Patien...
Lu-Ming Yang, Hao-Xian Liu, Si-Qing Li et al.· 0 citations
Long-term language-model agents rely on external memory across interactions. Atomic memories are particularly useful: their fine-grained semantic boundaries enable precise retrieval and direct comparison between observations. Yet accumulating atoms inevitably become redundant, overlapping, or conflicting. Existing meth...
Jianjie Zheng, Peng Lai, Sijie Cheng et al.· 0 citations
A unified reasoning reward model supporting multiple languages and evaluation paradigms, UniRRM uses a staged reasoning chain to dynamically generate task-generic and instruction-specific criteria, enabling fine-grained, input-adaptive judgments while maintaining consistency across languages.
Peng Lai, Yi-Chao Du, Junchao Wu et al.· 1 citation
AlignDiff, a preference data filtering framework driven by intrinsic model signals, first identifies samples with clear preferences using both positive and inverse signals, then prioritizes the more challenging samples based on the average negative log-likelihood gap, encouraging the model to learn richer information f...
This survey provides a comprehensive analysis of reasoning economy in both the post-training and test-time inference stages of LLMs, encompassing the cause of reasoning inefficiency, behavior analysis of different reasoning patterns, and potential solutions to achieve reasoning economy.