Shihan Dou

SPIEval: Evaluating Large Language Models as Mobile Assistants over Scattered Personal Information

Large language models (LLMs) are increasingly deployed as mobile assistants, where a key challenge is leveraging personal information scattered across multiple applications (apps) to complete user instructions. However, due to the lack of dedicated benchmarks, their capabilities remain poorly understood. To address thi...

MathCritique: Enhancing LLM Reasoning via Critique Models with Test-Time and Training-Time Supervision

JFTA-Bench: Evaluate LLM's Ability of Tracking and Analyzing Malfunctions Using Fault Trees

A novel textual representation of fault trees is proposed, and a benchmark for multi-turn dialogue systems that emphasizes robust interaction in complex environments is constructed, evaluating a model's ability to assist in malfunction localization.

Agents in the Large: Perception-Centered Architecture for Persistent Agents

Pera describes a persistent agent organized around perception and control components that continually perceive service-relevant signals from episodic task executions, internal context, and changes in the surrounding environment, and use these signals to construct lifecycle tasks.

MemArbiter: Decision-Time Memory Arbitration for Long-Horizon LLM Agents

Results show that function-aware memory arbitration enables accessible information to guide actions more effectively, and improves post-failure recovery and reduces failed-action repetition and state-action recurrence.