This evaluation characterizes how frontier agents combine source-level execution with application screenshots and graphical interaction to produce verified software changes, and examines performance across domains and task information requirements, alongside the development behaviors associated with successful repairs.
P. Wang, Chen-Hao Liang, Ze-Long Xu et al.· 0 citations
This work uncovers short context additions that redirect 61.4%-73.2% of each decision system's initially correct decisions toward a wrong option fixed in advance, exposing a weakness in current decision models.
Zi-Xiang Xu, Zirui Song, Chi-Yu Zhang et al.· 0 citations
How much does the noisy measurement add to learned quantum error mitigation? An accuracy table cannot say, because a model handed circuit structure can score well without reading the measurement at all. QEMScore adds the comparison that can. Each simulated circuit carries an exact ideal answer. The learned mitigator is...
Yue Zhao, Huayue Gu, Yu-Shun Dong et al.· 0 citations
Diverse Skill Routing is proposed, a diversity-aware reranking framework that uses a Determinantal Point Process to balance relevance and non-redundancy and improves recall and full coverage over a strong pointwise reranking baseline, with larger gains on multi-skill queries.
Wang Wei, Tiankai Yang, Samyadeep Basu et al.· 2 citations
WeClawArena is introduced, an auditable benchmark and runtime sandbox for multi-party owned-agent collaboration over personal workspaces and audits attack success from bounded runtime evidence, supporting diagnosis of task breakdown, privacy leakage, poisoned evidence, and invalid authority paths.
P. Wang, Ao-Jie Yuan, Haiyu Zhang et al.· 1 citation
CatchBench puts one auditor's question to three information states: the declared configuration before a run (PRE), a growing prefix of its trace (LIVE), and the finished trace (POST), which none scores all three under one task-method interface.
Yue Zhao, Meng-Yuan Li, Ruo-Lin Li et al.· 4 citations· ⚡1
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.