Bug validation asks a coding agent to produce an executable witness for a reported bug. The witness combines a concrete input with a testing harness and exposes faulty behavior during execution. Such evidence makes audit findings actionable, yet benchmark evaluation is difficult when cases reuse public historical bugs...
Hao-Min Qi, Xiang-Zhe Xu, Yi-Ming Huang et al.· 0 citations
Reliable automated research requires agents to vet data, verify analyses, and generate hypotheses grounded in trustworthy evidence, potentially reducing routine scientific workload while allowing scientists to focus on interpretation and discovery. Existing benchmarks often only assess analytical task completion or hyp...
Nan Huang, Mario Tapia-Pacheco, Kun Zhou et al.· 0 citations
It is found that predictive fit can diverge from scientific validity, memorization shapes whether models reproduce or move beyond published formulas, and the best-of-N study reveals a selection bottleneck.
Yi-Ming Huang, Zi-Chen Liu, Junxia Cui et al.· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.