Large language model (LLM) agents are increasingly capable of supporting scientific workflows, but their capabilities across diverse research activities remain difficult to assess systematically. Existing benchmarks often focus on selected scientific capabilities or task types, limiting a comprehensive characterization...
Tian-Yun Zhong, Wei Wang, Jia-Ling Li et al.· 0 citations
AOE (Arranged and Organized Extraction), a bilingual benchmark that evaluates LLMs'ability to distill long-form, multi-document inputs into structured tables, is introduced, revealing that structured extraction from long contexts remains a fundamental challenge - an"Area of Effect"(AOE) that impacts all tested systems.
Tian-Yun Zhong, Guozhao Mo, Yan-Jiang Liu et al.· arXiv.org· 0 citations
Results support treating PHD as a distinct target for evaluating how LLMs formulate investigative directions when final conclusions are withheld, with gains for several lower-performing models on HypoArena but regressions for other systems, including a top-performing model.
Tianyun Zhong, Wangyi Jiang, Wei Wang et al.· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.