Coding agents combine visual understanding with program execution, offering a new route to automated 3D creation. By translating images and instructions into editable programs, they bring reconstruction, generation, and editing into a shared workflow. Yet existing benchmarks fragment evaluation across task scopes and p...
Xin-Jie Lin, Qing-Le Liu, Wen-Yi Li et al.· 0 citations
Large language model agents are increasingly deployed as multi-agent systems rather than as a standalone model operating within a single context. Recent agent harnesses allow the model to instantiate worker agents and to author the executable workflow that coordinates them, shifting the design of system structure from...
Zheng Jiang, Fang Niu, Bo-Kai Peng et al.· 0 citations
Quantitative research is an iterative loop: a researcher proposes a market hypothesis, implements it as an executable signal, tests it on historical data, and decides what to keep. Existing benchmarks score formulas, code, or individual trading decisions, leaving open whether large language models (LLMs) can complete t...
Hou-De Qian, Zheng Jiang, Wen-Yi Li et al.· 0 citations
Video world models can produce visually convincing yet physically inconsistent sequences, raising concerns about their reliability for prediction and planning in embodied AI systems. Existing evaluations often rely on model-based judgments or reference videos, while direct physical tests largely focus on mechanics. We...
Ming-Ju Gao, Qing-Le Liu, Yu-Zhao Peng et al.· 1 citation
EmbodiedEvo is a self-evolving coding agent that distills physical feedback into reusable code Skills and diagnostic Lessons that improves weaker executors and shows the largest gains on memory-dependent tasks and successful reuse of coordinated procedures.
EmbodiedEvo is introduced, where a coding agent evolves reusable Skills and Lessons through feedback on a single Playground scene, then freezes the resulting harness for diverse new scenes, to retain and refine experience for later scenes.
PPTBench establishes a measurable testbed for studying visual coding and advancing agents toward more reliable visual creation, and shows that agents can generally produce valid slide files, but struggle to produce high-quality reconstructions that faithfully recover the semantics and visual structure of the target.
Xiao-Qiu Wang, Yi-Zhe Chi, Wen-Yi Li et al.· 0 citations
Text-to-music language models begin with a choice usually made by default: how to tokenize music, and the ordering is clean and surprising: representation, not model size, is the binding variable for distributional fidelity.
Junhao Chen, Ming-Jin Chen, Jingjia Mao et al.· 1 citation
This work presents AI4AI\mbox{-}Bench, 10 frozen research repositories spanning 10 training algorithm families, and releases the task suite, the evaluators and every scored submission, so that the measurement can be repeated as these systems change.
Yi-Zhe Chi, Wen-Yi Li, De-Yao Hong et al.· 11 citations· ⚡3
SWE Refactor Bench is introduced, a benchmark comprising 20 whole-repository migrations, covering 4 kinds of technical debt, and SWE Refactor Bench is positioned as a rigorous testbed for developing coding agents for reliable whole-repository migrations.
De-Yao Hong, Yi-Zhe Chi, Wen-Yi Li et al.· 4 citations· ⚡2
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.