Video world models can produce visually convincing yet physically inconsistent sequences, raising concerns about their reliability for prediction and planning in embodied AI systems. Existing evaluations often rely on model-based judgments or reference videos, while direct physical tests largely focus on mechanics. We...
Ming-Ju Gao, Qing-Le Liu, Yu-Zhao Peng et al.· 0 citations
Coding agents can construct executable programs and revise them through feedback, offering a foundation for embodied control without target-task fine-tuning. Through a programmatic robot interface, the agent observes the scene, executes control programs, and adapts them to the resulting physical state, serving as the c...
Wen-Yi Li, Yi-Zhe Chi, You-Jie Zheng et al.· 0 citations
PPTBench establishes a measurable testbed for studying visual coding and advancing agents toward more reliable visual creation, and shows that agents can generally produce valid slide files, but struggle to produce high-quality reconstructions that faithfully recover the semantics and visual structure of the target.
Xiao-Qiu Wang, Yi-Zhe Chi, Wen-Yi Li et al.· 0 citations
This work presents AI4AI\mbox{-}Bench, 10 frozen research repositories spanning 10 training algorithm families, and releases the task suite, the evaluators and every scored submission, so that the measurement can be repeated as these systems change.
Yi-Zhe Chi, Wen-Yi Li, De-Yao Hong et al.· 11 citations· ⚡3
SWE Refactor Bench is introduced, a benchmark comprising 20 whole-repository migrations, covering 4 kinds of technical debt, and SWE Refactor Bench is positioned as a rigorous testbed for developing coding agents for reliable whole-repository migrations.
De-Yao Hong, Yi-Zhe Chi, Wen-Yi Li et al.· 4 citations· ⚡2
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
