Verifiable instruction-following benchmarks often express each constraint through one fixed template. We test whether scores remain stable when the operational requirement is unchanged but its wording varies. We introduce WISE, a matched evaluation suite and reporting protocol instantiated on exact word count, keyword...
Qi Zhan, Seoyeon Jang, Zi-Han Dong et al.· 0 citations
It is found that weaker agents often fail before producing a valid artifact, whereas stronger agents increasingly fail on structural, geometric, and construction-process requirements, and CADWorld exposes a gap between general GUI competence and reliable execution of persistent, verifiable engineering workflows.
Zi-Han Dong, Yuan-Zhe Liu, Zhi-Yuan Ma et al.· 1 citation
Results suggest that AAPT performs best when candidate actions can be enumerated in advance, whereas reactive execution remains stronger when they cannot, and on an external benchmark, AAPT matches the overall performance of a reactive baseline, although the two methods exhibit complementary strengths.