A local recovery guarantee for teachers sufficiently close to the initial student in a shared parameterization under regularity conditions is proved, offering a conditional explanation for the success of self-RL teachers in the authors' experiments.
Lei Zhao, Qi-Chao Zhao, Bo-Wen Zuo et al.· 0 citations
Verifiable instruction-following benchmarks often express each constraint through one fixed template. We test whether scores remain stable when the operational requirement is unchanged but its wording varies. We introduce WISE, a matched evaluation suite and reporting protocol instantiated on exact word count, keyword...
Qi Zhan, Seoyeon Jang, Zi-Han Dong et al.· 0 citations
Video multimodal large language models (MLLMs) keep climbing video question answering benchmarks, yet shuffling the frames, masking the segment that supports the answer, or occluding the target object barely changes their predictions. The accuracy rests on appearance and language priors, not on the temporal evidence th...
Zhaolu Kang, Shi-Yu Liu, Tai-Long Luo et al.· 0 citations
DTC is introduced, a framework that estimates confidence by counting tokens at which two models strongly disagree during decoding, and finds that their count is almost negatively associated with answer accuracy, thereby serving as a simple yet effective signal for uncertainty quantification.
Fei-Yang Li, Sheng-Jing Liu, Qi Zhan et al.· 0 citations
It is found that weaker agents often fail before producing a valid artifact, whereas stronger agents increasingly fail on structural, geometric, and construction-process requirements, and CADWorld exposes a gap between general GUI competence and reliable execution of persistent, verifiable engineering workflows.
Zi-Han Dong, Yuan-Zhe Liu, Zhi-Yuan Ma et al.· 1 citation
Results suggest that AAPT performs best when candidate actions can be enumerated in advance, whereas reactive execution remains stronger when they cannot, and on an external benchmark, AAPT matches the overall performance of a reactive baseline, although the two methods exhibit complementary strengths.
A three-tier diagnostic taxonomy shows that verification/feedback and planning failures dominate execution/grounding errors, while a single scalar success rate can not explain, and connects these findings to newer long-horizon CUA benchmarks and derive stage-specific design rules for CUA evaluation.
Zi-Han Dong, Zhiyuan Ma, Zekun Wang et al.· arXiv.org· 2 citations
OrderProbe is introduced, a deterministic benchmark for structural reconstruction using fixed four-character expressions in Chinese, Japanese, and Korean, which have a unique canonical order and thus support exact-match scoring.