When a forecasting system already has a prediction from a market, crowd, or statistical model, when should it use a language model? Standalone accuracy cannot answer this question because a less accurate model may still improve the existing forecast. We study this distinction through relative competence, separating a m...
LLM-based agents are increasingly capable of generating complex 3D structures, with the potential to reshape how objects are designed and realized in the physical world. Yet, producing elegant geometry is fundamentally different from producing objects that can be built and perform their intended functions. Existing eva...
Jia-Teng Liu, Ru-Shi Wang, Cheng Qian et al.· 0 citations
Video models increasingly use memory to preserve information over long sequences, with the assumption that gains come from retrieving and using the correct past content. Standard memory ablations test whether memory helps, but not whether the retrieved content is responsible. We test this directly with read-time memory...
Aditi Tiwari, Akshit Bhalla, D. Prasad et al.· 0 citations
An unsuccessful LLM agent rollout contains more information than its final reward: the observations available to the agent, the actions it chose, and the environment's responses. Reusing this experience for learning requires identifying a decision to revise and testing a concrete alternative. We introduce the Agent Err...
Kun-Lun Zhu, Xu-Yan Ye, Yi-Bo Li et al.· 1 citation
Across Harness-Bench and NewtonBench, full-bank meta-skills improve macro-average performance by 8.95 percentage points over no-skill construction, and 12.02 points over direct delivery of the same bank to the Target.
Cheng Qian, Kun-Lun Zhu, Bei-Bin Li et al.· 0 citations
A competence gate is introduced that estimates domain-level source weights from resolved outcomes, shrinks uncertain estimates toward a global weight, and recalibrates the pooled forecast and provides a practical approach for selective model use based on measured marginal value.