Mixture-of-Experts (MoE) transformers scale capacity by activating only a few experts per token, but this sparsity creates a hidden reliability problem: when routing is imperfect, load-balanced models may send tokens to experts that are insufficiently trained for the assigned inputs. We propose Distributionally Robust...
NovelAPIBench is presented, an automated diagnostic framework that builds benchmarks around the API knowledge gaps of a target model, identifies novel APIs, generates executable coding tasks with separately controllable knowledge components, and classifies failures into six categories.
Jin-Nuo Liu, Yue Peng, Jin-Hang Niu et al.· 0 citations
CHORD (Coherence-aware Hidden-state Open-generation Reference Distance), a coherence-sensitive distributional metric, is introduced to establish representation design as central to reliable distributional evaluation.
Jin-Nuo Liu, Jun-Hao Zhu, Wei-Feng Jiang et al.· 0 citations
This empirical study investigates whether procedural LLM skills can be represented as directions in activation space and whether vector-space operations over these directions can express skill-level behaviors and finds that procedural skills admit a vector-space representation.
Xun-Yi Jiang, Jun-Da Wu, Yuxin Xiong et al.· 0 citations
EvoGenUI-Bench is introduced, a benchmark for multi-turn interface maintenance comprising 150 five-turn tasks and 750 turns across three scenarios: information presentation, executable interaction, and tool-grounded external state.
Yue Peng, Lan-Ke Xia, Zi-Han Wang et al.· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.