The findings go against current endeavors of defining, identifying, and targeting general intelligence as a tangible construct in language model development, since the strategy of targeting a single conceptual ability without support is left without support.
A reusable three-layer checklist-and-judge framework that evaluates interpreter-mediated conversation across semantic, pragmatic, and cultural-social dimensions, covering the naturalness, intent, and social appropriateness that fidelity metrics leave unmeasured is introduced.
Large language models (LLMs) display a unified"general factor"of capability across 10 benchmarks (a finding confirmed by our factor analysis of 156 models), yet they still struggle with simple, trivial tasks for humans. This is because current benchmarks focus on task completion, failing to probe the foundational cogni...