We Keep Evaluating LLMs, But Are We Learning Anything?

Abstract

Large language model (LLM) research has become increasingly dominated by benchmark-oriented evaluations that measure performance on predefined tasks and datasets. While such evaluations are useful for tracking progress, they often provide limited scientific insight into how these systems reason, generalize, fail, or support real-world applications. In this article, we argue that LLM research should move beyond incremental leaderboard improvements and adopt a broader scientific agenda inspired by the historical evolution of machine learning research. Drawing analogies with established research areas in machine learning, we propose a set of guidelines for conducting more rigorous and impactful LLM research. These guidelines include investigating internal reasoning mechanisms, developing calibration and uncertainty estimation methods, addressing bias and fairness systematically, studying generalization to novel domains, exploring benchmark-free evaluation paradigms, improving methodological transparency, and, more ambitiously, using LLMs as instruments for scientific discovery rather than merely as objects of evaluation.