LLM Benchmarking via Representation Multi-task Learning
A novel statistical framework based on representation Multi-task Learning (MTL) and an item response theory model is proposed, and overall and domain-specific LLM traits are defined through an Item Response Theory (IRT) model and an MTL approach to estimate these traits from item-level response data.