Danylo Smahliuk

EuroExec: Frontier Language Models Fall Short of Expert Judgment on European Executive Decision Tasks

Frontier LLMs are increasingly put to use on open-ended complex questions, different in nature from the ones they are typically evaluated on. We dedicate more than 4,000 human expert hours to evaluate a selection of six frontier LLMs on a member of this class of problems: EuroExec, our introduced human expert-based ben...