MineCEraft: Evaluating Language Models as Construction Engineers in the World of Minecraft
The MineCEraft benchmark comprises 723 domain-expert hand-crafted natural-language instructions with programmatically verifiable evaluation, providing a safe and controllable experimental environment for assessing LLMs'ability to perform realistic construction engineering tasks.
Sewoong Lee, Risham Sidhu, J. Hockenmaier et al.
· 0 citations