Skip to content

EngiWorld: What Can Frontier Agents Deliver in Professional Engineering Environments?

Sep 2026 · 0 citations · 48 references
Computer Science

TL;DR

EngiWorld provides the first rigorous foundation for measuring progress toward agents that operate professional engineering software end to end, and introduces an artifact-centric evaluation methodology built on a unified domain-verifier suite.

Abstract

Autonomous agents have made rapid progress in general-purpose computer use, yet reliable automation of professional industrial engineering remains out of reach. Engineering workflows require reasoning over geometric and physical constraints and keeping dependencies consistent across software and design stages. We present EngiWorld, a benchmark designed around the capabilities that real engineering projects require, from operating a single application to solving engineering problems end to end. It contains 1,301 expert-curated tasks spanning six engineering domains (CAD, CAE, CAM, BIM, EDA, and DCC) and 26 professional software applications, with both GUI and CLI interfaces. The tasks are organized into six types, corresponding to capabilities that real engineering projects require. We further introduce an artifact-centric evaluation methodology built on a unified suite of domain verifiers. The verifiers programmatically check whether final and intermediate artifacts are geometrically valid, physically feasible, and compliant with design rules. Beyond binary success, they also score how well optimized designs meet their specifications. Evaluation of seven frontier models reveals a substantial capability gap: the strongest model scores only 44.7/100, and just 3.6% of runs on tasks that require cross-software coordination succeed. EngiWorld provides a rigorous foundation for measuring progress toward agents that operate professional engineering software end to end.

View source

Similar papers

#artificial intelligence Preprint Sep 2026

CADWorld: Computer-Use Benchmark for Long-Horizon Computer-Aided Design

It is found that weaker agents often fail before producing a valid artifact, whereas stronger agents increasingly fail on structural, geometric, and construction-process requirements, and CADWorld exposes a gap between general GUI competence and reliable execution of persistent, verifiable engineering workflows.

Zi-Han Dong, Yuan-Zhe Liu, Zhi-Yuan Ma et al. · 1 citation

CUA-SWE: When Computer-Use Agents Meet Visual Software Engineering

This evaluation characterizes how frontier agents combine source-level execution with application screenshots and graphical interaction to produce verified software changes, and examines performance across domains and task information requirements, alongside the development behaviors associated with successful repairs.

P. Wang, Chen-Hao Liang, Ze-Long Xu et al. · 2 citations
#artificial intelligence Preprint Oct 2026

ArtifactArena: Evaluating Models by What They Build in the Physical World

To evaluate the frontier, we must measure models not by what they say, but by what they can engineer and build in grounded physical environments. We introduce \textsc{ArtifactArena}, an open-ended platform where models face a physically grounded hardware-software co-design challenge: engineering fully functional robots...

Kushagra Tiwary, David Mayo, Nikhil Behari et al. · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.