Skip to content

CADWorld: Computer-Use Benchmark for Long-Horizon Computer-Aided Design

Sep 2026 · 1 citation · 37 references
Computer Science

TL;DR

It is found that weaker agents often fail before producing a valid artifact, whereas stronger agents increasingly fail on structural, geometric, and construction-process requirements, and CADWorld exposes a gap between general GUI competence and reliable execution of persistent, verifiable engineering workflows.

Abstract

Computer-use agents are increasingly evaluated in realistic desktop environments, but existing benchmarks provide limited coverage of professional engineering workflows whose outputs are persistent, structured artifacts. Mechanical computer-aided design (CAD) is a particularly demanding setting: an agent must manipulate geometry and constraints over long interaction horizons while producing a native project whose dimensions, construction structure, and downstream engineering state remain valid. We introduce \textbf{CADWorld}, a benchmark for long-horizon computer use in FreeCAD. CADWorld contains 200 tasks spanning 11 mechanical-CAD workflow categories, including sketching, part modeling, assembly, CAM, FEM, measurement, mesh processing, and technical drawing. Agents operate through screenshots and GUI actions, while success is determined by task-specific executable checks over saved FreeCAD artifacts and auxiliary outputs, covering geometric properties, parametric structure, constraints, manufacturing state, and simulation results. Across seven current agents on the full benchmark, the strongest agent achieves 17.5\% success, compared with an 87.0\% expert reference pass. We find that weaker agents often fail before producing a valid artifact, whereas stronger agents increasingly fail on structural, geometric, and construction-process requirements. CADWorld therefore exposes a gap between general GUI competence and reliable execution of persistent, verifiable engineering workflows. Project accessible at https://cad-world.github.io.

View source

Similar papers

#artificial intelligence Preprint Sep 2026

EngiWorld: What Can Frontier Agents Deliver in Professional Engineering Environments?

EngiWorld provides the first rigorous foundation for measuring progress toward agents that operate professional engineering software end to end, and introduces an artifact-centric evaluation methodology built on a unified domain-verifier suite.

Hong-Cheng Gao, Hai-Long Qu, Yu-Ang Lei et al. · 0 citations
#large language models Open access Sep 2026

A multimodal benchmark for editable constraint preserving history based CAD modeling

Editable CAD generation requires access to procedural history rather than final geometry alone, yet widely used CAD datasets typically omit explicit geometric constraints, native parametric assets, or rich semantic labels. We introduce EPICCAD, a multimodal benchmark for history-based CAD modeling that aligns compact c...

Ze-He Yin, Wei-Xin Lin, Xiangbiao Kong · 0 citations
#machine learning Preprint Oct 2026

StepCAD: Mesh-to-CAD Code Generation via LLM Policy and Geometry-Guided Search

Recovering executable CAD programs from 3D meshes is challenging due to the compositional nature of CAD construction and the interaction between discrete modeling choices and continuous parameters. Many learning-based methods predict complete programs in a single pass and rely predominantly on sketch-extrude representa...

Ghadi Nehme, Faez Ahmed · 0 citations
Open access Sep 2026

Objectives and Key Results–Driven Multiagent Framework for Mechanical Design and Simulation

Advanced structural and materials design increasingly depends on complex geometries and microstructures spanning continuum and atomistic descriptions. Yet the early idea–design–simulation loop remains manual: engineers translate informal intent into modeling assumptions, select physics, assemble solver inputs, and revi...

Jie Tian, Lin Pang, Ji-Xin Hou et al. · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.