An Agent-centric Domain-Specific Language (aDSL) and a role-specialized multi-agent system to close the gap between programmatic interfaces and reasoning strengths of LLMs, which favor semantic structure and spatial relations over fragile numeric choices.
Abstract
Programmatic representations provide a compelling paradigm for 3D content creation, enabling fine-grained edits, interpretability, and explicit structural control. Yet, agentic workflows that rely on large language models (LLMs) to author 3D programs remain brittle, often failing to translate high-level intent into consistent low-level geometry. We attribute this fragility to a mismatch between existing programmatic interfaces and the reasoning strengths of LLMs, which favor semantic structure and spatial relations over fragile numeric choices. In this paper, we jointly design an Agent-centric Domain-Specific Language (aDSL) and a role-specialized multi-agent system to close this gap. aDSL bridges semantic logic and geometric constraints by emphasizing composability and spatial reasoning; it enables agents to manipulate geometry through relational operators instead of brittle absolute coordinates. Building on aDSL, our training-free multi-agent system follows a Plan-Execute-Critic loop to decompose requests, synthesize code, and iteratively repair errors and constraint violations using execution feedback. Experiments show that this co-design improves robustness, controllability, and faithfulness to user intent. Our method outperforms prior LLM-based baselines on text-to-shape and image-to-shape tasks while preserving explicit structure, editability, and interpretability. It also enables downstream applications such as articulated object creation and structured scene composition. Our code is available at https://github.com/sig-pku/aDSL.
Agentic Designer is proposed, a progressive, multi-agent framework that formulates structure-aware interior layout generation as an iterative and constraint-verified decision process and significantly outperforms state-of-the-art methods, demonstrating substantial improvements in strict structural adherence and functional design coherence.
Zhijing Yang, Hao-Cheng Lin, Zhihua Xu et al.· IEEE Transactions on Pattern...· 1 citation
CADIR, an agent-friendly executable intermediate representation for CAD generation and cross-backend editing, and a construction-graph retrieval method for text and image queries that supports both full-graph and subgraph retrieval, enabling agents to leverage complete models and modeling substructures.
Yu Liu, Jingzhe Ni, Yiming Chen et al.· 0 citations
Advances in large language models (LLMs) have sparked interest in automating parametric CAD modeling through natural language. Existing LLM-based approaches often treat CAD modeling as flat text generation, overlooking the hierarchical structure and geometric constraints inherent in CAD programs. We present CAD-Factory, a Text-to-CAD generation system for language-driven CAD modeling that explicitly models the structural and parametric semantics of CAD programs. Our core contribution is a new formulation of CAD generation as structured program synthesis, coupled with a learnable hierarchical CAD program representation that disentangles structural topology from parameter instantiation. Building on this formulation, CAD-Factory adopts a manager–programmer–reviewer architecture: a planning agent infers program hierarchy, a coding agent instantiates symbolic and numerical parameters, and an evaluation module enforces structural validity and geometric feasibility, which supports structure-aware reasoning, constraint-consistent generation, and interpretable program synthesis. An editor–viewer loop further enables part-aware code refinement through visual feedback, supporting iterative and controllable design workflows. We also contribute a process-oriented annotation pipeline and a Text–CAD dataset with parameter-free or partially specified descriptions that reflect real-world design expression. Extensive experiments demonstrate that CAD-Factory significantly improves structural correctness and geometric consistency across diverse CAD code generation and editing tasks, establishing a structured, interpretable, and robust framework for AI-assisted design.
Yang Liu, Daxuan Ren, Yijie Ding et al.· International Conference on...· 0 citations
Native 3D generators now recover impressive mesh geometry from a single image. However, a dense mesh stays soft where a machined object should be sharp, it carries no part decomposition, and it exposes no parameter a user could edit. To address this, we explore the paradigm of 3D shape as code, leveraging and scaling the coding ability of an LLM for 3D modeling. We introduce Procedura, a novel 3D modeling agent framework that writes an object as a procedural assembly, a parametric program whose named parts are joined by typed, machine-checkable mates. From a text prompt, the agent plans the object as an assembly graph and writes the program part by part, solving each placement from the mated frames rather than guessing it, and admitting a part only once compile, mate, and connectivity checks pass. A decoupled vision critic then refines the assembly one diagnosed fix at a time. Moreover, the same graph carries per-part materials and a simulator-validated articulation. We evaluate on P3D-Bench under its assembly judge, and with the same judge on MechBench-36, our hard-surface benchmark. On both, Procedura outperforms state-of-the-art native 3D generators and every prior 3D-code agent on judged quality, produces the sharpest edges of any method we evaluate, and is the only one whose output is an editable, part-structured program.
Youtian Lin, Yikang Yang, Zhanpeng Hu et al.· 0 citations
Creating dynamic and physically realistic 4D worlds from natural language descriptions is both fascinating and challenging. Traditional computer graphics methods rely on manual creation, requiring extensive human effort to fine-tune materials, motions, and visual fidelity. Recent advances in generative foundation models have sparked interest in learning to generate such 4D worlds from large-scale data; however, existing methods still struggle to ensure physical plausibility and controllability. In this work, we take a different path by leveraging foundation models to construct an agentic system that emulates how humans traditionally create 4D worlds, yet automates the entire process. We present GS-Agent, an end-to-end multi-agent framework that integrates physics engines in the loop to generate realistic, dynamic, and controllable 4D physical worlds from natural language. Inspired by how humans build 4D worlds, GS-Agent decomposes the task into entity management, covering 3D asset curation, material tuning, placement, and motion control, and rendering configuration, including camera and lighting manipulation. Multiple agents with distinct expertise interact with the physics engine via code, seek multimodal feedback, and collaborate to iteratively construct 4D worlds that align with the given descriptions. Experimental results show that GS-Agent effectively converts natural language into diverse and physically plausible 4D worlds exhibiting rich interactions among liquids, deformable objects, and rigid bodies, while achieving cinematic camera and lighting control. We envision GS-Agent as a foundation for a new paradigm in 4D world generation, empowering creative content creation and physical AI. Project page at https://umass-embodied-agi.github.io/gs-agent/
Hongxin Zhang, Chun-Tse Lin, Junyan Li et al.· 0 citations
3D geometry editing is a critical yet labor-intensive part of the graphics pipeline, requiring artists to translate creative intent into precise operations in complex professional software. Large language models (LLMs) have shown promise for script-based 3D creation, but script generation is less suited to perception-driven editing of arbitrary existing meshes, where execution must remain visually grounded and untouched regions should be preserved. We present a \emph{visual-centric}, training-free multi-agent system that edits existing 3D meshes directly in Blender by emulating the iterative workflow of human artists. Rather than generating scripts or regenerating geometry, our system operates through the Blender GUI: multimodal LLM agents observe the viewport, reason about the current mesh state, and execute localized edits through simulated user interactions. Experiments on a curated benchmark provide initial evidence that this agentic approach can follow natural language instructions, perform representative localized mesh edits, and preserve the overall identity of the input asset. Our results highlight a complementary regime for language-driven 3D editing: direct in-place modification of existing meshes within the native 3D editing workflow. We view this work as an exploratory step toward visual-centric agentic geometry editing in professional graphics software.
Bo Pang, Jiaqi Pan, Xiaochen Zhang et al.· 0 citations