AI Models Now Design Complex LEGO Sets as CAD Files Using Open-Source Tool

Artificial intelligence systems can now generate intricate LEGO models containing thousands of individual pieces, producing detailed CAD files through the assistance of LDraw Nova, a newly released open-source toolkit. The system harnesses the capabilities of frontier AI models such as GPT-6 Astra and Claude Opus 5.5 to construct elaborate designs from simple text prompts, including a 2,175-piece Sakura Garden model. These AI systems employ Python programming to generate the builds without manually specifying the placement of each brick. To date, none of the designs have been physically constructed with actual bricks; developer Carlos Antelo notes he "didn't even try."

According to the GitHub project's README, the approach is straightforward: "Give an AI agent a model idea. Guide it, let it build it." The Sakura Garden design emerged from a single prompt to Claude Opus 5.5 requesting "the most beautiful model that comes to your mind." Antelo released the project on Friday afternoon as the first official version, following three earlier attempts to get the toolkit functioning properly.

How LDraw Nova Works

LDraw is a text-based format in which each LEGO piece is represented by a single line specifying its position, according to Antelo. Multiple applications including LDView, LeoCAD, and Studio can open LDraw files, with each model stored as a separate file. The system operates through a multi-step process: an AI agent creates a plan in JSON that comprehensively describes the model and its subcomponents, converts that plan into a Python program, and then generates the LDraw source code. Once rendered as an image, the agent can examine the result, make modifications, and re-render, repeating this cycle until the design reaches completion. This approach sidesteps "(evil!) geometry math," since AI agents excel at "generating Python code."

LDraw Nova operates as a Docker web application compatible with major AI providers including OpenAI, Anthropic, and OpenRouter. The web interface includes a gallery attributing each design to the model that created it, along with the original prompt. The Sakura Garden and an incomplete Atlas Crane were both generated by Opus 5.5, while Astra produced the Cathedral and Tidal Observatory, and Opus 5 ("not 5.5!") created the Copper Bean apartments. The project itself was developed with assistance from OpenAI's Astra and Anthropic's Claude Opus 5.5.

Current Limitations and Challenges

Constructing these larger models with physical bricks would require sourcing thousands of individual pieces, a challenge Antelo considers impractical. The toolkit has notable constraints: "physics modeling is something Nova is currently lacking," meaning the system handles collision detection but cannot verify structural stability. Operating the system is not cost-free; Antelo estimates that running Astra to build a single Technic mechanism costs approximately $5 in token expenses.

Based on his current experience, Antelo observes that only the most advanced AI models can reliably generate large and accurate designs, and the overall process remains computationally intensive. VR support for the Meta Quest 3 headset is available but suffers from "many issues, performance issues." This project differs significantly from Carnegie Mellon University's LegoGPT, now available as BrickGPT on GitHub, which was trained on more than 47,000 LEGO structures and validates designs for both correctness and physical stability during generation.

Advanced Features and Future Direction

The toolkit can integrate multiple types of AI models. For instance, jev-rerank, described as a "semantic search tool with re-ranking backed by TypeSafe's Jev System One AI model," was employed in Astra's Cathedral design. The Jev decision model refines part searches through re-ranking, though this component is optional. When no TypeSafe API key is available, the system defaults to standard full-text search. This model was recently used successfully in multiple automated Pokémon Red playthroughs.

While Antelo advocates for higher-capacity AI models to achieve optimal results, one of his stated objectives is developing "tooling that can be used by low-end agents to iteratively build." The project's roadmap includes support for minifigures and Technic machines and engines. Using AI agents to accomplish tasks through code generation is not novel, but the sophistication continues to advance, as demonstrated by a demonstration posted Friday in which Astra completed World of Warcraft's orc starting area based on a single prompt.

Antelo's underlying motivation was to develop AI agents "capable of designing buildable, physical things." Currently, the pieces exist only as digital CAD models, though he is contemplating 3D-printing a smaller design to test the concept.

Bridging Digital Design and Physical Production

AI agents can also interface with 3D printing through Model Context Protocol servers. Kiln enables agents to upload files to a printer and initiate printing, while OctoEverywhere's server allows agents to monitor print progress and pause, cancel, or resume jobs. Such integrations could connect the gap between AI-generated code and tangible manufactured objects. Eventually, physical robots could potentially handle the assembly itself, though as Antelo suggests, that removes much of the appeal. For now, anyone interested can attempt to design the LEGO set they have always imagined.