ChipAgents and NVIDIA have developed Build-A-Chip-Bench to evaluate AIartificial intelligence agents on the full digital IC1. integrated circuitIC is the common term for an integrated circuit.2. ion chromatographyAnalytical method that separates and identifies the components of a mixture. design flow, from workload specification through verified GDS1. graphical design systemSoftware environment for creating technical designs and layouts graphically.2. graphical design softwareSoftware-related method or tool for development, control or data processing.3. Graphic Data SystemGDS is a data format for layout data in semiconductor design.4. Graphic Design SystemGDS is the final layout database format used for the benchmark’s physical output.. The benchmark is aimed at a gap in current chip-design AIartificial intelligence evaluation: passing isolated functional tasks is not enough when the real challenge is to carry a design through RTL1. resistor-transistor logicSemiconductor device or transistor technology used to control electrical currents and signals.2. register transfer levelRTL is the hardware description layer where design assumptions are often captured.3. Register-Transfer LevelRTL describes the hardware implementation stage targeted by the benchmark., verification, synthesis, and physical implementation without breaking power, performance, and area targets.
The benchmark is designed around realistic, moderately large flows, with designs ranging from 100K to 5M standard cells. Each task provides a workload specification, a fixed interface, and an open-source EDA1. electronic design automationSoftware tools for designing, simulating, verifying and laying out electronic circuits.2. equipment data acquisitionSystem or method for collecting, storing, exchanging or analyzing data. environment. The agent must handle architecture, microarchitecture, RTL1. resistor-transistor logicSemiconductor device or transistor technology used to control electrical currents and signals.2. register transfer levelRTL is the hardware description layer where design assumptions are often captured.3. Register-Transfer LevelRTL describes the hardware implementation stage targeted by the benchmark., verification, and physical implementation, while hidden verification IP1. image placement2. Internet protocolInterface or communications concept for transmitting data between devices and systems.3. Intellectual PropertyIP denotes licensable semiconductor design blocks or technology elements. checks behavior against held-out tests and a separate grader evaluates the final submission.
One representative task asks for an INT4 inference accelerator for Qwen3-0.6B using Verilator, OpenSTA, OpenROAD, and the Nangate45 PDKprocess design kitA PDK provides the technology files and design rules needed to design chips for a specific process.. The implementation must meet a 400 MHzmegahertzFrequency unit equal to one million cycles per second. timing target, fit within 6 mm², deliver at least 15 decode tokens per second, and stay below 10 W, with memory bandwidth fixed at 12.8 GB/s. The agent is also expected to choose the datapath architecture, buffering, scheduling, testbenches, EDA1. electronic design automationSoftware tools for designing, simulating, verifying and laying out electronic circuits.2. equipment data acquisitionSystem or method for collecting, storing, exchanging or analyzing data. flow scripts, and, where needed, manual floorplanning and placement.
The benchmark scores designs only if they pass functional verification and then a physical gate requiring a routed GDS1. graphical design systemSoftware environment for creating technical designs and layouts graphically.2. graphical design softwareSoftware-related method or tool for development, control or data processing.3. Graphic Data SystemGDS is a data format for layout data in semiconductor design.4. Graphic Design SystemGDS is the final layout database format used for the benchmark’s physical output. layout without hold-time or design-rule violations. Only then is PPA1. power, performance, and areaPPA is the main optimization target for advanced-node designs.2. power, performance, area scored against the task constraints; failures receive zero. Across the evaluated tasks, Claude Fable 5.1 reached the highest overall score at 16.6%, with functional and GDS1. graphical design systemSoftware environment for creating technical designs and layouts graphically.2. graphical design softwareSoftware-related method or tool for development, control or data processing.3. Graphic Data SystemGDS is a data format for layout data in semiconductor design.4. Graphic Design SystemGDS is the final layout database format used for the benchmark’s physical output. pass rates of 65% and 75%. GPT-6 Astra achieved the strongest pass rates at 85% for both gates, but its PPA1. power, performance, and areaPPA is the main optimization target for advanced-node designs.2. power, performance, area results lowered its overall score to 8.7%.
The runs also exposed where current agents still struggle. Some attempts improved physical closure at the cost of functional regressions, while others delayed architectural fixes and spent hours trying routing options before making the underlying design change. A single place-and-route iteration could take more than four hours, making runtime allocation and sequencing part of the challenge.
Those failure modes matter because the benchmark is not intended to measure only whether an agent can produce correct RTL1. resistor-transistor logicSemiconductor device or transistor technology used to control electrical currents and signals.2. register transfer levelRTL is the hardware description layer where design assumptions are often captured.3. Register-Transfer LevelRTL describes the hardware implementation stage targeted by the benchmark.. It is meant to show whether optimization steps preserve verified behavior, whether physical failures can be traced back to architectural causes, and whether the agent can converge on a design that is still valid after implementation. Under the tested toolchain and resource budgets, the benchmark suggests that AIartificial intelligence can already execute a specification-to-GDS1. graphical design systemSoftware environment for creating technical designs and layouts graphically.2. graphical design softwareSoftware-related method or tool for development, control or data processing.3. Graphic Data SystemGDS is a data format for layout data in semiconductor design.4. Graphic Design SystemGDS is the final layout database format used for the benchmark’s physical output. flow, but still has difficulty maintaining correctness while closing on PPA1. power, performance, and areaPPA is the main optimization target for advanced-node designs.2. power, performance, area.
The project is being extended with industry partners toward a live leaderboard and a broader task set, with the goal of comparing models on more demanding end-to-end chip-design workloads.