[Paper Review] ChipGPT: How far are we from natural language hardware design

Kaiyan Chang, Ying Wang+6 more|arXiv (Cornell University)|May 23, 2023

Ferroelectric and Negative Capacitance Devices27 citations

TL;DR

ChipGPT demonstrates a four-stage zero-code framework that uses LLMs to generate Verilog from natural language specifications, with an output manager and enumerative search to optimize PPA without retraining the model.

ABSTRACT

As large language models (LLMs) like ChatGPT exhibited unprecedented machine intelligence, it also shows great performance in assisting hardware engineers to realize higher-efficiency logic design via natural language interaction. To estimate the potential of the hardware design process assisted by LLMs, this work attempts to demonstrate an automated design environment that explores LLMs to generate hardware logic designs from natural language specifications. To realize a more accessible and efficient chip development flow, we present a scalable four-stage zero-code logic design framework based on LLMs without retraining or finetuning. At first, the demo, ChipGPT, begins by generating prompts for the LLM, which then produces initial Verilog programs. Second, an output manager corrects and optimizes these programs before collecting them into the final design space. Eventually, ChipGPT will search through this space to select the optimal design under the target metrics. The evaluation sheds some light on whether LLMs can generate correct and complete hardware logic designs described by natural language for some specifications. It is shown that ChipGPT improves programmability, and controllability, and shows broader design optimization space compared to prior work and native LLMs alone.

Motivation & Objective

• Explore the feasibility of generating hardware logic designs from natural language specifications using LLMs without retraining.
• Propose a scalable four-stage zero-code framework integrating prompt management, output correction, and design space search.
• Evaluate whether natural language hardware design via LLMs improves programmability, controllability, and design space for PPA-aware chip design.

Proposed method

• Specification split to translate natural language specs into structured prompts.
• Template-based prompt manager to generate Verilog code with interface-aware prompts.
• Output manager to correct, refine, and filter generated Verilog using machine and human feedback.
• Enumerative search over a generated design space to select the best design per target metrics (PPA).
• Design Compiler-based evaluation of power, area, and latency for PPAs across workloads.

Experimental results

Research questions

• RQ1 RQ1 How do natural language-based methods compare to traditional agile hardware design methods in programmability and expressiveness?
• RQ2 RQ2 Does ChipGPT improve PPA and code quality relative to baseline ChatGPT and other agile methods (HLS, Chisel)?
• RQ3 RQ3 Are results sensitive to workload variety and design complexity?
• RQ4 RQ4 Do the prompt principles (composition, interface model, post-addition) improve soundness of generated code?
• RQ5 RQ5 Does human feedback materially impact automation of the design flow?

Key findings

• Natural language methods substantially reduce design description length compared with HLS and Chisel, indicating higher programmability.
• ChipGPT with the four-stage framework improves PPA and code quality over baseline ChatGPT across workloads, with up to 2.01x quality improvement in lines corrected.
• ChipGPT shows improved programmability and broader design exploration space relative to prior agile methods and native LLMs, though gains vary by workload.
• Prompt design principles (interface model, post-addition, composition) contribute to soundness and correctness of generated Verilog.
• Automated output management plus enumerative search is necessary, as raw LLM rankings do not consistently align with PPA-based optimal choices; human feedback further aids correction when machine feedback is insufficient.