[論文レビュー] ChipGPT: How far are we from natural language hardware design

Kaiyan Chang, Ying Wang+6 more|arXiv (Cornell University)|May 23, 2023

Ferroelectric and Negative Capacitance Devices被引用数 27

ひとこと要約

ChipGPT は、自然言語仕様から Verilog を生成する四段階のゼロコードフレームワークを示し、出力マネージャと列挙検索を用いてモデルを再学習させずに PPA を最適化します。

ABSTRACT

As large language models (LLMs) like ChatGPT exhibited unprecedented machine intelligence, it also shows great performance in assisting hardware engineers to realize higher-efficiency logic design via natural language interaction. To estimate the potential of the hardware design process assisted by LLMs, this work attempts to demonstrate an automated design environment that explores LLMs to generate hardware logic designs from natural language specifications. To realize a more accessible and efficient chip development flow, we present a scalable four-stage zero-code logic design framework based on LLMs without retraining or finetuning. At first, the demo, ChipGPT, begins by generating prompts for the LLM, which then produces initial Verilog programs. Second, an output manager corrects and optimizes these programs before collecting them into the final design space. Eventually, ChipGPT will search through this space to select the optimal design under the target metrics. The evaluation sheds some light on whether LLMs can generate correct and complete hardware logic designs described by natural language for some specifications. It is shown that ChipGPT improves programmability, and controllability, and shows broader design optimization space compared to prior work and native LLMs alone.

研究の動機と目的

• LLMを用いて retraining せずに自然言語仕様からハードウェアの論理設計を生成する可能性を調査する。
• プロンプト管理、出力補正、設計空間探索を統合した、スケーラブルな四段階のゼロコードフレームワークを提案する。
• LLMによる自然言語ハードウェア設計がプログラミム性、制御性、およびPPAを重視したチップ設計の設計空間を改善するかを評価する。

提案手法

• 自然言語仕様を構造化されたプロンプトへ翻訳するための仕様分割。
• インタフェース対応プロンプトを用いてVerilogコードを生成するテンプレートベースのプロンプトマネージャ。
• 機械的および人的フィードバックを用いて生成されたVerilogを修正・洗練・フィルタリングする出力マネージャ。
• 生成された設計空間上の列挙検索を行い、ターゲット指標（PPA）に基づいて最適な設計を選択。
• ワークロード全体にわたるPPAsの電力、面積、遅延をDesign Compilerベースで評価。

実験結果

リサーチクエスチョン

• RQ1 RQ1 How do natural language-based methods compare to traditional agile hardware design methods in programmability and expressiveness?
• RQ2 RQ2 Does ChipGPT improve PPA and code quality relative to baseline ChatGPT and other agile methods (HLS, Chisel)?
• RQ3 RQ3 Are results sensitive to workload variety and design complexity?
• RQ4 RQ4 Do the prompt principles (composition, interface model, post-addition) improve soundness of generated code?
• RQ5 RQ5 Does human feedback materially impact automation of the design flow?

主な発見

Workload	Configuration	Power	Area	Latency
matrix mul	ChatGPT(Baseline)	105.24	179680.0	1
matrix mul	HLS	0.1946	2592.79	169
matrix mul	Chisel	28.5361	55983.6	1
matrix mul	Ours(ChipGPT)	13.14	952.4	1
mux4x1	ChatGPT(Baseline)	2.62E-03	11.2	1
mux4x1	HLS	2.62E-03	11.2	1
mux4x1	Chisel	2.62E-03	11.2	1
mux4x1	Ours(ChipGPT)	2.62E-03	11.2	1
3-8decoder	ChatGPT(Baseline)	2.60E-03	22.8	1
3-8decoder	HLS	7.03E-03	156.4	9
3-8decoder	Chisel	3.27E-03	24.4	1
3-8decoder	Ours(ChipGPT)	2.60E-03	22.8	1
button-count	ChatGPT(Baseline)	0.01	265.2	1
button-count	HLS	0.0078	200.4	9
button-count	Chisel	0.00834	146.8	1
button-count	Ours(ChipGPT)	0.0429	139.2	1
vector-matrix	ChatGPT(Baseline)	1.30	3451.2	2
vector-matrix	HLS	0.03	428.8	191
vector-matrix	Chisel	1.27	3400.0	1
vector-matrix	Ours(ChipGPT)	1.15	3144.0	1
adder-multi tree	ChatGPT(Baseline)	28.50	60070.4	1
adder-multi tree	HLS	0.09	1784.4	73
adder-multi tree	Chisel	28.54	55983.6	1
adder-multi tree	Ours(ChipGPT)	27.79	50498.8	1
accumulator	ChatGPT(Baseline)	0.02	174.0	1
accumulator	HLS	0.0110	204.0	17
accumulator	Chisel	0.0257	136.4	1
accumulator	Ours(ChipGPT)	0.03	136.0	1
Simple CPU	ChatGPT(Baseline)	2.57	23138.4	5
Simple CPU	HLS	0.10	2780.0	38
Simple CPU	Chisel	1.48	25346.0	3
Simple CPU	Ours(ChipGPT)	0.48	3240.8	5

• Natural language methods substantially reduce design description length compared with HLS and Chisel, indicating higher programmability.
• ChipGPT with the four-stage framework improves PPA and code quality over baseline ChatGPT across workloads, with up to 2.01x quality improvement in lines corrected.
• ChipGPT shows improved programmability and broader design exploration space relative to prior agile methods and native LLMs, though gains vary by workload.
• Prompt design principles (interface model, post-addition, composition) contribute to soundness and correctness of generated Verilog.
• Automated output management plus enumerative search is necessary, as raw LLM rankings do not consistently align with PPA-based optimal choices; human feedback further aids correction when machine feedback is insufficient.