[Paper Review] GCN-RL Circuit Designer: Transferable Transistor Sizing with Graph Neural Networks and Reinforcement Learning

Hanrui Wang, Kuan Wang+5 more|arXiv (Cornell University)|Apr 30, 2020

VLSI and FPGA Design Techniques28 references36 citations

TL;DR

A reinforcement learning framework augmented with graph convolutional networks (GCN) learns transferable transistor sizing across technology nodes and circuit topologies, outperforming traditional optimization and enabling cross-node/topology knowledge transfer.

ABSTRACT

Automatic transistor sizing is a challenging problem in circuit design due to the large design space, complex performance trade-offs, and fast technological advancements. Although there has been plenty of work on transistor sizing targeting on one circuit, limited research has been done on transferring the knowledge from one circuit to another to reduce the re-design overhead. In this paper, we present GCN-RL Circuit Designer, leveraging reinforcement learning (RL) to transfer the knowledge between different technology nodes and topologies. Moreover, inspired by the simple fact that circuit is a graph, we learn on the circuit topology representation with graph convolutional neural networks (GCN). The GCN-RL agent extracts features of the topology graph whose vertices are transistors, edges are wires. Our learning-based optimization consistently achieves the highest Figures of Merit (FoM) on four different circuits compared with conventional black-box optimization methods (Bayesian Optimization, Evolutionary Algorithms), random search, and human expert designs. Experiments on transfer learning between five technology nodes and two circuit topologies demonstrate that RL with transfer learning can achieve much higher FoMs than methods without knowledge transfer. Our transferable optimization method makes transistor sizing and design porting more effective and efficient.

Motivation & Objective

• Motivate automatic transistor sizing to handle large design spaces and rapid technology scaling.
• Enable knowledge transfer of sizing policies across technology nodes and circuit topologies.
• Embed circuit topology information into the RL optimization loop to improve optimization efficiency.
• Demonstrate performance gains over Bayesian Optimization, Evolutionary Algorithms, random search, and human designs.

Proposed method

• Represent circuits as graphs with transistors as nodes and wires as edges.
• Use a graph convolutional network (GCN) to embed topology information into the RL agent."
• Apply an actor-critic RL framework (DDPG) to generate continuous transistor sizing actions (W, L, M) for MOS devices and component values (R, C).
• Denormalize/refine actions to satisfy device constraints and simulate the circuit to obtain a Figure of Merit (FoM).
• Train the policy with transfer learning to enable knowledge transfer across technology nodes and topologies.
• Compare GCN-RL to baselines (Random, ES, BO, MACE, NG-RL) and human designs across four real-world circuits.

Experimental results

Research questions

• RQ1 Can GCN-RL achieve higher FoMs than traditional black-box optimizers on fixed-topology circuits?
• RQ2 Does incorporating topology via GCN improve optimization performance and convergence speed?
• RQ3 Can the RL agent transfer learned sizing policies across different technology nodes?
• RQ4 Can the RL agent transfer knowledge between different circuit topologies that share design principles?
• RQ5 How does transfer learning impact FoM when porting designs to newer or older nodes and to different topologies?

Key findings

• GCN-RL consistently achieves the highest FoM across four real-world circuit baselines compared to human designs, random search, NG-RL, ES, BO, and MACE.
• GCN-RL accelerates convergence relative to NG-RL due to effective topology embedding provided by GCN.
• Knowledge transfer to other technology nodes significantly improves FoM with limited additional training.
• Knowledge transfer between topologies requiring graph-based representations (GCN-RL) outperforms transfers without GCN (NG-RL).
• Transfer experiments show viable improvement when porting 180 nm learned policies to 250 nm, 130 nm, 65 nm, and 45 nm nodes, as well as between Two-TIA and Three-TIA topologies.