[Paper Review] On Joint Learning for Solving Placement and Routing in Chip Design

Ruoyu Cheng, Junchi Yan|arXiv (Cornell University)|Oct 30, 2021

VLSI and FPGA Design Techniques34 references34 citations

TL;DR

The paper introduces DeepPlace for end-to-end macro/standard-cell placement and DeepPR for joint placement and routing using reinforcement learning with a multi-view embedding and random network distillation, showing improved wirelength and faster training on ISPD-2005 benchmarks.

ABSTRACT

For its advantage in GPU acceleration and less dependency on human experts, machine learning has been an emerging tool for solving the placement and routing problems, as two critical steps in modern chip design flow. Being still in its early stage, there are fundamental issues: scalability, reward design, and end-to-end learning paradigm etc. To achieve end-to-end placement learning, we first propose a joint learning method termed by DeepPlace for the placement of macros and standard cells, by the integration of reinforcement learning with a gradient based optimization scheme. To further bridge the placement with the subsequent routing task, we also develop a joint learning approach via reinforcement learning to fulfill both macro placement and routing, which is called DeepPR. One key design in our (reinforcement) learning paradigm involves a multi-view embedding model to encode both global graph level and local node level information of the input macros. Moreover, the random network distillation is devised to encourage exploration. Experiments on public chip design benchmarks show that our method can effectively learn from experience and also provides intermediate placement for the post standard cell placement, within few hours for training.

Motivation & Objective

• Motivate and address scalability, reward design, and end-to-end learning gaps in ML-based placement and routing.
• Propose an end-to-end DeepPlace method for placing macros and standard cells sequentially via RL and gradient-based optimization.
• Develop DeepPR to jointly learn placement and routing through reinforcement learning.
• Introduce a multi-view embedding (CNN+GNN) for state representation and random network distillation to enhance exploration.

Proposed method

• Model placement as an RL problem with a discrete action space over grid positions on an n x n chip canvas.
• Use a policy network that fuses global CNN-based layout embedding with local GNN-based netlist embeddings to predict macro placement actions.
• Combine RL for macro placement with a gradient-based placer (DREAMPlace) for standard-cell placement to obtain full placement solutions.
• Extend to joint placement and routing (DeepPR) by training an RL agent for placement while routing uses final wirelength feedback as reward.
• Incorporate random network distillation to provide intrinsic rewards for exploration during placement.
• Evaluate using PPO-based training, HPWL as an RPC proxy for wirelength, and RUDY-based congestion metrics; optionally route with a separate router or RL-based router.

Experimental results

Research questions

• RQ1 Can end-to-end learning improve macro/standard-cell placement compared to sequential or analytic placers?
• RQ2 Does joint placement and routing via reinforcement learning yield better final wirelength than a separate placement-and-routing pipeline?
• RQ3 Does a multi-view embedding (CNN + GNN) plus intrinsic exploration reward (RND) improve learning efficiency and solution quality in chip placement?
• RQ4 How does integrating a gradient-based standard-cell placer with RL for macros affect runtime and scalability?
• RQ5 What are the trade-offs and limitations when scaling to larger macro counts and more complex designs?

Key findings

• DeepPlace (macro+standard-cell placement) yields better wirelength than sequential placement approaches and shows consistent improvements across ISPD-2005 circuits.
• DeepPR (joint placement and routing) achieves significantly better routing-aware wirelength than the separate pipeline, indicating the benefit of end-to-end optimization.
• The multi-view policy network (CNN + GNN) with RND intrinsic rewards substantially outperforms baselines, especially in exploration and final reward.
• DeepPlace offers about 4x speedup over RePlace on eight ISPD-2005 circuits for macro placement runtimes.
• Pretraining on macro-only nets accelerates convergence and stabilizes learning when extending to full placement and routing tasks.