[Paper Review] SoC-Tuner: An Importance-guided Exploration Framework for DNN-targeting SoC Design

Shixin Chen, Su Zheng+5 more|arXiv (Cornell University)|Dec 19, 2023

Advanced Memory and Neural Computing4 citations

TL;DR

SoC-Tuner is a novel, importance-guided exploration framework for designing DNN-optimized system-on-chip (SoC) architectures. It uses importance-based pruning, representative sampling, and information-guided multi-objective optimization with full VLSI flow evaluation to efficiently identify Pareto-optimal SoC configurations, outperforming prior methods in accuracy and convergence speed on DNN benchmarks.

ABSTRACT

Designing a system-on-chip (SoC) for deep neural network (DNN) acceleration requires balancing multiple metrics such as latency, power, and area. However, most existing methods ignore the interactions among different SoC components and rely on inaccurate and error-prone evaluation tools, leading to inferior SoC design. In this paper, we present SoC-Tuner, a DNN-targeting exploration framework to find the Pareto optimal set of SoC configurations efficiently. Our framework constructs a thorough SoC design space of all components and divides the exploration into three phases. We propose an importance-based analysis to prune the design space, a sampling algorithm to select the most representative initialization points, and an information-guided multi-objective optimization method to balance multiple design metrics of SoC design. We validate our framework with the actual very-large-scale-integration (VLSI) flow on various DNN benchmarks and show that it outperforms previous methods. To the best of our knowledge, this is the first work to construct an exploration framework of SoCs for DNN acceleration.

Motivation & Objective

• To address the challenge of inaccurate and incomplete design space exploration in DNN-targeting SoC design due to reliance on simplified analytical models.
• To reduce the time and expertise required for SoC exploration by automating the identification of high-quality configurations.
• To construct a comprehensive design space that includes interactions among all key SoC components, including CPU, accelerator, memory, and interconnects.
• To achieve accurate, hardware-accurate evaluation of latency, power, and area using full VLSI flow instead of approximated models.
• To enable automated, bias-free exploration of the Pareto-optimal trade-off space across multiple design metrics.

Proposed method

• Constructs a comprehensive design space by parameterizing all major SoC components, including CPU, accelerator (e.g., systolic array), memory hierarchy, and interconnects.
• Applies an importance-based analysis using the ICD algorithm to identify and prune less influential design parameters, reducing the design space by 30.16%.
• Employs a sampling algorithm to select representative initialization points for optimization, ensuring diverse and informative starting configurations.
• Utilizes an information-guided multi-objective optimization strategy that balances latency, power, and area using expected hypervolume improvement.
• Employs full VLSI flow for accurate evaluation of design points, replacing inaccurate analytical models.
• Integrates with Chipyard and Chisel for rapid, parameterized hardware generation and RTL simulation.

Experimental results

Research questions

• RQ1 How can we effectively reduce the design space of DNN-targeting SoCs while preserving the ability to find Pareto-optimal configurations?
• RQ2 To what extent does using full VLSI flow evaluation improve the accuracy of design space exploration compared to analytical models?
• RQ3 Can importance-based pruning and representative sampling significantly accelerate convergence to the Pareto-optimal set in multi-objective SoC exploration?
• RQ4 How does SoC-Tuner’s optimization strategy compare to state-of-the-art methods in terms of convergence speed and proximity to the true Pareto front?
• RQ5 Can the framework be generalized across diverse DNN workloads and maintain high efficiency and accuracy?

Key findings

• SoC-Tuner reduces the design space by 30.16% through importance-based pruning using the ICD algorithm, significantly improving exploration efficiency.
• The framework achieves a lower average distance to the reference Pareto set (ADRS) than baseline methods, demonstrating superior convergence to the true Pareto front.
• In latency-area and latency-power spaces, SoC-Tuner's learned Pareto set is much closer to the real Pareto optimal set than other methods, as shown in Fig. 4.
• The simplified analytical model used in prior work produces a Pareto set that diverges significantly from the actual metrics obtained via VLSI flow, proving its inaccuracy.
• SoC-Tuner achieves the lowest inference cycles across multiple DNNs, including ResNet50 and Transformer-based models, outperforming other exploration methods.
• The optimal SoC design generated by SoC-Tuner achieves the highest performance with minimal area overhead, as confirmed by VLSI area breakdown in Fig. 7(b).