[Paper Review] swTVM: Exploring the Automated Compilation for Deep Learning on Sunway Architecture

Changxi Liu, Hailong Yang+5 more|arXiv (Cornell University)|Apr 16, 2019

Advanced Neural Network Applications27 references4 citations

TL;DR

swTVM is a compiler extension to TVM that enables ahead-of-time compilation for Sunway's many-core architecture, leveraging core groups, DMA, and local memory for high-performance deep learning inference. It achieves 6.71x and 2.45x speedups over hand-optimized OpenACC code for AlexNet and VGG-19 respectively, marking the first automated deep learning compiler solution for Sunway with strong productivity and efficiency.

ABSTRACT

The flourish of deep learning frameworks and hardware platforms has been demanding an efficient compiler that can shield the diversity in both software and hardware in order to provide application portability. Among the exiting deep learning compilers, TVM is well known for its efficiency in code generation and optimization across diverse hardware devices. In the meanwhile, the Sunway many-core processor renders itself as a competitive candidate for its attractive computational power in both scientific and deep learning applications. This paper combines the trends in these two directions. Specifically, we propose swTVM that extends the original TVM to support ahead-of-time compilation for architecture requiring cross-compilation such as Sunway. In addition, we leverage the architecture features during the compilation such as core group for massive parallelism, DMA for high bandwidth memory transfer and local device memory for data locality, in order to generate efficient code for deep learning application on Sunway. The experimental results show the ability of swTVM to automatically generate code for various deep neural network models on Sunway. The performance of automatically generated code for AlexNet and VGG-19 by swTVM achieves 6.71x and 2.45x speedup on average than hand-optimized OpenACC implementations on convolution and fully connected layers respectively. This work is the first attempt from the compiler perspective to bridge the gap of deep learning and high performance architecture particularly with productivity and efficiency in mind. We would like to open source the implementation so that more people can embrace the power of deep learning compiler and Sunway many-core processor.

Motivation & Objective

• To address the growing need for portable, efficient deep learning compilation across diverse hardware and software stacks.
• To bridge the gap between deep learning frameworks and high-performance architectures like Sunway's many-core processor.
• To enable automated, ahead-of-time compilation for Sunway, which requires cross-compilation and specialized optimization.
• To improve productivity and performance in deep learning workloads on Sunway by leveraging architectural features such as core groups, DMA, and local memory.
• To provide a reusable, open-source compiler solution that supports efficient deployment of DNN models on Sunway systems.

Proposed method

• Extends TVM with support for ahead-of-time compilation targeting Sunway's many-core architecture.
• Introduces cross-compilation capabilities tailored for Sunway's unique compilation workflow.
• Leverages core group abstraction to exploit massive parallelism across Sunway's processing units.
• Utilizes DMA mechanisms to optimize high-bandwidth memory transfers between global and local memory.
• Employs local device memory for data locality optimization, reducing latency and improving bandwidth utilization.
• Integrates architecture-specific optimizations into TVM's code generation pipeline to produce efficient low-level code.

Experimental results

Research questions

• RQ1 How can a deep learning compiler be extended to support Sunway's many-core architecture with its unique compilation and execution model?
• RQ2 What optimizations are most effective in exploiting Sunway’s core groups, DMA, and local memory for deep neural network kernels?
• RQ3 Can automated code generation via an extended TVM achieve performance comparable or superior to hand-optimized implementations on Sunway?
• RQ4 How does the performance of automatically generated code compare to hand-tuned OpenACC implementations on standard DNN models?
• RQ5 To what extent can a unified compiler stack like TVM enable portable and efficient deep learning deployment on emerging high-performance architectures like Sunway?

Key findings

• swTVM successfully generates optimized code for various deep neural network models on the Sunway architecture.
• The automatically generated code for AlexNet achieves an average 6.71x speedup over hand-optimized OpenACC implementations on convolutional layers.
• For VGG-19, the automatically generated code achieves an average 2.45x speedup over OpenACC on fully connected layers.
• The performance gains demonstrate that automated compilation can effectively exploit Sunway’s architectural features such as core groups and DMA.
• The results validate that swTVM provides a viable, high-productivity path to high-performance deep learning on Sunway, outperforming hand-optimized code in key layers.
• The open-sourcing of swTVM enables broader community adoption and further exploration of deep learning on Sunway’s many-core platform.