[Paper Review] Accelerating Fully Connected Neural Network on Optical Network-on-Chip (ONoC)

Fei Dai, Yawen Chen+2 more|arXiv (Cornell University)|Sep 30, 2021

Photonic and Optical Devices40 references4 citations

TL;DR

This paper proposes a fine-grained parallel computing model for accelerating Fully Connected Neural Network (FCNN) training on Optical Network-on-Chip (ONoC) by optimizing core allocation across forward and back propagation stages. It derives the optimal number of cores per stage to minimize training time and evaluates three mapping strategies, showing up to 22.28% reduction in training time and 47.85% energy savings compared to ENoC under batch size 64.

ABSTRACT

Fully Connected Neural Network (FCNN) is a class of Artificial Neural Networks widely used in computer science and engineering, whereas the training process can take a long time with large datasets in existing many-core systems. Optical Network-on-Chip (ONoC), an emerging chip-scale optical interconnection technology, has great potential to accelerate the training of FCNN with low transmission delay, low power consumption, and high throughput. However, existing methods based on Electrical Network-on-Chip (ENoC) cannot fit in ONoC because of the unique properties of ONoC. In this paper, we propose a fine-grained parallel computing model for accelerating FCNN training on ONoC and derive the optimal number of cores for each execution stage with the objective of minimizing the total amount of time to complete one epoch of FCNN training. To allocate the optimal number of cores for each execution stage, we present three mapping strategies and compare their advantages and disadvantages in terms of hotspot level, memory requirement, and state transitions. Simulation results show that the average prediction error for the optimal number of cores in NN benchmarks is within 2.3%. We further carry out extensive simulations which demonstrate that FCNN training time can be reduced by 22.28% and 4.91% on average using our proposed scheme, compared with traditional parallel computing methods that either allocate a fixed number of cores or allocate as many cores as possible, respectively. Compared with ENoC, simulation results show that under batch sizes of 64 and 128, on average ONoC can achieve 21.02% and 12.95% on reducing training time with 47.85% and 39.27% on saving energy, respectively.

Motivation & Objective

• Address the communication bottleneck in FCNN training on many-core systems using ONoC's low-latency, high-throughput optical interconnects.
• Develop a fine-grained parallel computing model that balances computation and communication for FCNN training on ONoC.
• Derive the optimal number of cores for each execution stage (forward and back propagation) to minimize total training time.
• Propose and evaluate three core mapping strategies (FM, RRM, ORRM) considering hotspot level, memory requirements, and state transitions.
• Demonstrate significant performance and energy efficiency gains over traditional ENoC-based parallel computing methods.

Proposed method

• Propose a fine-grained parallel computing model that characterizes computation and communication costs in FCNN training on ONoC, enabling trade-off analysis between computation and communication.
• Derive analytical expressions for the optimal number of cores in forward and back propagation stages to minimize total training time, based on latency and bandwidth models.
• Design three core mapping strategies: Fixed Mapping (FM), Row-Reuse Mapping (RRM), and Optimized Row-Reuse Mapping (ORRM), each with distinct trade-offs in hotspot, memory usage, and state transitions.
• Use wavelength division multiplexing (WDM) to enable simultaneous multi-wavelength optical communication in waveguides, leveraging ONoC’s high bandwidth and low delay.
• Model memory requirements per core based on distributed storage of neural network parameters (weights and partial sums) in SRAM.
• Conduct extensive simulations using NN benchmarks to validate core allocation predictions and compare performance against fixed-core and fully parallelized ENoC-based methods.

Experimental results

Research questions

• RQ1 What is the optimal number of cores required for each execution stage (forward and back propagation) in FCNN training on ONoC to minimize total training time?
• RQ2 How do different core mapping strategies (FM, RRM, ORRM) compare in terms of hotspot level, memory requirement, and state transitions?
• RQ3 Can the proposed model accurately predict the optimal core allocation with low error across diverse neural network benchmarks?
• RQ4 To what extent can ONoC reduce FCNN training time and energy consumption compared to ENoC under varying batch sizes and core counts?

Key findings

• The average prediction error for the optimal number of cores across NN benchmarks is within 2.3%, validating the accuracy of the proposed analytical model.
• The proposed method reduces FCNN training time by 22.28% on average compared to traditional methods that allocate a fixed number of cores.
• The method reduces training time by 4.91% on average compared to methods that allocate as many cores as possible.
• Compared to ENoC, ONoC achieves 21.02% and 12.95% reductions in training time under batch sizes 64 and 128, respectively.
• Energy consumption is reduced by 47.85% and 39.27% on average with ONoC compared to ENoC under batch sizes 64 and 128, respectively.
• The ORRM mapping strategy offers the best balance among hotspot level, memory usage, and state transitions, outperforming FM and RRM in overall efficiency.