[Paper Review] On-Chip Optical Convolutional Neural Networks

Hengameh Bagherian, Scott A. Skirlo+4 more|arXiv (Cornell University)|Aug 9, 2018

Neural Networks and Reservoir Computing1 references57 citations

TL;DR

The paper proposes a photonics-based convolutional neural network architecture implemented on a chip, using MZI networks, optical nonlinearities, and delay lines to perform fast, low-power CNN inference with time-multiplexed optical computations. It also discusses a hybrid optical-electronic approach and practical challenges.

ABSTRACT

Convolutional Neural Networks (CNNs) are a class of Artificial Neural Networks(ANNs) that employ the method of convolving input images with filter-kernels for object recognition and classification purposes. In this paper, we propose a photonics circuit architecture which could consume a fraction of energy per inference compared with state of the art electronics.

Motivation & Objective

• Motivate the need for energy-efficient, high-throughput CNN inference beyond GPUs and traditional electronics.
• Introduce an integrated photonics architecture capable of implementing convolutional kernels, pooling, and nonlinearities on chip.
• Detail a time-multiplexed optical matrix-multiplication approach to realize kernel-dot-product computations across CNN layers.
• Discuss practical implementation options, including an optical-only and an optical-electronic hybrid setup, and outline engineering challenges.

Proposed method

• Use a network of Mach-Zehnder interferometers (MZIs) to implement kernel matrices via Reck encoding of unitary components from SVD decomposition M_i = U Σ V.
• Encode kernel/pixel patches as coherent optical pulses and perform patch-by-patch dot-products with kernel matrices to obtain time-series outputs.
• Apply optical nonlinearity (e.g., graphene saturable absorbers) to outputs, enabling CNN-like activation functions.
• Use optical delay lines to repatch kernel dot-products into new input patches for the next layer, enabling multi-layer CNN operation on a photonic platform.
• Treat pooling as strided convolution, and implement repatching to form next-layer patches through a delay-line/splitter network.
• Optionally discuss an optical-electronic hybrid configuration to mitigate implementation challenges and leverage analog CMOS nonlinearities and memory elements.

Experimental results

Research questions

• RQ1 Can a photonic integrated circuit perform CNN convolution, pooling, and nonlinearities at comparable accuracy to electronic counterparts?
• RQ2 What energy and speed advantages can optical convolution deliver for typical CNNs (e.g., AlexNet scale) and what are the limiting factors?
• RQ3 How do errors in MZI phase encoding and nonlinearity affect inference accuracy in an optical CNN?
• RQ4 What are viable architectures (fully optical vs. hybrid) to maximize throughput and minimize power while remaining on-chip?

Key findings

• An optical convolutional architecture could achieve on the order of a million inferences per second with about 2 mJ per inference.
• The proposed system is claimed to be ~30x faster than GPU-enabled AlexNet inference while using similar total power
• A fully on-chip photonic implementation relies on MZI networks for matrix multiplication, optical nonlinearities, and precise time-multiplexed repatching via delay lines.
• A hybrid optical-electronic setup could reduce chip area and complexity by implementing difficult optical components with CMOS analog circuits and memories.
• The energy and performance claims depend on practical device bandwidth and detector speeds, with a detailed energy analysis provided in the appendices.