[Paper Review] Evolution of Convolutional Neural Network (CNN): Compute vs Memory bandwidth for Edge AI

Dwith Chenna|arXiv (Cornell University)|Sep 24, 2023

Advanced Memory and Neural Computing4 citations

TL;DR

This paper analyzes the evolving trade-off between compute and memory bandwidth in Convolutional Neural Networks (CNNs) for Edge AI, tracing architectural progression from early models to state-of-the-art designs. It identifies that increasing model complexity escalates both computational demands and memory access, offering insights for designing efficient CNN architectures and hardware accelerators optimized for resource-constrained edge devices.

ABSTRACT

Convolutional Neural Networks (CNNs) have greatly influenced the field of Embedded Vision and Edge Artificial Intelligence (AI), enabling powerful machine learning capabilities on resource-constrained devices. This article explores the relationship between CNN compute requirements and memory bandwidth in the context of Edge AI. We delve into the historical progression of CNN architectures, from the early pioneering models to the current state-of-the-art designs, highlighting the advancements in compute-intensive operations. We examine the impact of increasing model complexity on both computational requirements and memory access patterns. The paper presents a comparison analysis of the evolving trade-off between compute demands and memory bandwidth requirements in CNNs. This analysis provides insights into designing efficient architectures and potential hardware accelerators in enhancing CNN performance on edge devices.

Motivation & Objective

• To understand the historical evolution of CNN architectures in relation to compute and memory bandwidth demands.
• To investigate how increasing model complexity affects computational requirements and memory access patterns in Edge AI systems.
• To identify the performance bottlenecks arising from the imbalance between compute and memory bandwidth in edge-deployed CNNs.
• To provide design insights for efficient CNN architectures and hardware accelerators tailored for edge devices with limited resources.
• To guide future development of energy-efficient, high-performance inference systems in embedded vision applications.

Proposed method

• Traces the progression of CNN architectures from early models (e.g., AlexNet, VGG) to modern state-of-the-art designs (e.g., EfficientNet, MobileNet variants).
• Analyzes compute requirements using FLOPs (floating-point operations) and memory bandwidth using activation and weight tensor access patterns.
• Compares the growth trends of compute and memory bandwidth across different CNN families over time.
• Evaluates the impact of architectural innovations such as depthwise separable convolutions and neural architecture search on the compute-memory trade-off.
• Uses empirical data from published models to quantify the evolution of compute and memory bandwidth demands.
• Proposes a framework for assessing the efficiency of CNNs based on the ratio of compute to memory bandwidth, enabling performance prediction on edge hardware.

Experimental results

Research questions

• RQ1 How have the compute and memory bandwidth requirements of CNNs evolved from early to modern architectures in the context of Edge AI?
• RQ2 What is the relative growth rate of compute demands versus memory bandwidth demands across different CNN families?
• RQ3 To what extent do architectural innovations such as depthwise separable convolutions reduce the memory bandwidth burden without increasing compute?
• RQ4 What are the dominant bottlenecks in current CNN inference pipelines on edge devices—compute or memory bandwidth?
• RQ5 How can hardware accelerators be co-designed with CNN architectures to balance compute and memory bandwidth constraints?

Key findings

• The compute requirements of CNNs have grown significantly over time, with FLOPs increasing by orders of magnitude from AlexNet to modern models.
• Memory bandwidth demands have also increased substantially, often outpacing improvements in on-chip memory bandwidth, making memory access a key bottleneck.
• Architectural innovations like depthwise separable convolutions have reduced FLOPs by up to 8x compared to standard convolutions, but memory bandwidth savings are less pronounced.
• The ratio of compute to memory bandwidth has not improved proportionally, indicating that memory access is increasingly the limiting factor in edge inference.
• Modern CNNs such as EfficientNet and MobileNet achieve high accuracy with lower FLOPs, but their memory bandwidth requirements remain high due to large activation tensors.
• The paper concludes that future hardware accelerators must prioritize memory bandwidth efficiency to unlock the full potential of advanced CNNs on edge devices.