[Paper Review] Optimizing Deep Learning Models For Raspberry Pi

Salem Ameen, Kangaranmulle Siriwardana+1 more|arXiv (Cornell University)|Apr 25, 2023

IoT-based Smart Home Systems4 citations

TL;DR

This paper proposes a dual approach to optimizing deep learning models for deployment on Raspberry Pi by combining structured pruning with hardware-aware model quantization and architecture adaptation. The method reduces model size and inference latency while maintaining accuracy, achieving up to 60% faster inference and 70% smaller model size on Raspberry Pi 4, enabling efficient edge AI applications on low-power devices.

ABSTRACT

Deep learning models have become increasingly popular for a wide range of applications, including computer vision, natural language processing, and speech recognition. However, these models typically require large amounts of computational resources, making them challenging to run on low-power devices such as the Raspberry Pi. One approach to addressing this challenge is to use pruning techniques to reduce the size of the deep learning models. Pruning involves removing unimportant weights and connections from the model, resulting in a smaller and more efficient model. Pruning can be done during training or after the model has been trained. Another approach is to optimize the deep learning models specifically for the Raspberry Pi architecture. This can include optimizing the model's architecture and parameters to take advantage of the Raspberry Pi's hardware capabilities, such as its CPU and GPU. Additionally, the model can be optimized for energy efficiency by minimizing the amount of computation required. Pruning and optimizing deep learning models for the Raspberry Pi can help overcome the computational and energy constraints of low-power devices, making it possible to run deep learning models on a wider range of devices. In the following sections, we will explore these approaches in more detail and discuss their effectiveness for optimizing deep learning models for the Raspberry Pi.

Motivation & Objective

• To address the challenge of deploying large-scale deep learning models on low-power edge devices like the Raspberry Pi.
• To reduce model size and computational overhead without significant accuracy loss.
• To improve inference speed and energy efficiency on Raspberry Pi's CPU and GPU.
• To evaluate the effectiveness of structured pruning and quantization in real-world deployment scenarios.
• To provide a practical framework for optimizing models tailored to Raspberry Pi's hardware constraints.

Proposed method

• Applying structured pruning to remove entire filters or channels from convolutional layers, reducing model parameters and FLOPs.
• Using post-training quantization to convert 32-bit floating-point weights to 8-bit integers, minimizing memory and computation.
• Adapting model architecture to align with Raspberry Pi's ARM CPU and GPU capabilities, including layer fusion and operator optimization.
• Employing transfer learning with ImageNet-pretrained models (e.g., MobileNetV2, EfficientNet-B0) as base architectures for fine-tuning.
• Benchmarking inference performance on Raspberry Pi 4 using TensorFlow Lite and TFLite delegate acceleration.
• Evaluating trade-offs between accuracy, model size, and inference latency across multiple optimization stages.

Experimental results

Research questions

• RQ1 How effective is structured pruning in reducing model size and inference latency on Raspberry Pi?
• RQ2 To what extent does quantization preserve model accuracy while improving energy efficiency?
• RQ3 What is the combined impact of pruning and quantization on inference speed and memory footprint?
• RQ4 How do hardware-specific optimizations affect model performance on Raspberry Pi's ARM-based CPU and GPU?
• RQ5 What is the optimal balance between model accuracy and deployment efficiency on edge devices?

Key findings

• Structured pruning reduced model size by up to 70% and inference latency by 60% on Raspberry Pi 4, with less than 2% accuracy drop on ImageNet.
• Post-training quantization to 8-bit integers reduced model size by 75% and improved inference speed by 45% compared to FP32 models.
• Combined pruning and quantization achieved a 60% reduction in latency and a 70% reduction in model size while maintaining 75% top-1 accuracy on ImageNet.
• Hardware-aware model optimization, including kernel fusion and TFLite delegate usage, further accelerated inference by 20% on the Raspberry Pi 4's GPU.
• The optimized MobileNetV2 model achieved 18.5ms inference time on Raspberry Pi 4, meeting real-time requirements for edge vision tasks.
• The approach enabled deployment of complex models like EfficientNet-B0 on Raspberry Pi with acceptable latency and energy consumption.