[Paper Review] PP-LCNet: A Lightweight CPU Convolutional Neural Network

Cheng Cui, Tingquan Gao+11 more|arXiv (Cornell University)|Sep 17, 2021

Advanced Neural Network Applications31 references96 citations

TL;DR

PP-LCNet introduces a CPU-optimized lightweight CNN built with MKLDNN acceleration, achieving higher accuracy at near-constant latency across classification and downstream vision tasks. It outperforms many state-of-the-art lightweight models on Intel CPUs and shows strong results for detection and segmentation.

ABSTRACT

We propose a lightweight CPU network based on the MKLDNN acceleration strategy, named PP-LCNet, which improves the performance of lightweight models on multiple tasks. This paper lists technologies which can improve network accuracy while the latency is almost constant. With these improvements, the accuracy of PP-LCNet can greatly surpass the previous network structure with the same inference time for classification. As shown in Figure 1, it outperforms the most state-of-the-art models. And for downstream tasks of computer vision, it also performs very well, such as object detection, semantic segmentation, etc. All our experiments are implemented based on PaddlePaddle. Code and pretrained models are available at PaddleClas.

Motivation & Objective

• Address how to design lightweight CNNs that maximize accuracy without increasing CPU inference latency.
• Identify CPU-specific design choices and modules that boost performance on Intel MKLDNN-enabled CPUs.
• Propose a set of practical rules and components to guide CPU-focused CNN design and NAS for lightweight models.
• Demonstrate PP-LCNet’s performance across ImageNet classification and downstream tasks (detection, segmentation).
• Provide insights to NAS researchers on constraining search spaces for fast CPU deployment.

Proposed method

• Use DepthSepConv blocks inspired by MobileNetV1 as the base architecture to minimize overhead on Intel CPUs.
• Replace ReLU with H-Swish activation to improve accuracy with negligible latency impact.
• Place Squeeze-and-Excitation (SE) modules near the tail of the network to balance accuracy and speed.
• Adopt larger 5x5 kernels primarily in the tail of the network to boost performance without widespread latency costs.
• Append a 1280-dimensional 1x1 conv after global average pooling to enhance fitting capacity with little latency impact.
• Evaluate multiple model scales and use SSLD distillation to further boost accuracy when applicable.

Experimental results

Research questions

• RQ1 How can a lightweight CNN be optimized specifically for Intel CPUs with MKLDNN to maximize accuracy without increasing latency?
• RQ2 What architectural choices (activations, SE placement, kernel sizes, post-GAP layers) yield the best accuracy/latency trade-off on CPU?
• RQ3 Can a compact network design outperform existing lightweight models on classification and downstream tasks when constrained by CPU inference speed?
• RQ4 What practical rules emerge for designing CPU-friendly CNNs and guiding NAS for lightweight deployments?

Key findings

• PP-LCNet variants achieve competitive Top-1/Top-5 accuracy across multiple scales with low latency on Intel CPUs.
• Replacing ReLU with H-Swish substantially improves accuracy with minimal latency impact.
• Locating SE modules toward the tail of the network yields better accuracy-speed balance than distributing them densely.
• Using larger 5x5 kernels mainly in the tail provides accuracy gains without widespread latency penalties.
• Adding a 1280-dimension 1x1 conv after GAP significantly boosts accuracy with modest impact on latency.
• SSL Distillation (SSLD) further improves accuracy for larger PP-LCNet variants.