[Paper Review] CoCoPIE: Making Mobile AI Sweet As PIE --Compression-Compilation Co-Design Goes a Long Way

Shaoshan Liu, Bin Ren+2 more|arXiv (Cornell University)|Mar 14, 2020

Advanced Neural Network Applications55 references13 citations

TL;DR

CoCoPIE introduces a compression-compiler co-design framework that enables real-time mobile AI on commodity devices without specialized hardware, achieving up to 180× faster DNN pruning and outperforming ASIC/FPGA accelerators in speed and energy efficiency by jointly optimizing model compression and compilation for mobile CPUs and GPUs.

ABSTRACT

Assuming hardware is the major constraint for enabling real-time mobile intelligence, the industry has mainly dedicated their efforts to developing specialized hardware accelerators for machine learning and inference. This article challenges the assumption. By drawing on a recent real-time AI optimization framework CoCoPIE, it maintains that with effective compression-compiler co-design, it is possible to enable real-time artificial intelligence on mainstream end devices without special hardware. CoCoPIE is a software framework that holds numerous records on mobile AI: the first framework that supports all main kinds of DNNs, from CNNs to RNNs, transformer, language models, and so on; the fastest DNN pruning and acceleration framework, up to 180X faster compared with current DNN pruning on other frameworks such as TensorFlow-Lite; making many representative AI applications able to run in real-time on off-the-shelf mobile devices that have been previously regarded possible only with special hardware support; making off-the-shelf mobile devices outperform a number of representative ASIC and FPGA solutions in terms of energy efficiency and/or performance.

Motivation & Objective

• To challenge the industry-wide assumption that specialized hardware is essential for real-time mobile AI.
• To demonstrate that effective compression-compiler co-design can achieve real-time inference on mainstream mobile processors without custom accelerators.
• To reduce time-to-market, cost, and technical barriers associated with custom AI hardware by leveraging general-purpose processors.
• To enable broader adoption of real-time AI on existing mobile and IoT devices through software-only optimization.
• To extend the co-design principle beyond DNNs to holistic AI application optimization and privacy-aware deployment.

Proposed method

• Introduces a dual-component framework: CoCo-Gen for automated, high-performance DNN compression and CoCo-Tune for intelligent compilation and optimization.
• Employs a synergistic design where model compression (pruning and quantization) is guided by low-level hardware execution patterns to improve compilation efficiency.
• Leverages insights from compiler design—such as modularity and composability—to guide compression decisions for better code generation and performance.
• Uses a demand-aware co-design approach that aligns compression choices with processor preferences (e.g., memory access patterns, instruction-level parallelism).
• Applies end-to-end optimization from high-level DNN models (CNNs, RNNs, Transformers) down to executable code for mobile CPUs and GPUs.
• Supports data-free pruning via synthetic data generation to enable optimization in privacy-constrained environments.

Experimental results

Research questions

• RQ1 Can real-time AI inference be achieved on mainstream mobile devices without specialized AI accelerators through software co-design?
• RQ2 How does compression-compiler co-design reduce tuning time and improve performance compared to traditional independent compression and compilation pipelines?
• RQ3 To what extent can general-purpose processors outperform custom ASICs and FPGAs in terms of speed and energy efficiency for mobile AI workloads?
• RQ4 How can co-design principles be extended beyond DNN models to optimize entire AI applications and heterogeneous deployment systems?
• RQ5 Can privacy-preserving DNN optimization be achieved without access to original training data, using synthetic data for pruning?

Key findings

• CoCoPIE achieves up to 180× faster DNN pruning compared to TensorFlow Lite and other leading frameworks, drastically reducing optimization time.
• CoCoPIE enables real-time inference for diverse DNNs—including CNNs, RNNs, Transformers, and large language models—on off-the-shelf mobile devices previously requiring specialized hardware.
• On-device performance with CoCoPIE exceeds that of multiple ASIC and FPGA solutions in both speed and energy efficiency, particularly on 28–65nm AI accelerators.
• The framework supports all major DNN architectures and operations, making it the first to unify support across CNNs, RNNs, and transformer-based models.
• CoCoPIE’s data-free pruning capability allows model optimization without access to original training data, expanding applicability in privacy-sensitive scenarios.
• The framework enables previously infeasible real-time AI use cases, such as real-time video style transfer on mobile cameras and real-time HD video upscaling in low-bandwidth conditions.