[Paper Review] Edge Impulse: An MLOps Platform for Tiny Machine Learning

Shawn Hymel, Colby Banbury+14 more|arXiv (Cornell University)|Nov 2, 2022

Advanced Neural Network Applications86 citations

TL;DR

Edge Impulse presents a cloud-based MLOps platform for TinyML, addressing data collection, DSP integration, cross-hardware optimization, AutoML, and deployment to diverse embedded targets with an extensible, scalable architecture.

ABSTRACT

Edge Impulse is a cloud-based machine learning operations (MLOps) platform for developing embedded and edge ML (TinyML) systems that can be deployed to a wide range of hardware targets. Current TinyML workflows are plagued by fragmented software stacks and heterogeneous deployment hardware, making ML model optimizations difficult and unportable. We present Edge Impulse, a practical MLOps platform for developing TinyML systems at scale. Edge Impulse addresses these challenges and streamlines the TinyML design cycle by supporting various software and hardware optimizations to create an extensible and portable software stack for a multitude of embedded systems. As of Oct. 2022, Edge Impulse hosts 118,185 projects from 50,953 developers.

Motivation & Objective

• Identify challenges in embedded TinyML workflows including data collection, DSP, development, deployment, and monitoring.
• Propose and describe an end-to-end platform (Edge Impulse) to streamline TinyML development and deployment.
• Demonstrate architecture supporting data-centric, extensible, and cross-stack optimization for heterogeneous embedded hardware.
• Evaluate the platform’s performance, portability, and scalability across multiple devices and models.

Proposed method

• Describe data collection and DSP preprocessing integrated into the platform.
• Present an extensible inferencing library and EON Compiler for reduced RAM/ROM usage.
• Outline AutoML (EON Tuner) to optimize DSP and NN configurations under hardware constraints.
• Explain deployment workflows including code generation, SDKs, and cross-platform compilation.
• Demonstrate performance evaluation across multiple devices and task benchmarks (KWS, VWW, IC).

Experimental results

Research questions

• RQ1 How can an MLOps platform address data collection, preprocessing, and deployment fragmentation in TinyML?
• RQ2 What mechanisms (AutoML, DSP tuning, cross-architecture optimizations) enable portable, efficient TinyML models across heterogeneous hardware?
• RQ3 What is the impact of DSP vs. NN co-design on latency and memory for embedded tasks?
• RQ4 How effective is the Edge Impulse approach in real-world industry, academia, and research deployments?
• RQ5 What are the resource and scalability profiles of Edge Impulse across multiple embedded platforms?

Key findings

• The platform supports end-to-end TinyML workflows with data ingestion, DSP, model training, evaluation, and deployment across diverse hardware.
• AutoML (EON Tuner) enables exploration of DSP and NN configurations given target hardware constraints, aiding rapid design space exploration.
• The EON Compiler reduces RAM/ROM usage by generating C++-level code that bypasses the TFLM interpreter, improving efficiency.
• Empirical evaluation across Nano 33 BLE Sense, ESP-EYE, and Raspberry Pi Pico shows varying latency and resource usage for KWS, VWW, and IC tasks under float32 and int8 regimes.
• DSP preprocessing often dominates total latency for certain tasks (e.g., keyword spotting), highlighting the importance of co-design between DSP and NN components.
• The platform emphasizes extensibility, Kubernetes-based scalable infrastructure, and REST APIs to integrate with existing ML pipelines.