[Paper Review] A Machine Learning-oriented Survey on Tiny Machine Learning

Luigi Capogrosso, Federico Cunico+3 more|arXiv (Cornell University)|Sep 21, 2023

Advanced Neural Network ApplicationsComputer Science3 citations

TL;DR

This survey provides a comprehensive, ML-focused analysis of Tiny Machine Learning (TinyML) from 2018 to 2023, systematically reviewing three implementation workflows—ML-oriented, HW-oriented, and co-design—while introducing a taxonomy of model optimization techniques and learning algorithms. It identifies co-design as the most promising path forward, highlighting benchmarking, memory constraints, data quality, and lack of standardized models and public datasets as key unresolved challenges.

ABSTRACT

The emergence of Tiny Machine Learning (TinyML) has positively revolutionized the field of Artificial Intelligence by promoting the joint design of resource-constrained IoT hardware devices and their learning-based software architectures. TinyML carries an essential role within the fourth and fifth industrial revolutions in helping societies, economies, and individuals employ effective AI-infused computing technologies (e.g., smart cities, automotive, and medical robotics). Given its multidisciplinary nature, the field of TinyML has been approached from many different angles: this comprehensive survey wishes to provide an up-to-date overview focused on all the learning algorithms within TinyML-based solutions. The survey is based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodological flow, allowing for a systematic and complete literature survey. In particular, firstly we will examine the three different workflows for implementing a TinyML-based system, i.e., ML-oriented, HW-oriented, and co-design. Secondly, we propose a taxonomy that covers the learning panorama under the TinyML lens, examining in detail the different families of model optimization and design, as well as the state-of-the-art learning techniques. Thirdly, this survey will present the distinct features of hardware devices and software tools that represent the current state-of-the-art for TinyML intelligent edge applications. Finally, we discuss the challenges and future directions.

Motivation & Objective

• To provide a systematic, ML-focused overview of TinyML research from 2018 to 2023, addressing the field’s multidisciplinary complexity.
• To formalize and compare the three main implementation workflows: ML-oriented, HW-oriented, and co-design.
• To present a comprehensive taxonomy of model optimization and learning techniques specific to TinyML.
• To evaluate current hardware platforms and software tools for TinyML deployment.
• To identify unresolved challenges such as benchmarking, memory limits, data quality, and lack of standard models and public datasets.

Proposed method

• The survey employs the PRISMA methodology for systematic literature review, ensuring completeness and reproducibility.
• It classifies TinyML systems based on three distinct design workflows: ML-focused, hardware-focused, and co-design, with emphasis on collaborative design.
• A detailed taxonomy is proposed, categorizing learning techniques by optimization family (e.g., quantization, knowledge distillation, neural architecture search) and model design.
• The study evaluates state-of-the-art TinyML frameworks and tools, including their compatibility with embedded MCUs and low-power constraints.
• Hardware platforms are analyzed based on performance, memory, and energy efficiency, with comparative analysis in Table IV.
• Unresolved issues are identified through critical synthesis of current literature, focusing on benchmarking, memory, data quality, and model standardization.

Figure 1: A glance at the latest hardware developed for TinyML reveals a notable trend: recent advances are focused on the goal of minimizing power consumption. This means that the main focus right now is making sure that ML can run on devices with limited resources.

Experimental results

Research questions

• RQ1 What are the three primary design workflows for implementing TinyML systems, and how do they differ in practice and performance?
• RQ2 Which model optimization and learning techniques are most effective for resource-constrained edge devices in TinyML?
• RQ3 How do current hardware platforms (MCUs, FPGAs, TPUs) compare in terms of energy efficiency, memory usage, and inference speed for TinyML applications?
• RQ4 What are the major unresolved challenges hindering the advancement and deployment of TinyML systems in real-world scenarios?
• RQ5 Why is there a lack of standardized benchmarks, public datasets, and widely accepted models in the TinyML ecosystem?

Key findings

• Co-design workflows, where ML and hardware engineers collaborate from the outset, show the most promise for future advancements in TinyML.
• Despite growing interest, no widely accepted baseline model exists for TinyML on microcontroller units (MCUs), unlike MobileNet in mobile edge AI.
• The lack of standardized benchmarks for training, inference, and system-level performance remains a major barrier to fair comparison and progress.
• Memory constraints—particularly limited SRAM and flash—remain a critical challenge, especially for deep learning models.
• Data quality is a pivotal but often overlooked factor; real-world data scarcity and poor quality can mislead model evaluation and deployment.
• While deep learning dominates TinyML applications, non-deep learning methods like TEDA are used sporadically, indicating limited adoption outside specific niches.

Figure 2: The number of papers on TinyML published so far, and the surveys on the topic. As evident, there is an exponential growth in the number of research papers, and it’s worth noting that our survey not only stands as the most recent but also uniquely concentrates on the ML perspective, disting