[Paper Review] TinyOL: TinyML with Online-Learning on Microcontrollers

Haoyu Ren, Darko Anicic+1 more|arXiv (Cornell University)|Mar 15, 2021

Data Stream Mining Techniques19 references4 citations

TL;DR

TinyOL introduces a novel framework for on-device incremental learning on microcontrollers (MCUs), enabling real-time model adaptation using online learning. By attaching a lightweight online learning layer to pre-trained neural networks, TinyOL allows MCUs to update weights incrementally from streaming data without retraining from scratch, demonstrating improved performance on anomaly detection and classification tasks with minimal resource overhead.

ABSTRACT

Tiny machine learning (TinyML) is a fast-growing research area committed to democratizing deep learning for all-pervasive microcontrollers (MCUs). Challenged by the constraints on power, memory, and computation, TinyML has achieved significant advancement in the last few years. However, the current TinyML solutions are based on batch/offline settings and support only the neural network's inference on MCUs. The neural network is first trained using a large amount of pre-collected data on a powerful machine and then flashed to MCUs. This results in a static model, hard to adapt to new data, and impossible to adjust for different scenarios, which impedes the flexibility of the Internet of Things (IoT). To address these problems, we propose a novel system called TinyOL (TinyML with Online-Learning), which enables incremental on-device training on streaming data. TinyOL is based on the concept of online learning and is suitable for constrained IoT devices. We experiment TinyOL under supervised and unsupervised setups using an autoencoder neural network. Finally, we report the performance of the proposed solution and show its effectiveness and feasibility.

Motivation & Objective

• Address the limitations of static, batch-trained models in TinyML by enabling on-device model adaptation to dynamic environments.
• Overcome challenges in IoT deployment such as remote device updates, concept drift, and data transmission costs.
• Enable incremental learning on bare-metal MCUs without requiring full retraining or external cloud support.
• Demonstrate feasibility of on-device post-training and dynamic layer structure adaptation using autoencoders on low-power hardware.
• Provide a flexible, lightweight system that enhances model accuracy over time using streaming field data.

Proposed method

• Integrate a lightweight online learning layer into existing neural networks running on MCUs, operating in a streaming, incremental fashion.
• Use stochastic gradient descent (SGD) with one-pass updates per incoming data sample, discarding data after a single update.
• Implement a modular C++ library that can be attached to any pre-trained neural network as an additional layer.
• Support dynamic reconfiguration of the output layer structure when new classes are introduced, enabling on-the-fly adaptation.
• Utilize autoencoders for unsupervised feature learning and reconstruction error as input for classification tasks.
• Apply F1-score and macro F1-score metrics to evaluate performance under both online and offline training regimes.

Experimental results

Research questions

• RQ1 Can incremental online learning be effectively implemented on resource-constrained microcontrollers for TinyML applications?
• RQ2 How does the performance of online learning compare to traditional batch training under limited computational and memory resources?
• RQ3 Can on-device model adaptation improve model accuracy over time when new data patterns emerge?
• RQ4 To what extent can the system dynamically adjust its architecture (e.g., number of output classes) in response to new data?
• RQ5 Is it feasible to perform post-training and model remodeling directly on MCUs without relying on cloud-based retraining?

Key findings

• TinyOL enables incremental on-device training on MCUs using streaming data, allowing models to adapt to new data patterns without retraining from scratch.
• The online learning model achieved competitive performance compared to offline training, especially when data and computational resources were limited.
• F1-scores improved with more streaming data, and the macro F1-score increased steadily with each learning iteration.
• The system successfully enabled an unsupervised autoencoder to acquire classification capabilities for normal, stuck, and tilted fan modes.
• The encoder’s frozen weights limited performance on abnormal classes, suggesting that full-weight fine-tuning could further improve results.
• TinyOL demonstrated feasibility and effectiveness in real-world scenarios on an Arduino Nano 33 BLE Sense board with a 64MHz Cortex-M4 CPU and 256KB RAM.