[Paper Review] Automated Driver Testing for Small Footprint Embedded Systems

Sara C.M. de Souza, Rogério Atem de Carvalho|arXiv (Cornell University)|May 4, 2021

Software Testing and Debugging Techniques5 references4 citations

TL;DR

This paper presents a low-cost, noninvasive testing framework for drivers in small-footprint embedded systems using a Device Under Test (DUT), a Double device simulating external peripherals, and a host computer running test orchestration code in Python. The method enables effective driver testing across multiple communication protocols—including SPI, I2C, UART, CAN, and BLE—by exposing implementation flaws, such as BLE initialization instability, and has been validated through real test cases with reproducible results on ESP32 using MicroPython.

ABSTRACT

Embedded systems represent a billionaire market and are present in most of the processors produced in the world.Embedded software interacts with external peripherals such as sensors and actuators through drivers. The strong interaction between drivers and external peripherals often hamper embedded software development, in special the testing task, giving that the physical environment may not be deterministic and difficult to be recreated during the tests, the hardware may not be available or its presence may not be desirable or costly. Aiming at addressing these problems, this paper introduces a solution to test drivers on microcontrollers, based on a method that uses three components: a Device Under Test (DUT), a Double device, and a computer. The computer runs a test orchestration code, while the DUT runs the test target code, and the Double plays the role of the real external peripherals that interact with the DUT. The proposed solution was successfully implemented and validated using different protocols.

Motivation & Objective

• To address the challenge of testing device drivers in small-footprint embedded systems where physical hardware is unavailable, expensive, or risky to use during development.
• To reduce testing costs and complexity in embedded systems, where up to 80% of project resources may be spent on testing, especially in safety-critical applications.
• To provide a practical, reusable, and transparent solution for testing drivers without modifying production code or requiring special debug modes.
• To identify and report implementation issues in driver libraries, such as BLE initialization instability, through automated test execution.
• To enable reproducible, deterministic testing of embedded drivers using simulated peripherals (Doubles) across multiple communication protocols.

Proposed method

• The framework uses three components: a Device Under Test (DUT) running the driver code, a Double device emulating external peripherals, and a host computer running Python-based test orchestration code.
• The test code on the computer sends commands via serial communication to both the DUT and the Double, then compares expected vs. actual results to validate functionality.
• The Double device runs 'Double Code'—a simulated peripheral implementation in MicroPython—that mimics real hardware behavior, such as BLE advertising or SPI communication.
• Each test case is isolated and follows Test-Driven Development (TDD) principles, with test code written before the production driver code.
• The system supports multiple protocols (SPI, I2C, UART, CAN, BLE) by leveraging the ESP32’s protocol diversity and MicroPython’s interactive terminal for real-time feedback.
• The solution is noninvasive: no code changes or debug modes are required on the DUT, and testing occurs in a deterministic, repeatable environment.

Experimental results

Research questions

• RQ1 How can driver testing in small-footprint embedded systems be made practical and cost-effective when physical peripherals are unavailable or impractical?
• RQ2 To what extent can a hardware Double device simulate real peripherals to enable reliable, repeatable testing of embedded drivers?
• RQ3 Can automated test frameworks built on this architecture detect real implementation flaws in driver libraries, such as timing issues in BLE initialization?
• RQ4 How effective is this framework in supporting multiple communication protocols (e.g., SPI, I2C, BLE) within a single, reusable testing architecture?
• RQ5 What role does test-driven development play in improving the reliability and maintainability of embedded drivers in resource-constrained systems?

Key findings

• The framework successfully detected instability in BLE driver initialization, which was later confirmed and reported to the relevant GitHub community, demonstrating its effectiveness in uncovering real implementation issues.
• Test cases for SPI and BLE protocols revealed instability due to immature libraries on the target platform, highlighting the framework’s role in early bug detection.
• The solution achieved deterministic test execution by replacing non-deterministic physical hardware with a simulated Double device, enabling repeatable and reliable testing.
• The use of MicroPython on ESP32 enabled interactive test execution and real-time result inspection, improving developer feedback and debugging efficiency.
• The framework’s noninvasive nature allowed testing without modifying production code or requiring special debug configurations, reducing integration overhead.
• All artifacts, including libraries, templates, and test cases, were published on GitHub with clear documentation, promoting community adoption and collaboration.