[Paper Review] Agile Reactive Navigation for A Non-Holonomic Mobile Robot Using A Pixel Processor Array

Yanan Liu, Laurie Bose+7 more|arXiv (Cornell University)|Sep 27, 2020

Robotics and Sensor-Based Localization21 references4 citations

TL;DR

This paper presents an agile reactive navigation system for a non-holonomic mobile robot using a Pixel Processor Array (PPA) to perform real-time image processing directly on the sensor, enabling high-speed navigation through gates at up to 3.88 m/s. By processing visual features like target patterns directly on the SCAMP-5 chip at over 2000 fps, the system bypasses traditional image transfer bottlenecks and achieves real-time control with minimal computational overhead.

ABSTRACT

This paper presents an agile reactive navigation strategy for driving a non-holonomic ground vehicle around a preset course of gates in a cluttered environment using a low-cost processor array sensor. This enables machine vision tasks to be performed directly upon the sensor's image plane, rather than using a separate general-purpose computer. We demonstrate a small ground vehicle running through or avoiding multiple gates at high speed using minimal computational resources. To achieve this, target tracking algorithms are developed for the Pixel Processing Array and captured images are then processed directly on the vision sensor acquiring target information for controlling the ground vehicle. The algorithm can run at up to 2000 fps outdoors and 200fps at indoor illumination levels. Conducting image processing at the sensor level avoids the bottleneck of image transfer encountered in conventional sensors. The real-time performance of on-board image processing and robustness is validated through experiments. Experimental results demonstrate that the algorithm's ability to enable a ground vehicle to navigate at an average speed of 2.20 m/s for passing through multiple gates and 3.88 m/s for a 'slalom' task in an environment featuring significant visual clutter.

Motivation & Objective

• Enable high-speed, reactive navigation for non-holonomic mobile robots in cluttered environments using low-cost, low-power vision processing.
• Overcome the latency and power inefficiency of conventional vision pipelines that rely on off-sensor image transfer and general-purpose computing.
• Develop a direct control loop linking visual features extracted on the sensor to robot actuation, eliminating coordinate transformation overhead.
• Achieve robust performance under motion disturbances and variable lighting by leveraging on-sensor image processing and filtering.
• Demonstrate real-time agility in complex tasks such as gate passage and slalom navigation using minimal computational resources.

Proposed method

• Utilized the SCAMP-5 Pixel Processor Array (PPA) to perform on-sensor image processing, enabling parallel computation directly on the image plane.
• Implemented iterative flooding and neighborhood-based filtering algorithms on the PPA to extract and denoise target features (e.g., circular disks) from visual patterns.
• Extracted center coordinates of four target dots from the patterns directly on the PPA, eliminating the need for coordinate system transformations.
• Transmitted only the processed feature coordinates (not full images) to a Raspberry Pi for PID-based control of the robot’s steering and speed.
• Designed visual patterns with encoded turning instructions (left/right turns) to guide slalom navigation, interpreted directly by the PPA.
• Achieved high frame rates (up to 2000 fps outdoors, 200 fps indoors) by leveraging the PPA’s parallel processing and minimizing data movement.

Experimental results

Research questions

• RQ1 Can on-sensor image processing using a Pixel Processor Array enable real-time, agile reactive navigation for a non-holonomic robot in cluttered environments?
• RQ2 To what extent does eliminating image transfer bottlenecks improve response time and navigation speed compared to conventional vision pipelines?
• RQ3 How effective is the direct mapping of visual feature coordinates to robot control commands in reducing computational overhead and improving agility?
• RQ4 What is the maximum achievable navigation speed for a reactive system using only on-sensor processing and minimal external computation?
• RQ5 How robust is the system under motion-induced disturbances and variable lighting conditions during high-speed operation?

Key findings

• The system achieved an average navigation speed of 2.20 m/s while passing through multiple gates, with a maximum speed of 2.50 m/s as recorded by VICON tracking.
• In a high-speed 'slalom' task, the robot reached a peak velocity of 3.88 m/s, equivalent to 139 km/h for a full-sized car, demonstrating exceptional agility.
• The SCAMP-5 PPA processed images at over 2000 fps under outdoor lighting, significantly reducing latency compared to traditional pipelines.
• The total system processing delay was approximately 472 μs, with the majority of time (328 μs) spent on coordinate extraction and transmission to the Raspberry Pi.
• The direct link between on-sensor feature detection and robot control eliminated the need for complex coordinate transformations, improving robustness and reducing computational load.
• The system demonstrated robust performance under motion-induced vibrations and visual clutter, maintaining accurate navigation during high-speed maneuvers.