[Paper Review] ATIS + SpiNNaker: a Fully Event-based Visual Tracking Demonstration

Arren Glover, Alan B. Stokes+2 more|arXiv (Cornell University)|Dec 3, 2019

Advanced Memory and Neural Computing2 references4 citations

TL;DR

This paper presents a fully event-based visual tracking system using the ATIS event camera and SpiNNaker neuromorphic hardware, leveraging an asynchronous particle filter algorithm that runs in parallel across SpiNNaker's cores. The system achieves real-time tracking at over 1 kHz with hundreds of particles, demonstrating robustness to motion-induced noise and scalability through parallel processing with minimal latency overhead.

ABSTRACT

The Asynchronous Time-based Image Sensor (ATIS) and the Spiking Neural Network Architecture (SpiNNaker) are both neuromorphic technologies that "unconventionally" use binary spikes to represent information. The ATIS produces spikes to represent the change in light falling on the sensor, and the SpiNNaker is a massively parallel computing platform that asynchronously sends spikes between cores for processing. In this demonstration we show these two hardware used together to perform a visual tracking task. We aim to show the hardware and software architecture that integrates the ATIS and SpiNNaker together in a robot middle-ware that makes processing agnostic to the platform (CPU or SpiNNaker). We also aim to describe the algorithm, why it is suitable for the "unconventional" sensor and processing platform including the advantages as well as challenges faced.

Motivation & Objective

• To integrate ATIS event-based cameras and SpiNNaker neuromorphic processors into a unified robotic middleware for platform-agnostic processing.
• To demonstrate real-time visual tracking using asynchronous event streams and parallel spiking computation.
• To validate the suitability of a particle filter for event-based vision by adapting it to asynchronous updates and continuous motion modeling.
• To evaluate the scalability and performance of the system on neuromorphic hardware compared to traditional CPU implementations.

Proposed method

• The system uses the ATIS sensor to generate asynchronous binary spikes representing light intensity changes, avoiding frame-based sampling.
• A particle filter is implemented on the SpiNNaker platform, where each particle's state (position, size) and weight are updated based on event stream likelihood.
• Event-driven filter updates occur in real-time as events arrive, increasing update frequency during high-motion states.
• Likelihood estimation is computed incrementally based on temporal event order, assuming a Markov process over an unknown time window.
• A constant-position motion model is proposed, enabled by the continuous spatiotemporal resolution of event cameras, avoiding frame-jump artifacts.
• Communication between SpiNNaker cores is managed via a networked architecture: ROI filters process events in parallel, and particle nodes synchronize state and weight information across cores.

Experimental results

Research questions

• RQ1 Can a particle filter be effectively adapted to process asynchronous, continuous event streams from an ATIS camera?
• RQ2 How does parallel execution on SpiNNaker improve scalability and update rate compared to traditional CPU-based implementations?
• RQ3 What are the performance and latency characteristics of a fully event-based tracking pipeline using neuromorphic hardware?
• RQ4 To what extent is the system robust to background clutter and self-motion-induced noise during tracking?
• RQ5 Can the particle filter maintain high accuracy while scaling to hundreds of particles with minimal communication overhead on SpiNNaker?

Key findings

• The system achieved real-time tracking with update rates exceeding 1 kHz, even with hundreds of particles, demonstrating high computational efficiency.
• The particle filter achieved over 2 kHz update frequency when required, indicating strong responsiveness to rapid motion.
• The system remained robust to false detections caused by background clutter during robot self-motion, thanks to the event-based likelihood model.
• Communication latency between particles was less than 4.6 μs per particle, significantly lower than the computational scaling seen in CPU-based implementations.
• The number of particles could be scaled up to over 500 on a single 48-chip SpiNNaker board after instantiation of 240 ROI filter vertices, with minimal performance degradation.
• The main performance bottleneck was synchronization among particles, as all particles must wait for each other before proceeding, unlike fully independent feed-forward networks.