[Paper Review] Graph-based Asynchronous Event Processing for Rapid Object Recognition

Yijin Li, Han Zhou+5 more|arXiv (Cornell University)|Aug 28, 2023

Advanced Memory and Neural Computing4 citations

TL;DR

This paper proposes SlideGCN, a graph-based asynchronous event processing framework that enables efficient, event-wise recognition in real time by introducing incremental graph convolution and a specialized radius search for event clouds. It reduces computational complexity by up to 100× compared to naive sliding-window methods while achieving state-of-the-art accuracy and enabling early recognition with high confidence.

ABSTRACT

Different from traditional video cameras, event cameras capture asynchronous events stream in which each event encodes pixel location, trigger time, and the polarity of the brightness changes. In this paper, we introduce a novel graph-based framework for event cameras, namely SlideGCN. Unlike some recent graph-based methods that use groups of events as input, our approach can efficiently process data event-by-event, unlock the low latency nature of events data while still maintaining the graph's structure internally. For fast graph construction, we develop a radius search algorithm, which better exploits the partial regular structure of event cloud against k-d tree based generic methods. Experiments show that our method reduces the computational complexity up to 100 times with respect to current graph-based methods while keeping state-of-the-art performance on object recognition. Moreover, we verify the superiority of event-wise processing with our method. When the state becomes stable, we can give a prediction with high confidence, thus making an early recognition. Project page: \url{https://zju3dv.github.io/slide_gcn/}.

Motivation & Objective

• To address the trade-off between low latency and high accuracy in event-based object recognition.
• To enable true event-wise processing in graph-based methods without sacrificing performance.
• To reduce computational complexity in graph construction and inference for sparse, asynchronous event streams.
• To support early recognition by detecting stable prediction states with high confidence during event processing.
• To design a radius search algorithm tailored to the 2D grid structure of event clouds for faster graph construction.

Proposed method

• Introduces slide convolution, an incremental graph convolution method that reuses features from previous graph states and only updates features around newly added nodes.
• Derives propagation rules for multi-layer GCN to handle dynamic graph topologies across time steps with minimal recomputation.
• Proposes a pixel-queue-based radius search algorithm that exploits the 2D grid structure of event clouds, reducing search cost by half and enabling O(1) insertion/deletion operations.
• Employs a state-aware module that monitors prediction confidence and triggers early recognition when stability is reached.
• Uses a sliding window strategy with recursive graph updates to maintain temporal coherence while minimizing redundant computation.
• Combines event-wise processing with graph neural networks to preserve sparsity and temporal resolution of event data.

Experimental results

Research questions

• RQ1 Can graph-based methods achieve low-latency, event-wise processing without compromising recognition accuracy?
• RQ2 How can graph convolution be made incremental to reduce computational cost in dynamic event streams?
• RQ3 Can a specialized radius search algorithm for 2D event clouds significantly accelerate graph construction?
• RQ4 Does event-wise processing enable earlier and more confident recognition compared to batch-based methods?
• RQ5 What is the trade-off between latency, computational cost, and accuracy in event-based recognition?

Key findings

• SlideGCN reduces computational complexity by up to 100× compared to naive sliding-window graph convolution methods.
• The method achieves state-of-the-art accuracy (0.669) on stream-based object recognition, comparable to batch-wise methods with a 50ms window (0.761).
• SlideGCN enables early recognition with high confidence, responding at 12ms (event index 3118), while batch-wise methods require 50ms (event index 19154).
• The proposed radius search algorithm cuts search time by 50% and reduces insertion/deletion cost to O(1) by leveraging the 2D grid structure of event clouds.
• Cumulative MFLOPS for event-wise processing is 17,711 (batch size 1), which is higher than batch-wise (1,152), but allows for significantly lower latency.
• With a mini-batch size of 100, SlideGCN achieves a balance between low latency and reduced computational load, offering a practical trade-off for real-time deployment.