[Paper Review] Embodied Event-Driven Random Backpropagation.

Jacques Kaiser, Alexander W. Friedrich+5 more|arXiv (Cornell University)|Apr 9, 2019

Advanced Memory and Neural Computing24 references4 citations

TL;DR

This paper proposes Event-Driven Random Backpropagation (eRBP), a spiking neural network learning rule that enables efficient, low-power representation learning from dynamic vision sensor (DVS) event streams. By integrating eRBP with a covert attention mechanism for translation invariance and deploying it in a robotic system with microsaccadic eye movements, the method achieves human-level affordance classification within 100ms, demonstrating real-time, energy-efficient visual processing in autonomous robots.

ABSTRACT

Spike-based communication between biological neurons is sparse and unreliable. This enables the brain to process visual information from the eyes efficiently. Taking inspiration from biology, artificial spiking neural networks coupled with silicon retinas attempt to model these computations. Recent findings in machine learning allowed the derivation of a family of powerful synaptic plasticity rules approximating backpropagation for spiking networks. Are these rules capable of processing real-world visual sensory data? In this paper, we evaluate the performance of Event-Driven Random Backpropagation (eRBP) at learning representations from event streams provided by a Dynamic Vision Sensor (DVS). First, we show that eRBP matches state-of-the-art performance on DvsGesture with the addition of a simple covert attention mechanism. By remapping visual receptive fields relatively to the center of the motion, this attention mechanism provides translation invariance at low computational cost compared to convolutions. Second, we successfully integrate eRBP in a real robotic setup, where a robotic arm grasps objects with respect to detected visual affordances. In this setup, visual information is actively sensed by a DVS mounted on a robotic head performing microsaccadic eye movements. We show that our method quickly classifies affordances within 100ms after microsaccade onset, comparable to human performance reported in behavioral study. Our results suggest that advances in neuromorphic technology and plasticity rules enable the development of autonomous robots operating at high speed and low energy budget.

Motivation & Objective

• To evaluate whether event-driven backpropagation rules can effectively learn representations from real-world visual sensory data captured by dynamic vision sensors (DVS).
• To enhance eRBP with a covert attention mechanism that provides translation invariance without relying on costly convolutional operations.
• To validate the method in a real robotic system where visual affordances are actively sensed via microsaccadic eye movements.
• To demonstrate that the system achieves fast, human-level classification speed (within 100ms post-microsaccade) in real-time robotic tasks.

Proposed method

• The paper employs Event-Driven Random Backpropagation (eRBP), a synaptic plasticity rule that approximates backpropagation in spiking neural networks using sparse, event-based signals.
• A covert attention mechanism is introduced to remap visual receptive fields relative to the center of motion, enabling translation invariance with minimal computational overhead.
• The method is integrated into a robotic system with a DVS mounted on a robotic head that performs microsaccadic eye movements to actively sample visual input.
• Spike-based communication and event-driven learning allow for low-energy, high-speed processing of dynamic visual streams.
• The network is trained to classify visual affordances based on event streams generated during object interaction tasks.
• Performance is evaluated on the DvsGesture benchmark and in a real robotic grasping setup under real-time constraints.

Experimental results

Research questions

• RQ1 Can eRBP effectively learn visual representations from real-world DVS event streams in a robotic setting?
• RQ2 How does the proposed covert attention mechanism improve translation invariance compared to standard convolutional approaches in event-based learning?
• RQ3 Can eRBP achieve human-level speed in visual affordance classification when combined with active sensing via microsaccadic eye movements?
• RQ4 What is the computational and energy efficiency of eRBP in real-time robotic perception tasks?

Key findings

• eRBP achieves state-of-the-art performance on the DvsGesture benchmark when combined with the covert attention mechanism.
• The covert attention mechanism provides translation invariance at a significantly lower computational cost than traditional convolutional layers.
• The system classifies visual affordances within 100ms after microsaccade onset, matching the reaction time reported in human behavioral studies.
• The integration of eRBP into a robotic platform enables real-time, low-energy visual processing using event-based sensory input.
• The method demonstrates robust performance in active sensing scenarios where visual input is sampled through controlled microsaccadic movements.
• The results suggest that eRBP, combined with neuromorphic sensing and attention mechanisms, enables high-speed, low-power operation in autonomous robotic systems.