[Paper Review] EVDodge: Embodied AI For High-Speed Dodging On A Quadrotor Using Event Cameras.

Nitin J. Sanket, Chethan M. Parameshwara+5 more|arXiv (Cornell University)|Jun 7, 2019

Robotics and Sensor-Based Localization9 references14 citations

TL;DR

This paper presents EVDodge, a real-time, on-board deep learning system for high-speed dodging on quadrotors using a single event camera. It employs shallow neural networks to estimate ego-motion and dynamic obstacle motion in simulation, transferring directly to real-world scenarios with 70% success across diverse obstacles and low-light conditions—marking the first end-to-end learning-based solution for event camera-based dynamic obstacle avoidance on quadrotors.

ABSTRACT

Dynamic obstacle avoidance on quadrotors requires low latency. A class of sensors that are particularly suitable for such scenarios are event cameras. In this paper, we present a deep learning -- based solution for dodging multiple dynamic obstacles on a quadrotor with a single event camera and on-board computation. Our approach uses a series of shallow neural networks for estimating both the ego-motion and the motion of independently moving objects. The networks are trained in simulation and directly transfer to the real world without any fine-tuning or retraining. We successfully evaluate and demonstrate the proposed approach in many real-world experiments with obstacles of different shapes and sizes, achieving an overall success rate of 70% including objects of unknown shape and a low light testing scenario. To our knowledge, this is the first deep learning -- based solution to the problem of dynamic obstacle avoidance using event cameras on a quadrotor. Finally, we also extend our work to the pursuit task by merely reversing the control policy, proving that our navigation stack can cater to different scenarios.

Motivation & Objective

• To enable real-time, on-board dynamic obstacle avoidance on quadrotors under high-speed conditions.
• To address the challenge of low-latency perception in dynamic environments using event cameras.
• To develop a learning-based system that transfers directly from simulation to real-world deployment without fine-tuning.
• To demonstrate robustness across diverse obstacle shapes, sizes, and low-light conditions.
• To extend the framework to other navigation tasks, such as pursuit, via policy reversal.

Proposed method

• A series of shallow neural networks are trained in simulation to estimate both ego-motion and motion of independently moving obstacles from event camera data.
• The networks process asynchronous event streams, extracting spatiotemporal features to infer motion vectors in real time.
• The system is trained entirely in simulation using synthetic data with diverse obstacle dynamics and environmental conditions.
• The trained policy is deployed directly on the quadrotor with no real-world fine-tuning, ensuring low-latency inference.
• Control policy is reversed to enable pursuit tasks, demonstrating the versatility of the navigation stack.
• Onboard computation ensures real-time performance, critical for high-speed flight.

Experimental results

Research questions

• RQ1 Can a learning-based system using event cameras achieve real-time dynamic obstacle avoidance on a quadrotor with minimal latency?
• RQ2 How well does a simulation-trained policy generalize to real-world scenarios without fine-tuning?
• RQ3 What is the performance of such a system when encountering obstacles of unknown shape and in low-light conditions?
• RQ4 Can the same perception and control stack be repurposed for different navigation tasks like pursuit?
• RQ5 What level of success can be achieved in high-speed dodging scenarios using only a single event camera?

Key findings

• The system achieves a 70% success rate in real-world high-speed dodging experiments, including obstacles of unknown shape.
• The simulation-to-reality transfer is effective without any fine-tuning, demonstrating strong generalization.
• The method operates successfully in low-light conditions, highlighting the advantages of event camera sensing.
• The same perception and control stack enables pursuit tasks through policy reversal, indicating broad applicability.
• The use of shallow networks enables real-time inference on-board, satisfying the low-latency requirements of high-speed flight.
• This is the first learning-based solution for dynamic obstacle avoidance on quadrotors using event cameras in real-world deployment.