[Paper Review] Flappy Hummingbird: An Open Source Dynamic Simulation of Flapping Wing Robots and Animals

Fei Fan, Zhan Tu+3 more|arXiv (Cornell University)|Feb 25, 2019

Biomimetic flight and propulsion mechanisms24 references4 citations

TL;DR

This paper presents an open-source, high-fidelity dynamic simulation for flapping wing micro air vehicles (FWMAVs) and animals, using a physics-based model with DART and blade element aerodynamics. It enables accurate sim-to-real transfer of both linear and deep reinforcement learning controllers, validated by matching experimental flight performance in hovering and fast maneuvering tasks.

ABSTRACT

Insects and hummingbirds exhibit extraordinary flight capabilities and can simultaneously master seemingly conflicting goals: stable hovering and aggressive maneuvering, unmatched by small scale man-made vehicles. Flapping Wing Micro Air Vehicles (FWMAVs) hold great promise for closing this performance gap. However, design and control of such systems remain challenging due to various constraints. Here, we present an open source high fidelity dynamic simulation for FWMAVs to serve as a testbed for the design, optimization and flight control of FWMAVs. For simulation validation, we recreated the hummingbird-scale robot developed in our lab in the simulation. System identification was performed to obtain the model parameters. The force generation, open-loop and closed-loop dynamic response between simulated and experimental flights were compared and validated. The unsteady aerodynamics and the highly nonlinear flight dynamics present challenging control problems for conventional and learning control algorithms such as Reinforcement Learning. The interface of the simulation is fully compatible with OpenAI Gym environment. As a benchmark study, we present a linear controller for hovering stabilization and a Deep Reinforcement Learning control policy for goal-directed maneuvering. Finally, we demonstrate direct simulation-to-real transfer of both control policies onto the physical robot, further demonstrating the fidelity of the simulation.

Motivation & Objective

• To address the lack of accessible, high-fidelity simulation tools for flapping wing micro air vehicles (FWMAVs) and bio-inspired fliers.
• To enable rapid design optimization and control algorithm development for FWMAVs without costly physical prototyping.
• To validate simulation fidelity through direct sim-to-real transfer of control policies on a physical hummingbird-scale robot.
• To provide a benchmark environment for control theory, reinforcement learning, and biological flight studies.
• To support integration with ROS and Gazebo for advanced robotics research in perception, localization, and control.

Proposed method

• Developed a C++-based simulation with Python bindings using the DART physics engine for multi-body dynamics and contact modeling.
• Implemented a customized quasi-steady blade element theory aerodynamic model calibrated via force/torque measurements on a physical robot.
• Performed system identification on the physical FWMAV to extract accurate physical parameters (mass, inertia, spring stiffness, etc.).
• Enabled OpenAI Gym compatibility for seamless integration with reinforcement learning and control theory research.
• Designed a linear cascading PID controller for position tracking and a deep deterministic policy gradient (DDPG) policy for goal-directed maneuvering.
• Applied dynamics randomization during DRL training to improve robustness and ensure sim-to-real transferability.

Experimental results

Research questions

• RQ1 Can a high-fidelity simulation accurately replicate the dynamic behavior of a physical flapping wing micro air vehicle?
• RQ2 To what extent can control policies trained in simulation be successfully transferred to real-world FWMAV hardware without re-tuning?
• RQ3 How well does the simulation capture unsteady aerodynamics and nonlinear flight dynamics critical for agile maneuvers?
• RQ4 Can the simulation serve as a reliable benchmark for training and evaluating reinforcement learning policies in flapping flight?
• RQ5 How does the inclusion of tactile feedback from motor and contact dynamics enhance control and planning in simulation?

Key findings

• The simulation achieved high fidelity in open-loop state transition dynamics, with state transition error matching experimental data within acceptable bounds.
• Closed-loop control using a tuned PID controller showed near-identical performance between simulation and experiment, with RMS tracking errors differing by less than 10% across all axes.
• The DDPG-based reinforcement learning policy successfully learned a fast, multi-axis maneuver from [0,0,0] to [-0.21,0,0] with 180° yaw change, mimicking hummingbird escape behavior.
• The sim-to-real transfer of both the PID and DRL controllers succeeded without any domain randomization or re-tuning, confirming simulation fidelity.
• The simulation accurately captured nonlinear dynamics, including roll torque offsets and thrust misalignment, which were observed in both simulation and real flight.
• The simulation supports advanced robotics research by enabling integration with ROS and Gazebo for perception, localization, and tactile feedback modeling.