[Paper Review] Conservation AI: Live Stream Analysis for the Detection of Endangered Species Using Convolutional Neural Networks and Drone Technology

Carl Chalmers, Paul Fergus+2 more|arXiv (Cornell University)|Oct 16, 2019

Fish Ecology and Management Studies28 references19 citations

TL;DR

This paper presents Conservation AI, a real-time detection system using drone-mounted cameras and a fine-tuned Faster R-CNN ResNet-101 model to identify endangered rhinos and poaching-related vehicles. Trained on 350 images per class, the system achieves a mean average precision (mAP) of 0.83 at IOU 0.50, demonstrating a scalable, low-cost solution for early anti-poaching intervention using off-the-shelf drones and deep learning.

ABSTRACT

Many different species are adversely affected by poaching. In response to this escalating crisis, efforts to stop poaching using hidden cameras, drones and DNA tracking have been implemented with varying degrees of success. Limited resources, costs and logistical limitations are often the cause of most unsuccessful poaching interventions. The study presented in this paper outlines a flexible and interoperable framework for the automatic detection of animals and poaching activity to facilitate early intervention practices. Using a robust deep learning pipeline, a convolutional neural network is trained and implemented to detect rhinos and cars (considered an important tool in poaching for fast access and artefact transportation in natural habitats) in the study, that are found within live video streamed from drones Transfer learning with the Faster RCNN Resnet 101 is performed to train a custom model with 350 images of rhinos and 350 images of cars. Inference is performed using a frame sampling technique to address the required trade-off control precision and processing speed and maintain synchronisation with the live feed. Inference models are hosted on a web platform using flask web serving, OpenCV and TensorFlow 1.13. Video streams are transmitted from a DJI Mavic Pro 2 drone using the Real-Time Messaging Protocol (RMTP). The best trained Faster RCNN model achieved a mAP of 0.83 @IOU 0.50 and 0.69 @IOU 0.75 respectively. In comparison an SSD-mobilenetmodel trained under the same experimental conditions achieved a mAP of 0.55 @IOU .50 and 0.27 @IOU 0.75.The results demonstrate that using a FRCNN and off-the-shelf drones is a promising and scalable option for a range of conservation projects.

Motivation & Objective

• Address the growing threat of wildlife poaching through automated, real-time detection systems.
• Overcome limitations of traditional anti-poaching methods such as high costs, logistical constraints, and delayed response times.
• Develop a scalable, low-cost solution using off-the-shelf drones and deep learning for conservation applications.
• Enable early intervention by detecting both endangered animals and poaching-related vehicles in live video streams.
• Create a flexible, interoperable framework deployable across diverse conservation projects with minimal infrastructure.

Proposed method

• Utilize a DJI Mavic Pro 2 drone to stream live video via the Real-Time Messaging Protocol (RTMP).
• Implement a frame sampling technique to balance processing speed and detection accuracy while maintaining synchronization with the live feed.
• Apply transfer learning with Faster R-CNN based on ResNet-101 to fine-tune a custom object detection model on 350 rhino and 350 vehicle images.
• Host the inference model on a web platform using Flask, OpenCV, and TensorFlow 1.13 for real-time processing and visualization.
• Train and evaluate the model using mean average precision (mAP) at IOU thresholds of 0.50 and 0.75 to assess detection robustness.
• Compare performance against an SSD-Mobilenet baseline under identical experimental conditions to validate model effectiveness.

Experimental results

Research questions

• RQ1 Can a deep learning model trained on a limited dataset of 350 rhino and 350 vehicle images achieve reliable real-time detection in live drone video streams?
• RQ2 How does the performance of Faster R-CNN compare to SSD-Mobilenet in detecting rhinos and vehicles under the same conditions?
• RQ3 To what extent can transfer learning with pre-trained models reduce the need for large annotated datasets in wildlife conservation applications?
• RQ4 Can a system using off-the-shelf drones and open-source software achieve sufficient speed and accuracy for practical anti-poaching deployment?
• RQ5 What trade-offs exist between processing speed, detection precision, and synchronization with live video streams in real-time wildlife monitoring?

Key findings

• The best-performing Faster R-CNN model achieved a mean average precision (mAP) of 0.83 at IOU 0.50 and 0.69 at IOU 0.75, indicating strong detection performance.
• The SSD-Mobilenet baseline model achieved significantly lower mAP scores of 0.55 at IOU 0.50 and 0.27 at IOU 0.75, highlighting the superior accuracy of Faster R-CNN for this task.
• The system successfully maintained synchronization with live video streams through a frame sampling strategy that balanced speed and precision.
• The deployment pipeline using Flask, OpenCV, and TensorFlow 1.13 enabled real-time inference and web-based visualization of detection results.
• The results demonstrate that using a pre-trained Faster R-CNN with transfer learning on a small dataset is effective for detecting key wildlife and poaching indicators.
• The framework is scalable and adaptable, offering a practical, low-cost solution for conservation projects using commercially available drones and deep learning.