[Paper Review] Pytorch-Wildlife: A Collaborative Deep Learning Framework for Conservation

Andrés Hernández, Zhongqi Miao+5 more|arXiv (Cornell University)|May 21, 2024

Species Distribution and Climate Change7 citations

TL;DR

Pytorch-Wildlife is an open-source PyTorch-based platform for accessible, scalable wildlife detection and classification, featuring a model zoo, a user interface, and real-world applications in the Amazon and Galápagos.

ABSTRACT

The alarming decline in global biodiversity, driven by various factors, underscores the urgent need for large-scale wildlife monitoring. In response, scientists have turned to automated deep learning methods for data processing in wildlife monitoring. However, applying these advanced methods in real-world scenarios is challenging due to their complexity and the need for specialized knowledge, primarily because of technical challenges and interdisciplinary barriers. To address these challenges, we introduce Pytorch-Wildlife, an open-source deep learning platform built on PyTorch. It is designed for creating, modifying, and sharing powerful AI models. This platform emphasizes usability and accessibility, making it accessible to individuals with limited or no technical background. It also offers a modular codebase to simplify feature expansion and further development. Pytorch-Wildlife offers an intuitive, user-friendly interface, accessible through local installation or Hugging Face, for animal detection and classification in images and videos. As two real-world applications, Pytorch-Wildlife has been utilized to train animal classification models for species recognition in the Amazon Rainforest and for invasive opossum recognition in the Galapagos Islands. The Opossum model achieves 98% accuracy, and the Amazon model has 92% recognition accuracy for 36 animals in 90% of the data. As Pytorch-Wildlife evolves, we aim to integrate more conservation tasks, addressing various environmental challenges. Pytorch-Wildlife is available at https://github.com/microsoft/CameraTraps.

Motivation & Objective

• Address the need for accessible, scalable, and transparent deep learning tools for wildlife monitoring.
• Provide an open-source, modular framework to create, modify, and share conservation models.
• Enable end-to-end workflows from data ingestion to detection, classification, and visualization.
• Demonstrate real-world applications in biodiversity-rich regions to validate usability and performance.

Proposed method

• Introduce Pytorch-Wildlife as an open-source framework built on PyTorch with a modular architecture.
• Provide a model zoo including MegaDetectorV5 and three animal recognition models trained in the Amazon, Galápagos, and Serengeti.
• Offer a user interface for single/batch image and single video detection with adjustable thresholds.
• Connect datasets to LILA-BC for training/validation and include a classification fine-tuning module.
• Deliver data preprocessing/post-processing utilities and visualization tools.
• Include a compatibility layer with COCO output by default and options to integrate with Timelapse and EcoAssist.

Figure 1 : Overview figure of the Pytorch-wildlife framework. First, it connects to the datasets available on LILA-BC for ease of training and validation; second, it offers detection and classification models with pretrained weights in a variety of datasets; finally, it comes with a user interface a

Experimental results

Research questions

• RQ1 How can an open-source, modular framework improve accessibility and adoption of deep learning in wildlife conservation?
• RQ2 What are the performance and efficiency trade-offs of a compact detector ( MegaDetectorV6-compact ) versus MegaDetectorV5 on standard datasets?
• RQ3 Can the framework support real-world workflows from detection to genus/species classification with human-in-the-loop validation?
• RQ4 How does the model zoo promote transparency and comparability through standardized evaluation and community feedback?

Key findings

• MDv6-compact achieves higher recall (0.85) than MDv5 (0.73) with only one-sixth the parameters (22M vs 121M).
• The MDv6-compact model reaches 0.92 precision and 0.85 recall with 0.84 mAP on the MegaDetectorV5 training data, using YOLOv9-compact.
• Using MegaDetectorV5 for detection, the Amazon dataset yields 90% of images with recognition accuracy 92% on the high-confidence subset.
• Galápagos videos achieve 98% accuracy for opossum vs non-opossum classification via majority voting across frames.
• A human-in-the-loop threshold of 0.98 delivers high confidence results while reducing human validation to about 10% of detections.
• The framework ties into LILA-BC datasets and provides a community-led model zoo with a leaderboard and feedback.”

Figure 2 : Pytorch-wildlife user interface. It allows the user to load detection and classification models, as well as to perform single image detection, batch image detection, and single video detection with a confidence threshold for human validation.