[Paper Review] Choosing an Appropriate Platform and Workflow for Processing Camera Trap Data using Artificial Intelligence

Juliana Vélez, Paula J. Castiblanco-Camacho+4 more|arXiv (Cornell University)|Feb 4, 2022

Species Distribution and Climate Change4 citations

TL;DR

This paper evaluates four AI platforms—Wildlife Insights, MegaDetector, MLWIC2, and Conservation AI—for processing camera trap data using deep learning. It demonstrates that while species classification accuracy remains low due to poor recall, models like MegaDetector achieve high precision and recall in detecting 'animal' presence, enabling efficient filtering of blank images and semi-automated workflows to reduce manual effort in ecological monitoring.

ABSTRACT

Camera traps have transformed how ecologists study wildlife species distributions, activity patterns, and interspecific interactions. Although camera traps provide a cost-effective method for monitoring species, the time required for data processing can limit survey efficiency. Thus, the potential of Artificial Intelligence (AI), specifically Deep Learning (DL), to process camera-trap data has gained considerable attention. Using DL for these applications involves training algorithms, such as Convolutional Neural Networks (CNNs), to automatically detect objects and classify species. To overcome technical challenges associated with training CNNs, several research communities have recently developed platforms that incorporate DL in easy-to-use interfaces. We review key characteristics of four AI-powered platforms -- Wildlife Insights (WI), MegaDetector (MD), Machine Learning for Wildlife Image Classification (MLWIC2), and Conservation AI -- including data management tools and AI features. We also provide R code in an open-source GitBook, to demonstrate how users can evaluate model performance, and incorporate AI output in semi-automated workflows. We found that species classifications from WI and MLWIC2 generally had low recall values (animals that were present in the images often were not classified to the correct species). Yet, the precision of WI and MLWIC2 classifications for some species was high (i.e., when classifications were made, they were generally accurate). MD, which classifies images using broader categories (e.g., "blank" or "animal"), also performed well. Thus, we conclude that, although species classifiers were not accurate enough to automate image processing, DL could be used to improve efficiencies by accepting classifications with high confidence values for certain species or by filtering images containing blanks.

Motivation & Objective

• Address the challenge of high manual labor in processing large volumes of camera trap images.
• Identify technical and usability differences among major AI platforms for ecological image analysis.
• Provide practical guidance on selecting platforms and designing workflows based on user expertise and data needs.
• Demonstrate how to evaluate model performance using standard metrics like precision, recall, and F1 score.
• Promote the integration of AI tools into semi-automated workflows to enhance efficiency in wildlife monitoring.

Proposed method

• Evaluated four AI platforms—Wildlife Insights, MegaDetector, MLWIC2, and Conservation AI—based on software requirements, data management, and AI capabilities.
• Used R code and real data to compute and compare performance metrics: precision, recall, and F1 score for species and object detection.
• Applied confidence threshold tuning in MegaDetector to balance precision and recall for animal detection.
• Evaluated model performance on a dataset of 1,000 camera trap images with known species presence.
• Integrated AI outputs with human review processes to create semi-automated workflows.
• Shared open-source code and guidelines via a GitBook to support reproducibility and community use.

Experimental results

Research questions

• RQ1 Which AI platforms offer the most effective balance of usability, technical requirements, and performance for ecologists processing camera trap data?
• RQ2 How do precision, recall, and F1 scores vary across species classifiers in Wildlife Insights and MLWIC2 compared to general object detectors like MegaDetector?
• RQ3 To what extent can AI models reduce manual effort in camera trap data processing, particularly in filtering blank images?
• RQ4 What role do confidence thresholds play in optimizing detection performance for animal presence in images?
• RQ5 How can AI outputs be integrated into semi-automated workflows to improve efficiency without compromising data quality?

Key findings

• Species classifiers in Wildlife Insights and MLWIC2 showed high precision (e.g., >95%) but low recall (e.g., <65%), meaning many animals were missed despite accurate classifications when made.
• MegaDetector achieved high precision (98%) and high recall (93%) for the 'Animal' class at a 0.65 confidence threshold, indicating strong performance in detecting animals.
• Lowering the confidence threshold in MegaDetector from 0.65 to 0.1 increased recall to 97% but reduced precision to 95%, showing a trade-off that can be tuned based on workflow needs.
• Despite high precision for some species, overall species classification accuracy was insufficient for full automation, necessitating expert review to catch missed detections.
• AI platforms like MegaDetector are most effective not for species-level classification, but for filtering blank images, significantly reducing the workload for subsequent manual analysis.
• The integration of AI outputs with human review in semi-automated workflows offers a practical path forward, especially when combined with open-source tools and shared data repositories like LILA BC.