[论文解读] Choosing an Appropriate Platform and Workflow for Processing Camera Trap Data using Artificial Intelligence

Juliana Vélez, Paula J. Castiblanco-Camacho+4 more|arXiv (Cornell University)|Feb 4, 2022

Species Distribution and Climate Change被引用 4

一句话总结

本文评估了四种AI平台——Wildlife Insights、MegaDetector、MLWIC2 和 Conservation AI——利用深度学习处理相机陷阱数据的表现。研究结果表明，尽管物种分类准确率因召回率低而表现不佳，但像 MegaDetector 这类模型在检测‘动物’存在方面实现了高精度与高召回率，从而能够高效过滤空白图像，并支持半自动工作流程，显著减少生态监测中的手动工作量。

ABSTRACT

Camera traps have transformed how ecologists study wildlife species distributions, activity patterns, and interspecific interactions. Although camera traps provide a cost-effective method for monitoring species, the time required for data processing can limit survey efficiency. Thus, the potential of Artificial Intelligence (AI), specifically Deep Learning (DL), to process camera-trap data has gained considerable attention. Using DL for these applications involves training algorithms, such as Convolutional Neural Networks (CNNs), to automatically detect objects and classify species. To overcome technical challenges associated with training CNNs, several research communities have recently developed platforms that incorporate DL in easy-to-use interfaces. We review key characteristics of four AI-powered platforms -- Wildlife Insights (WI), MegaDetector (MD), Machine Learning for Wildlife Image Classification (MLWIC2), and Conservation AI -- including data management tools and AI features. We also provide R code in an open-source GitBook, to demonstrate how users can evaluate model performance, and incorporate AI output in semi-automated workflows. We found that species classifications from WI and MLWIC2 generally had low recall values (animals that were present in the images often were not classified to the correct species). Yet, the precision of WI and MLWIC2 classifications for some species was high (i.e., when classifications were made, they were generally accurate). MD, which classifies images using broader categories (e.g., "blank" or "animal"), also performed well. Thus, we conclude that, although species classifiers were not accurate enough to automate image processing, DL could be used to improve efficiencies by accepting classifications with high confidence values for certain species or by filtering images containing blanks.

研究动机与目标

• 解决处理大量相机陷阱图像所面临的高人力成本挑战。
• 识别主要AI平台在生态图像分析中的技术与可用性差异。
• 根据用户专业水平和数据需求，提供选择平台和设计工作流程的实际指导。
• 展示如何使用标准指标（如精确率、召回率和F1分数）评估模型性能。
• 推动将AI工具整合到半自动工作流程中，以提升野生动物监测效率。

提出的方法

• 基于软件要求、数据管理和AI能力，评估了四种AI平台——Wildlife Insights、MegaDetector、MLWIC2 和 Conservation AI。
• 使用R代码和真实数据计算并比较了物种分类与目标检测的性能指标：精确率、召回率和F1分数。
• 在MegaDetector中应用置信度阈值调优，以在动物检测的精确率与召回率之间取得平衡。
• 在已知物种存在的1,000张相机陷阱图像数据集上评估了模型性能。
• 将AI输出与人工审核流程整合，构建半自动工作流程。
• 通过GitBook分享开源代码和指南，以支持可复现性与社区使用。

实验结果

研究问题

• RQ1 对于处理相机陷阱数据的生态学家而言，哪些AI平台在可用性、技术要求和性能之间提供了最有效的平衡？
• RQ2 与通用目标检测器（如MegaDetector）相比，Wildlife Insights 和 MLWIC2 中的物种分类器在精确率、召回率和F1分数上的表现有何差异？
• RQ3 AI模型在多大程度上可以减少相机陷阱数据处理中的手动工作量，特别是在过滤空白图像方面？
• RQ4 置信度阈值在优化图像中动物存在的检测性能方面发挥什么作用？
• RQ5 如何将AI输出整合到半自动工作流程中，以在不降低数据质量的前提下提升效率？

主要发现

• Wildlife Insights 和 MLWIC2 中的物种分类器表现出高精确率（例如 >95%），但召回率较低（例如 <65%），意味着尽管分类准确，仍遗漏了大量动物。
• 在0.65置信度阈值下，MegaDetector 对 'Animal' 类别的精确率为98%，召回率为93%，表明其在检测动物方面表现优异。
• 将MegaDetector的置信度阈值从0.65降低至0.1，可使召回率提升至97%，但精确率下降至95%，显示出可根据工作流程需求进行调节的权衡关系。
• 尽管某些物种的精确率较高，但整体物种分类准确率仍不足以实现完全自动化，因此仍需专家审核以捕捉漏检目标。
• AI平台如MegaDetector最有效的用途并非物种级别分类，而是用于过滤空白图像，从而显著减轻后续人工分析的工作量。
• 将AI输出与人工审核相结合的半自动工作流程，为提升效率提供了切实可行的路径，尤其当与开源工具及共享数据仓库（如LILA BC）结合使用时。