[Paper Review] Deep Learning Pipeline for Automated Visual Moth Monitoring: Insect Localization and Species Classification

Dimitri Korsch, Paul Bodesheim+1 more|arXiv (Cornell University)|Jan 1, 2021

Species Distribution and Climate Change10 citations

TL;DR

This paper presents a deep learning pipeline for automated visual monitoring of 200 Central European moth species using a moth scanner. It combines a single-shot MultiBox detector (SSD) for insect localization and a fine-tuned CNN with unsupervised part estimation for species classification, improving overall classification accuracy from 79.62% to 88.05% on uncropped scanner images.

ABSTRACT

Biodiversity monitoring is crucial for tracking and counteracting adverse trends in population fluctuations. However, automatic recognition systems are rarely applied so far, and experts evaluate the generated data masses manually. Especially the support of deep learning methods for visual monitoring is not yet established in biodiversity research, compared to other areas like advertising or entertainment. In this paper, we present a deep learning pipeline for analyzing images captured by a moth scanner, an automated visual monitoring system of moth species developed within the AMMOD project. We first localize individuals with a moth detector and afterward determine the species of detected insects with a classifier. Our detector achieves up to 99.01% mean average precision and our classifier distinguishes 200 moth species with an accuracy of 93.13% on image cutouts depicting single insects. Combining both in our pipeline improves the accuracy for species identification in images of the moth scanner from 79.62% to 88.05%.

Motivation & Objective

• Address the challenge of labor-intensive manual insect monitoring in biodiversity research.
• Develop an automated pipeline for visual moth monitoring using deep learning to reduce reliance on expert labor.
• Improve species classification accuracy in real-world images where multiple moths and small object scales are common.
• Evaluate the impact of different training strategies, pre-training datasets, and part-based modeling on classification performance.
• Demonstrate the effectiveness of a two-stage pipeline combining detection and classification for real-world deployment in ecological monitoring systems.

Proposed method

• Employ a single-shot MultiBox detector (SSD) for object localization, trained on a custom dataset of moth images captured by the AMMOD moth scanner.
• Use mean average precision (mAP) with IoU thresholds of 0.5 and 0.75 to evaluate detection performance.
• Train a convolutional neural network (CNN) for species classification on cropped images of single moths, using transfer learning from ImageNet and iNaturalist pre-trained models.
• Implement a fine-tuning strategy that either freezes the feature extractor or fine-tunes the entire network.
• Integrate an unsupervised part estimator into the classifier to improve feature representation and generalization.
• Combine the trained detector and classifier into a pipeline, applying detection first to localize moths before classifying each detected insect in uncropped scanner images.

Experimental results

Research questions

• RQ1 Can a deep learning-based object detector achieve high mAP for moth localization in real-world, cluttered scanner images?
• RQ2 How does the choice of pre-training dataset (ImageNet vs. iNaturalist) affect species classification accuracy on single-moth images?
• RQ3 To what extent does fine-tuning the entire CNN versus only the final fully connected layer improve classification performance?
• RQ4 Does incorporating unsupervised part estimation enhance classification accuracy, especially on challenging or rare species?
• RQ5 How much does a preceding detection step improve classification accuracy on uncropped, multi-moth images compared to direct classification on full images?

Key findings

• The SSD-based moth detector achieves a mean average precision (mAP@0.5) of 99.01% and mAP@0.75 of 88.88% on the test set.
• The CNN classifier reaches 93.13% accuracy when fine-tuned on the entire network and pre-trained on iNaturalist, with unsupervised part estimation.
• Fine-tuning the entire CNN instead of only the final layer improves classification accuracy by 26% compared to using only a feature extractor.
• Pre-training on iNaturalist instead of ImageNet increases classification accuracy by approximately 4% when fine-tuning the full network.
• The integration of unsupervised part estimation boosts classification accuracy by 2.6 percentage points when using iNaturalist pre-training.
• The full pipeline improves classification accuracy on uncropped scanner images from 79.62% (classifier-only) to 88.05% (detector + classifier), demonstrating the value of localization in complex scenes.