[Paper Review] Plant identification based on noisy web data: the amazing performance of deep learning (LifeCLEF 2017)

Hervé Goëau, Pierre Bonnet+1 more|ArXiv.org|Sep 25, 2025

Smart Agriculture and AI6 references55 citations

TL;DR

The paper analyzes LifeCLEF 2017 plant identification using trusted (EOL) versus noisy web data, showing CNNs trained on noisy data perform very well and ensembles outperform single models.

ABSTRACT

The 2017-th edition of the LifeCLEF plant identification challenge is an important milestone towards automated plant identification systems working at the scale of continental floras with 10.000 plant species living mainly in Europe and North America illustrated by a total of 1.1M images. Nowadays, such ambitious systems are enabled thanks to the conjunction of the dazzling recent progress in image classification with deep learning and several outstanding international initiatives, such as the Encyclopedia of Life (EOL), aggregating the visual knowledge on plant species coming from the main national botany institutes. However, despite all these efforts the majority of the plant species still remain without pictures or are poorly illustrated. Outside the institutional channels, a much larger number of plant pictures are available and spread on the web through botanist blogs, plant lovers web-pages, image hosting websites and on-line plant retailers. The LifeCLEF 2017 plant challenge presented in this paper aimed at evaluating to what extent a large noisy training dataset collected through the web and containing a lot of labelling errors can compete with a smaller but trusted training dataset checked by experts. To fairly compare both training strategies, the test dataset was created from a third data source, i.e. the Pl@ntNet mobile application that collects millions of plant image queries all over the world. This paper presents more precisely the resources and assessments of the challenge, summarizes the approaches and systems employed by the participating research groups, and provides an analysis of the main outcomes.

Motivation & Objective

• Assess whether noisy web data can match or exceed trusted expert-labeled data for large-scale plant ID.
• Evaluate CNN-based approaches on a 10K-species plant dataset with both trusted and noisy training sets.
• Compare different training strategies and architectures to understand data quality vs. model complexity in plant identification.

Proposed method

• Construct three datasets: EOL10K (trusted, 256,287 images), Web10K (noisy, 1.1M images), and Pl@ntNet test set.
• Evaluate up to 4 runs per group using CNNs trained from scratch or fine-tuned, including ensembles and data augmentation.
• Use Mean Reciprocal Rank (MRR) as the primary evaluation metric on the Pl@ntNet test set.
• Experiment with various architectures (GoogLeNet/Inception variants, ResNet variants, VGGNet, AlexNet) and training tricks (bagging, pretraining on ImageNet, bootstrapping, filtering of noisy data).
• Analyze results across organ-based test subsets and long-tail species distribution to assess robustness and biodiversity-friendly performance.
• Discuss potential efficiency improvements (knowledge distillation) for deployment in large-scale, real-time apps.

Experimental results

Research questions

• RQ1 Can very large, noisy web-trained CNNs achieve competitive performance compared to trusted datasets in large-scale plant identification?
• RQ2 Do ensemble methods and data augmentation compensate for label noise and class imbalance in Web10K?
• RQ3 What is the impact of training data source (trusted vs noisy) on MRR across different plant organs and long-tail species?
• RQ4 Is filtering noisy labels beneficial or detrimental to final performance in this task?

Key findings

• CNN-based systems achieve high performance with a median MRR around 0.8 and a top result of 0.92.
• Best results come from ensembles trained on both trusted and noisy data rather than a single data source.
• Some runs trained only on noisy data outperform many trusted-data-only methods, highlighting the value of data diversity.
• Filtering noisy data often degrades performance compared to using the full noisy dataset (no filtration).
• Ensembles (e.g., 60 model distributions in Mario TSA Berlin Run 4) outperform single models, and newer architectures (Inception-ResNet-v2, Inception-v4) contribute to gains when combined with bagging and augmentation.
• Data augmentation and bootstrapping are key to achieving high accuracy in the presence of label noise and class imbalance.
• None of the top results relied solely on a single architecture; rather, combining multiple CNNs and training strategies yielded the best performance.
• The study suggests that training on noisy data acts as a form of regularization, enhancing generalization in biodiversity informatics contexts.
• Computational demands are high for state-of-the-art ensemble approaches, motivating exploration of knowledge distillation for deployment.