[Paper Review] Plant identification in an open-world (LifeCLEF 2016)

Hervé Goëau, Pierre Bonnet+1 more|ArXiv.org|Sep 25, 2025

Smart Agriculture and AI9 references53 citations

TL;DR

The LifeCLEF 2016 plant identification task evaluated open-set recognition on over 110k images across 1000 Western European plant species, comparing CNN-based systems and highlighting challenges in rejecting unknown classes.

ABSTRACT

The LifeCLEF plant identification challenge aims at evaluating plant identification methods and systems at a very large scale, close to the conditions of a real-world biodiversity monitoring scenario. The 2016-th edition was actually conducted on a set of more than 110K images illustrating 1000 plant species living in West Europe, built through a large-scale participatory sensing platform initiated in 2011 and which now involves tens of thousands of contributors. The main novelty over the previous years is that the identification task was evaluated as an open-set recognition problem, i.e. a problem in which the recognition system has to be robust to unknown and never seen categories. Beyond the brute-force classification across the known classes of the training set, the big challenge was thus to automatically reject the false positive classification hits that are caused by the unknown classes. This overview presents more precisely the resources and assessments of the challenge, summarizes the approaches and systems employed by the participating research groups, and provides an analysis of the main outcomes.

Motivation & Objective

• Evaluate plant identification methods at large scale under open-set conditions close to real-world biodiversity monitoring.
• Assess robustness to unknown and unseen plant categories while identifying known species.
• Provide a benchmark dataset and metrics to study open-set performance and unknown-class rejection.
• Analyze how different CNN-based and hybrid approaches perform under distractor-rich test sets.

Proposed method

• Use a training set from PlantCLEF 2015 enriched with ground truth for test images.
• Construct a test set from Pl@ntNet queries including known and unknown classes (open-set).
• Evaluate submissions with a mean Average Precision in an open-set setting (mAP-open) and a variant focusing on invasive species (mAP-open-invasive).
• Allow up to 4 runs per group with CNN and non-CNN baselines, including ensembles and meta-data usage.
• Assess rejection strategies for unknown classes and report performance under varying novelty levels.

Experimental results

Research questions

• RQ1 How well do CNN-based plant identification systems perform in an open-world setting with many unknown classes?
• RQ2 What impact do unknown-class distractors have on mAP in open-set plant identification?
• RQ3 Do explicit unknown-class rejection strategies improve robustness, and under what novelty conditions?
• RQ4 How does performance degrade as the proportion of unknown queries increases in a streaming-like scenario?
• RQ5 What is the relative contribution of architecture, ensembles, and meta-data in open-set plant identification performance?

Key findings

• CNN-based systems dominated the top results, with the top 26 runs using CNNs.
• Best configuration achieved mAP-open 0.718 for invasive-species monitoring, with gains mainly from observation-level pooling.
• Open-set distractors degrade performance across all systems; however, CNNs remain relatively robust to unknown classes.
• When novelty is high, mean average precision drops significantly (e.g., below 0.45 when only 25% of queries are known).
• Rejection strategies provided limited additional benefits over CNN baselines under moderate novelty, suggesting room for adaptive open-set rejection methods.