[Paper Review] Plant Taxonomy Meets Plant Counting: A Fine-Grained, Taxonomic Dataset for Counting Hundreds of Plant Species

Jinyu Xu, Tianqi Hu+5 more|arXiv (Cornell University)|Mar 22, 2026

Smart Agriculture and AI0 citations

TL;DR

TPC–268 is the first large-scale plant-counting benchmark that integrates full taxonomic hierarchy, enabling taxonomy-aware, class-agnostic counting across multi-scale plant imagery.

ABSTRACT

Visually cataloging and quantifying the natural world requires pushing the boundaries of both detailed visual classification and counting at scale. Despite significant progress, particularly in crowd and traffic analysis, the fine-grained, taxonomy-aware plant counting remains underexplored in vision. In contrast to crowds, plants exhibit nonrigid morphologies and physical appearance variations across growth stages and environments. To fill this gap, we present TPC-268, the first plant counting benchmark incorporating plant taxonomy. Our dataset couples instance-level point annotations with Linnaean labels (kingdom -> species) and organ categories, enabling hierarchical reasoning and species-aware evaluation. The dataset features 10,000 images with 678,050 point annotations, includes 268 countable plant categories over 242 plant species in Plantae and Fungi, and spans observation scales from canopy-level remote sensing imagery to tissue-level microscopy. We follow the problem setting of class-agnostic counting (CAC), provide taxonomy-consistent, scale-aware data splits, and benchmark state-of-the-art regression- and detection-based CAC approaches. By capturing the biodiversity, hierarchical structure, and multi-scale nature of botanical and mycological taxa, TPC-268 provides a biologically grounded testbed to advance fine-grained class-agnostic counting. Dataset and code are available at https://github.com/tiny-smart/TPC-268.

Motivation & Objective

• Motivate counting in the plant domain as a fine-grained, taxonomy-aware problem distinct from traditional crowd/vehicle counting.
• Introduce TPC–268, a large-scale dataset with taxonomic labels and multi-scale imagery for robust counting across growth stages and environments.
• Enable hierarchical reasoning by annotating instances with Linnaean taxonomy (kingdom to species) and organ categories.
• Provide taxonomy-consistent data splits to rigorously evaluate generalization to unseen species within taxonomic gaps.

Proposed method

• Define Pseudo-class-agnostic counting (CAC) for plants and build a dataset that ties counting instances to hierarchical taxonomy.
• Annotate 10,000 images with 678,050 points and 30,000 bounding boxes across 242 species, organized into 268 countable categories.
• Provide 7-dimensional taxonomic vectors per species and auxiliary organ metadata to enable multi-level reasoning.
• Partition data with an MILP-based scheme to ensure taxonomic independence and balanced density across train/val/test sets.
• Benchmark state-of-the-art CAC approaches (regression-based and detection-based) on the new dataset to study generalization across taxonomic and scale variations.
• Explore incorporation of taxonomy information (text prompts) into counting models to assess inductive biases from biological structure.

Experimental results

Research questions

• RQ1 Can existing class-agnostic counting (CAC) models generalize to hundreds of fine-grained plant species when evaluation respects taxonomic hierarchy?
• RQ2 How does incorporating taxonomic and organ-level information affect counting accuracy across scales and densities?
• RQ3 What are the relative strengths of regression-based versus detection-based CAC methods for dense, structurally entangled plant imagery?
• RQ4 Does cross-dataset transfer from generic object counting datasets to plant counting degrade performance, and can plant-specific taxonomy improve robustness?
• RQ5 To what extent do taxonomic priors enable zero-shot or few-shot generalization across related plant taxa?

Key findings

• TPC–268 contains 10,000 images with 678,050 points and 30,000 bounding boxes across 242 species and 268 categories.
• Taxonomy-aware splits (species-organization level) enable rigorous zero-shot counting evaluation across taxonomic gaps.
• Regression-based CAC models generally outperform detection-based approaches on this dataset, with LOCA achieving the best test performance among regulators.
• Incorporating taxonomic information as textual prompts or hierarchical taxonomy improves counting performance, evidencing a practical inductive bias from biological structure.
• Cross-dataset transfer reveals that plant counting is more challenging when trained on FSC–147 and tested on TPC–268, while training on TPC–268 better generalizes to FSC–147 than the opposite direction.
• Fine-grained analyses show counting difficulty is driven by taxonomic and morphological complexity (e.g., Brassicaceae and Poaceae) and scales (microscopic vs macroscopic), not just data quantity.