[Paper Review] Agricultural Plant Cataloging and Establishment of a Data Framework from UAV-based Crop Images by Computer Vision

Maurice Günder, Facundo Ramón Ispizua Yamati+4 more|arXiv (Cornell University)|Jan 8, 2022

Smart Agriculture and AI38 references29 citations

TL;DR

This paper presents a computer vision-based workflow for automated temporal and spatial cataloging of individual plants from UAV-based crop images, enabling large-scale spatio-temporal analysis. By leveraging RGB image processing, peak detection, and geometric alignment, the method achieves high accuracy in plant identification—comparable to deep learning—while supporting downstream tasks like disease severity classification with minimal manual effort.

ABSTRACT

UAV-based image retrieval in modern agriculture enables gathering large amounts of spatially referenced crop image data. In large-scale experiments, however, UAV images suffer from containing a multitudinous amount of crops in a complex canopy architecture. Especially for the observation of temporal effects, this complicates the recognition of individual plants over several images and the extraction of relevant information tremendously. In this work, we present a hands-on workflow for the automatized temporal and spatial identification and individualization of crop images from UAVs abbreviated as "cataloging" based on comprehensible computer vision methods. We evaluate the workflow on two real-world datasets. One dataset is recorded for observation of Cercospora leaf spot - a fungal disease - in sugar beet over an entire growing cycle. The other one deals with harvest prediction of cauliflower plants. The plant catalog is utilized for the extraction of single plant images seen over multiple time points. This gathers large-scale spatio-temporal image dataset that in turn can be applied to train further machine learning models including various data layers. The presented approach improves analysis and interpretation of UAV data in agriculture significantly. By validation with some reference data, our method shows an accuracy that is similar to more complex deep learning-based recognition techniques. Our workflow is able to automatize plant cataloging and training image extraction, especially for large datasets.

Motivation & Objective

• Address the challenge of identifying and tracking individual plants across multiple UAV images in dense crop canopies.
• Develop a fully automated, scalable workflow for plant cataloging that avoids reliance on expensive sensors or manual annotation.
• Enable the creation of large-scale, time-resolved image datasets for training machine learning models in precision agriculture.
• Support applications such as disease progression analysis and phenotypic trait extraction by providing consistent, georeferenced plant data.
• Demonstrate the method’s robustness across diverse crop types, including sugar beet and cauliflower, under real-world field conditions.

Proposed method

• Process multi-temporal, multi-spectral UAV images using plant-soil segmentation to isolate individual plants.
• Apply adaptive Gaussian blur filtering to enhance peak detection for plant center localization in RGB images.
• Group detected plant peaks across time points using spatial proximity and temporal consistency, enabling temporal tracking.
• Integrate ground control points (GCPs) for georeferencing to ensure sub-centimeter accuracy in global coordinates (WGS84).
• Use seeding line detection and geometric alignment to refine plant cataloging and reduce false positives.
• Extract time-series image patches for individual plants, enabling downstream analysis and model training.

Experimental results

Research questions

• RQ1 Can a computer vision-based workflow achieve accurate, automated plant cataloging from UAV RGB images without requiring deep learning or specialized sensors?
• RQ2 How effective is the proposed method in tracking individual plants across multiple time points in dense, complex crop canopies?
• RQ3 To what extent can the resulting plant catalog support downstream machine learning tasks such as disease severity classification?
• RQ4 How does the performance of the proposed method compare to more complex deep learning-based approaches in terms of accuracy and scalability?
• RQ5 Can the framework be generalized across different crop types and field conditions, including those with high weed density or closed canopies?

Key findings

• The proposed workflow achieves plant detection accuracy comparable to complex deep learning methods, demonstrating its viability for large-scale applications.
• The method successfully enables the extraction of consistent, time-resolved image patches for individual plants across multiple UAV overpasses.
• A trained ResNet-50 model using extracted plant images achieved a confusion matrix indicating strong classification performance, with 90.7% of predictions correctly assigned to the true disease severity class 1 (completely healthy).
• The framework supports high-throughput data generation, enabling the creation of large-scale, annotated datasets for training robust machine learning models.
• The use of only RGB images for peak detection proves sufficient for reliable plant cataloging, reducing hardware and data acquisition barriers.
• The workflow is highly automatable and scalable, with parallel processing feasible across multiple fields or sub-regions, as demonstrated in the sugar beet dataset with many small plots.