[Paper Review] Potato Crop Stress Identification in Aerial Images using Deep Learning-based Object Detection

Sujata Butte, Aleksandar Vakanski+3 more|arXiv (Cornell University)|Jun 14, 2021

Smart Agriculture and AIAgricultural and Biological Sciences82 references64 citations

TL;DR

This paper proposes Retina-UNet-Ag, a deep learning-based object detection model that integrates low-level semantic dense features into a feature pyramid network to identify drought-stressed Russet Burbank potato plants in low-altitude aerial RGB and multispectral images. The method achieves a mean Dice Score Coefficient of 0.74 on test images, demonstrating robust performance in detecting plant stress at the individual plant level under real field conditions.

ABSTRACT

Recent research on the application of remote sensing and deep learning-based analysis in precision agriculture demonstrated a potential for improved crop management and reduced environmental impacts of agricultural production. Despite the promising results, the practical relevance of these technologies for field deployment requires novel algorithms that are customized for analysis of agricultural images and robust to implementation on natural field imagery. The paper presents an approach for analyzing aerial images of a potato (Solanum tuberosum L.) crop using deep neural networks. The main objective is to demonstrate automated spatial recognition of healthy vs. stressed crop at a plant level. Specifically, we examine premature plant senescence resulting in drought stress on Russet Burbank potato plants. We propose a novel deep learning (DL) model for detecting crop stress, named Retina-UNet-Ag. The proposed architecture is a variant of Retina-UNet and includes connections from low-level semantic representation maps to the feature pyramid network. The paper also introduces a dataset of aerial field images acquired with a Parrot Sequoia camera. The dataset includes manually annotated bounding boxes of healthy and stressed plant regions. Experimental validation demonstrated the ability for distinguishing healthy and stressed plants in field images, achieving an average dice score coefficient (DSC) of 0.74. A comparison to related state-of-the-art DL models for object detection revealed that the presented approach is effective for this task. The proposed method is conducive toward the assessment and recognition of potato crop stress in aerial field images collected under natural conditions.

Motivation & Objective

• To develop a deep learning model capable of detecting and localizing stressed potato plants in real-world aerial imagery.
• To address challenges in field image analysis, including plant overlap, variable lighting, and lack of clear boundaries in dense canopies.
• To create and release a publicly available dataset of labeled aerial images for potato crop stress detection.
• To improve the generalization and robustness of object detection models for agricultural applications beyond traditional image classification.

Proposed method

• Proposes Retina-UNet-Ag, a variant of Retina-UNet that injects low-level semantic dense feature maps into the feature pyramid network (FPN) for enhanced spatial localization.
• Utilizes a Parrot Sequoia multispectral UAV to collect high-resolution RGB and multispectral images at 3 meters altitude over potato fields.
• Applies instance segmentation-style bounding box annotations for healthy and stressed plants, accounting for overlapping plants and mixed-leaf conditions.
• Trains the model using a combination of RGB and spectral band composites (R-G-NIR, R-NIR-RE, R-G-RE) to evaluate performance across image types.
• Employs transfer learning with ImageNet-pretrained backbones (e.g., ResNet) and fine-tunes on the custom potato dataset.
• Evaluates performance using standard object detection metrics: Dice Score Coefficient (DSC), Intersection over Union (IoU), Precision, and Recall.

Experimental results

Research questions

• RQ1 Can a deep learning-based object detection model accurately identify and localize individual potato plants under drought stress in real aerial field images?
• RQ2 How does the integration of low-level semantic features into the feature pyramid network improve detection performance for plant stress in complex field scenes?
• RQ3 How does model performance vary between high-resolution RGB and lower-resolution multispectral images captured from the same UAV platform?
• RQ4 To what extent does the proposed model generalize to natural field conditions with overlapping plants, variable lighting, and mixed-healthy/stressed leaves?
• RQ5 Can the model achieve high precision and recall in distinguishing stressed from healthy potato plants despite the absence of clear visual boundaries?

Key findings

• Retina-UNet-Ag achieved a mean Dice Score Coefficient (DSC) of 0.74 on the test set, with 0.723 for healthy plants and 0.756 for stressed plants.
• The model outperformed four state-of-the-art object detection models (Mask R-CNN, RetinaNet, Faster R-CNN, YOLO v3) across all metrics, especially in recall and precision for stressed plants.
• Performance on multispectral images was lower than on RGB images, with a DSC of 0.536 for R-G-NIR and 0.588 for R-NIR-RE, attributed to lower resolution (960×1280 vs. 3456×4608 pixels).
• The model successfully detected and localized stressed plants in nearly all test images, even in cases of overlapping plants and mixed-leaf conditions.
• The study introduces and publicly releases a new dataset of 100+ labeled aerial images of potato crops under drought stress, supporting future research.
• The results demonstrate that incorporating low-level semantic features improves detection robustness in complex agricultural scenes with poor boundary definition.