[Paper Review] Estimates of maize plant density from UAV RGB images using Faster-RCNN detection model: impact of the spatial resolution

Kaaviya Velumani, Raúl López‐Lozano+5 more|arXiv (Cornell University)|May 25, 2021

Remote Sensing in AgricultureEnvironmental Science74 references72 citations

TL;DR

This study evaluates Faster-RCNN for maize plant density estimation from UAV RGB images across varying ground sampling distances (GSD), demonstrating that model performance degrades significantly when trained on high-resolution (GSD ≈ 0.3 cm) images and applied to low-resolution (GSD ≈ 0.6 cm) data. The key contribution is that training on mixed-resolution data or using GAN-based super-resolution improves low-resolution detection, though native low-resolution images remain problematic due to poor image quality.

ABSTRACT

Early-stage plant density is an essential trait that determines the fate of a genotype under given environmental conditions and management practices. The use of RGB images taken from UAVs may replace traditional visual counting in fields with improved throughput, accuracy and access to plant localization. However, high-resolution (HR) images are required to detect small plants present at early stages. This study explores the impact of image ground sampling distance (GSD) on the performances of maize plant detection at 3-5 leaves stage using Faster-RCNN. Data collected at HR (GSD=0.3cm) over 6 contrasted sites were used for model training. Two additional sites with images acquired both at high and low (GSD=0.6cm) resolution were used for model evaluation. Results show that Faster-RCNN achieved very good plant detection and counting (rRMSE=0.08) performances when native HR images are used both for training and validation. Similarly, good performances were observed (rRMSE=0.11) when the model is trained over synthetic low-resolution (LR) images obtained by down-sampling the native training HR images, and applied to the synthetic LR validation images. Conversely, poor performances are obtained when the model is trained on a given spatial resolution and applied to another spatial resolution. Training on a mix of HR and LR images allows to get very good performances on the native HR (rRMSE=0.06) and synthetic LR (rRMSE=0.10) images. However, very low performances are still observed over the native LR images (rRMSE=0.48), mainly due to the poor quality of the native LR images. Finally, an advanced super-resolution method based on GAN (generative adversarial network) that introduces additional textural information derived from the native HR images was applied to the native LR validation images. Results show some significant improvement (rRMSE=0.22) compared to bicubic up-sampling approach.

Motivation & Objective

• To assess the accuracy and robustness of Faster-RCNN for maize plant detection using high-resolution UAV RGB images (GSD ≈ 0.3 cm).
• To evaluate the generalization capacity of the model when applied to images with different spatial resolutions than used in training.
• To investigate the effectiveness of data augmentation techniques—specifically down-sampling and super-resolution—in improving detection performance across resolution domains.
• To compare the performance of bicubic up-sampling versus GAN-based super-resolution (Cycle-ESRGAN) for enhancing low-resolution validation images.

Proposed method

• Faster-RCNN object detection model is trained and validated on UAV RGB images collected at multiple sites across France with varying flight altitudes.
• High-resolution images (GSD ≈ 0.3 cm) from six contrasted sites are used for training, while two additional sites provide both high- and low-resolution (GSD ≈ 0.6 cm) images for evaluation.
• Synthetic low-resolution images are generated by down-sampling the native high-resolution training images to simulate real low-resolution data.
• A GAN-based super-resolution model (Cycle-ESRGAN) is applied to native low-resolution validation images to enhance texture and improve detection features.
• Performance is evaluated using rRMSE, accuracy (Ac), and average precision (AP) across different training and validation configurations.
• The model is also fine-tuned with combined high- and low-resolution data to assess generalization and robustness across resolution domains.

Experimental results

Research questions

• RQ1 How does ground sampling distance (GSD) affect the performance of Faster-RCNN in detecting maize plants at the three-to-five-leaf stage?
• RQ2 Can a model trained on high-resolution images generalize effectively to low-resolution images acquired at higher altitudes?
• RQ3 To what extent do data augmentation techniques such as down-sampling and super-resolution improve detection performance on low-resolution images?
• RQ4 How do bicubic up-sampling and GAN-based super-resolution compare in enhancing low-resolution images for maize plant detection?
• RQ5 What role does image quality—particularly texture loss and compression artifacts—play in limiting detection performance on native low-resolution images?

Key findings

• Faster-RCNN achieved excellent performance with rRMSE = 0.08 when trained and validated on native high-resolution images (GSD ≈ 0.3 cm).
• The model maintained good performance (rRMSE = 0.11) when trained on synthetic low-resolution images (down-sampled from high-res) and applied to synthetic low-resolution validation images.
• Performance degraded significantly (rRMSE = 0.48) when the model trained on high-resolution data was applied to native low-resolution images, primarily due to poor image quality and lack of texture.
• Training on a mixed dataset of high- and low-resolution images yielded strong results: rRMSE = 0.06 on native high-resolution and rRMSE = 0.10 on synthetic low-resolution images.
• GAN-based super-resolution (Cycle-ESRGAN) improved detection on native low-resolution images (rRMSE = 0.22) compared to bicubic up-sampling (rRMSE = 0.48), though performance remained far below that of high-resolution data.
• Super-resolution images introduced artifacts such as false green pixels in shadows or background features, increasing false positives in some cases, indicating limitations in realism and generalization.