[Paper Review] Performance Evaluation of YOLOv8 Model Configurations, for Instance Segmentation of Strawberry Fruit Development Stages in an Open Field Environment

Abdul-Razak Alhassan Gamani, Ibrahim Arhin+1 more|arXiv (Cornell University)|Aug 11, 2024

Horticultural and Viticultural ResearchAgricultural and Biological Sciences3 citations

TL;DR

This study evaluates YOLOv8 model configurations for instance segmentation of ripe and unripe strawberry fruits in open-field conditions, using a custom dataset. The YOLOv8n variant achieved the highest mAP of 80.9% and fastest inference speed of 24.2 ms, outperforming larger variants in both accuracy and speed for real-time field applications.

ABSTRACT

Accurate identification of strawberries during their maturing stages is crucial for optimizing yield management, and pest control, and making informed decisions related to harvest and post-harvest logistics. This study evaluates the performance of YOLOv8 model configurations for instance segmentation of strawberries into ripe and unripe stages in an open field environment. The YOLOv8n model demonstrated superior segmentation accuracy with a mean Average Precision (mAP) of 80.9\%, outperforming other YOLOv8 configurations. In terms of inference speed, YOLOv8n processed images at 12.9 milliseconds, while YOLOv8s, the least-performing model, processed at 22.2 milliseconds. Over 86 test images with 348 ground truth labels, YOLOv8n detected 235 ripe fruit classes and 51 unripe fruit classes out of 251 ground truth ripe fruits and 97 unripe ground truth labels, respectively. In comparison, YOLOv8s detected 204 ripe fruits and 37 unripe fruits. Overall, YOLOv8n achieved the fastest inference speed of 24.2 milliseconds, outperforming YOLOv8s, YOLOv8m, YOLOv8l, and YOLOv8x, which processed images at 33.0 milliseconds, 44.3 milliseconds, 53.6 milliseconds, and 62.5 milliseconds, respectively. These results underscore the potential of advanced object segmentation algorithms to address complex visual recognition tasks in open-field agriculture effectively to address complex visual recognition tasks in open-field agriculture effectively.

Motivation & Objective

• To evaluate the performance of multiple YOLOv8 model configurations for instance segmentation of strawberry fruit development stages in open-field environments.
• To identify the optimal YOLOv8 configuration balancing accuracy and inference speed for real-time agricultural applications.
• To support automated yield estimation and harvest decision-making by accurately detecting ripe and unripe strawberries.
• To address challenges in field conditions such as occlusions, variable lighting, and complex backgrounds using deep learning.

Proposed method

• The study employed YOLOv8n, YOLOv8s, YOLOv8m, YOLOv8l, and YOLOv8x models for instance segmentation of strawberry fruits.
• A dataset of 240 images with 348 ground truth labels (251 ripe, 97 unripe) was used for training and evaluation.
• Model performance was evaluated using mAP at IoU threshold of 0.5, along with inference speed, precision, recall, and F1-score metrics.
• Performance was visualized using Precision-Recall, F1-Confidence, and Recall-Confidence curves to assess detection reliability.
• The YOLOv8n model was selected for its optimal balance of speed and accuracy, validated through quantitative comparison across all configurations.
• Inference speed was measured in milliseconds per image across all model variants under identical conditions.

Figure 1 : Block diagram illustrating the strawberry instance segmentation workflow and yield estimation

Experimental results

Research questions

• RQ1 Which YOLOv8 model configuration achieves the highest instance segmentation accuracy for ripe and unripe strawberry fruits in open-field conditions?
• RQ2 How do different YOLOv8 model sizes compare in terms of inference speed and real-time feasibility for field deployment?
• RQ3 What is the impact of model architecture on detection recall and precision for small, occluded, or partially visible strawberry fruits?
• RQ4 Can YOLOv8n effectively generalize to real-world open-field environments with complex backgrounds and variable lighting?

Key findings

• YOLOv8n achieved the highest mean Average Precision (mAP) of 80.9% across all classes, outperforming all other YOLOv8 configurations.
• YOLOv8n demonstrated the fastest inference speed at 24.2 milliseconds per image, significantly outperforming YOLOv8s (33.0 ms), YOLOv8m (44.3 ms), YOLOv8l (53.6 ms), and YOLOv8x (62.5 ms).
• YOLOv8n detected 235 ripe and 51 unripe fruit instances out of 251 and 97 ground truth labels, respectively, showing high detection recall.
• YOLOv8s, the least accurate model, detected only 204 ripe and 37 unripe fruits, indicating a significant performance gap.
• The Recall-Confidence curve showed 93% recall at the lowest confidence threshold, indicating strong detection sensitivity.
• The F1-Confidence curve indicated an average precision of 78% at a confidence threshold of 0.524, confirming robust balance between precision and recall.

Figure 2 : ripe and unripe fruits