[Paper Review] Leaf segmentation through the classification of edges

Jonathan Bell, Hannah Dee|arXiv (Cornell University)|Apr 5, 2019

Leaf Properties and Growth MeasurementAgricultural and Biological Sciences18 references17 citations

TL;DR

This paper proposes a novel edge classification approach using a shallow convolutional neural network (CNN) to segment leaves in top-down images of Arabidopsis thaliana plants, enabling accurate leaf counting and area estimation even under heavy occlusion. By classifying Canny-detected edges into background, plant edge, leaf-on-leaf edge, or internal noise, the method achieves robust region-based segmentation that outperforms existing techniques on occluded rosette plants.

ABSTRACT

We present an approach to leaf level segmentation of images of Arabidopsis thaliana plants based upon detected edges. We introduce a novel approach to edge classification, which forms an important part of a method to both count the leaves and establish the leaf area of a growing plant from images obtained in a high-throughput phenotyping system. Our technique uses a relatively shallow convolutional neural network to classify image edges as background, plant edge, leaf-on-leaf edge or internal leaf noise. The edges themselves were found using the Canny edge detector and the classified edges can be used with simple image processing techniques to generate a region-based segmentation in which the leaves are distinct. This approach is strong at distinguishing occluding pairs of leaves where one leaf is largely hidden, a situation which has proved troublesome for plant image analysis systems in the past. In addition, we introduce the publicly available plant image dataset that was used for this work.

Motivation & Objective

• Address the challenge of leaf-level segmentation in high-throughput phenotyping, particularly under leaf-on-leaf occlusion common in rosette plants.
• Improve upon whole-plant segmentation by enabling individual leaf counting and area estimation, which are critical for dynamic growth analysis.
• Develop a method that is robust to low-resolution, out-of-focus, and distorted images typical in automated phenotyping setups.
• Create and release a publicly available, high-quality dataset to support future research in plant image analysis and deep learning.
• Demonstrate transferability of the trained model across different datasets, including the LSC A2 benchmark, to validate generalization potential.

Proposed method

• Detect edges in top-down plant images using the Canny edge detector as the initial input to the segmentation pipeline.
• Train a shallow convolutional neural network to classify each detected edge into one of four categories: background, plant edge, leaf-on-leaf edge, or internal leaf noise.
• Use the classified edges as input to a region-growing or morphological segmentation algorithm to generate distinct leaf regions.
• Leverage the edge classification to resolve ambiguities in overlapping leaves, especially in occluded regions where traditional methods fail.
• Apply the trained CNN to new images without retraining, enabling deployment in high-throughput phenotyping systems with minimal user interaction.
• Ensure scalability and automation by avoiding human-in-the-loop processing, relying solely on trained models and image processing post-classification.

Experimental results

Research questions

• RQ1 Can edge classification using a shallow CNN improve leaf segmentation accuracy in occluded rosette plants compared to traditional whole-plant segmentation?
• RQ2 How well does the edge classification model generalize across different imaging setups and datasets, such as the LSC A2 benchmark?
• RQ3 To what extent can the method handle low-resolution, out-of-focus, and distorted images typical in high-throughput phenotyping platforms?
• RQ4 Does the use of a publicly available, annotated dataset (ALED) enable better model generalization and community adoption in plant image analysis?
• RQ5 Can the method achieve high accuracy in leaf counting and segmentation without relying on prior knowledge of leaf shape or geometric models?

Key findings

• The proposed edge classification method achieves strong performance in segmenting leaves even under heavy occlusion, particularly in leaf-on-leaf overlap regions.
• The model trained on the ALED dataset generalizes reasonably well to the LSC A2 benchmark dataset, indicating cross-dataset transferability.
• Leaf counting results are competitive with or better than those reported in the 2015 Leaf Segmentation Challenge, demonstrating improved accuracy.
• The method shows robustness to image quality variations such as low resolution (5 MP), slight defocus, and barrel distortion, common in automated phenotyping systems.
• The use of a shallow CNN enables efficient inference, supporting high-throughput automation without requiring high-end computational resources.
• The release of the ALED dataset with detailed annotations is expected to accelerate progress in plant image analysis and deep learning for phenotyping.