[論文レビュー] Deep Leaf Segmentation Using Synthetic Data

Daniel B. Ward, Peyman Moghadam+1 more|arXiv (Cornell University)|Jul 28, 2018

Smart Agriculture and AI参考文献 11被引用数 73

ひとこと要約

論文は、Leafインスタンスセグメンテーションのための限られた実データを、Blenderベースのパイプラインで生成した合成アラビドプシス葉で拡張し、特に実データと合成データを組み合わせた場合にCVPPP LSCで最先端の結果を達成する。

ABSTRACT

Automated segmentation of individual leaves of a plant in an image is a prerequisite to measure more complex phenotypic traits in high-throughput phenotyping. Applying state-of-the-art machine learning approaches to tackle leaf instance segmentation requires a large amount of manually annotated training data. Currently, the benchmark datasets for leaf segmentation contain only a few hundred labeled training images. In this paper, we propose a framework for leaf instance segmentation by augmenting real plant datasets with generated synthetic images of plants inspired by domain randomisation. We train a state-of-the-art deep learning segmentation architecture (Mask-RCNN) with a combination of real and synthetic images of Arabidopsis plants. Our proposed approach achieves 90% leaf segmentation score on the A1 test set outperforming the-state-of-the-art approaches for the CVPPP Leaf Segmentation Challenge (LSC). Our approach also achieves 81% mean performance over all five test datasets.

研究の動機と目的

• 葉のインスタンスセグメンテーションのラベル付きデータの不足に対処する。
• ドメインランダム化を活用したアラビドプシス葉の合成データ生成パイプラインを提案する。
• 実データ・合成データ・混合データの影響をCVPPP Leaf Segmentation Challengeの結果で評価する。

提案手法

• 実データCVPPP A1と10,000葉の合成植物画像の混合を用いてResNet101バックボーンのMask-RCNNを訓練する。
• Blenderで1枚のインスピレーション葉を設計し、変形・テクスチャリング・葉の配置をしてローゼットを形成することで、合成アラビドプシス植物を生成する。
• テクスチャを varied textures (30 textures) and backgrounds (random plant pot backgrounds) and apply Gaussian blur to blend with backgrounds を用いて合成葉をレンダリングし、背景と自然に馴染ませる。
• 実データと合成データを均等なバッチで訓練し、標準的なデータ拡張（ flips, rotations, zoom, crop）を適用する。
• 特徴抽出器のフリーズを試し、ドメインランダム化アプローチと比較する。

実験結果

リサーチクエスチョン

• RQ1 Synthetic plant images can improve leaf instance segmentation when combined with real data for CVPPP LSC?
• RQ2 How does texture variation and background randomisation in synthetic data affect segmentation performance?
• RQ3 Does freezing pre-trained features interact positively or negatively with synthetic data for deformable plant models?
• RQ4 What are the limits of synthetic data in matching real leaf geometry across species (Arabidopsis vs tobacco)?

主な発見

Training Regime	A1	A2	A3	A4	A5	Mean (if provided)
CVPPP-A1 (real)	87	71	59	73	70	71
Synthetic-plant	81	69	41	84	73	74
Synthetic-leaf	82	74	32	86	75	74
Synthetic-COCO	57	48	25	48	43	44
Synthetic+real	90	81	51	88	82	81

• Training with both real and synthetic data (Synthetic+real) achieves 90.0 SBD on A1, 81.0 on A2, 51.0 on A3, 88 on A4, and 82 on A5, averaging 81 across all five test sets.
• Synthetic data alone generally underperforms compared to real data on A1, A2, A4, but exceeds real data on some sets when texture variation is controlled.
• Per-leaf texture (Synthetic-leaf) improves performance across most test sets except A3, illustrating the importance of texture realism for deformable leaves.
• Randomised textures sampled from COCO (Synthetic-COCO) performs worst, highlighting the need for domain-appropriate textures.
• Mixing real and synthetic data provides the best overall generalisation, improving robustness and reducing false positives from background artifacts (e.g., fiducial markers).
• Freezing the feature extractor when training on synthetic data degrades results in this study, suggesting deformable plant geometry challenges the rigid-object assumption of some domain adaptation approaches.