[论文解读] Eff-3DPSeg: 3D organ-level plant shoot segmentation using annotation-efficient point clouds

Liyi Luo, Xintong Jiang+5 more|arXiv (Cornell University)|Dec 20, 2022

Smart Agriculture and AI被引用 4

一句话总结

该论文提出Eff-3DPSeg，一种弱监督的3D植物茎叶分割框架，通过视角瓶颈损失进行自监督预训练，并仅使用0.5%的标注点进行微调。其在茎和叶分割任务中实现了高精度表现——精确率95.1%、召回率96.6%、mIoU达92.2%，实现了仅需极少标注成本的器官级表型分析。

ABSTRACT

Reliable and automated 3D plant shoot segmentation is a core prerequisite for the extraction of plant phenotypic traits at the organ level. Combining deep learning and point clouds can provide effective ways to address the challenge. However, fully supervised deep learning methods require datasets to be point-wise annotated, which is extremely expensive and time-consuming. In our work, we proposed a novel weakly supervised framework, Eff-3DPSeg, for 3D plant shoot segmentation. First, high-resolution point clouds of soybean were reconstructed using a low-cost photogrammetry system, and the Meshlab-based Plant Annotator was developed for plant point cloud annotation. Second, a weakly-supervised deep learning method was proposed for plant organ segmentation. The method contained: (1) Pretraining a self-supervised network using Viewpoint Bottleneck loss to learn meaningful intrinsic structure representation from the raw point clouds; (2) Fine-tuning the pre-trained model with about only 0.5% points being annotated to implement plant organ segmentation. After, three phenotypic traits (stem diameter, leaf width, and leaf length) were extracted. To test the generality of the proposed method, the public dataset Pheno4D was included in this study. Experimental results showed that the weakly-supervised network obtained similar segmentation performance compared with the fully-supervised setting. Our method achieved 95.1%, 96.6%, 95.8% and 92.2% in the Precision, Recall, F1-score, and mIoU for stem leaf segmentation and 53%, 62.8% and 70.3% in the AP, AP@25, and AP@50 for leaf instance segmentation. This study provides an effective way for characterizing 3D plant architecture, which will become useful for plant breeders to enhance selection processes.

研究动机与目标

• 为降低全监督3D植物茎叶分割在点云数据上的高标注成本。
• 开发一种弱监督深度学习框架，实现在极少人工标注数据下实现高精度分割。
• 实现可靠提取茎直径、叶宽、叶长等器官级表型特征。
• 在Pheno4D等公开数据集上验证方法的泛化能力。
• 为植物育种者提供可扩展的解决方案，以加速表型分析工作流程。

提出的方法

• 使用低成本摄影测量系统重建高分辨率的3D大豆茎叶点云。
• 开发基于Meshlab的Plant Annotator工具，实现点云标注的高效处理。
• 利用自监督损失——视角瓶颈损失（Viewpoint Bottleneck loss），对深度神经网络进行预训练，以从原始点云中学习内在结构表征。
• 仅使用0.5%的逐点标注对预训练模型进行微调，实现器官级分割。
• 将训练好的模型应用于提取三种表型特征：茎直径、叶宽和叶长。
• 在自建大豆数据集和公开的Pheno4D数据集上评估方法的泛化性能。

实验结果

研究问题

• RQ1 在原始3D点云上进行自监督预训练，是否能显著减少植物茎叶分割对人工标注的依赖？
• RQ2 当仅使用0.5%的标注点时，弱监督分割模型的性能与全监督基线相比如何？
• RQ3 所提出方法在不同植物物种和数据集（如Pheno4D）上的泛化能力如何？
• RQ4 该分割模型能否可靠提取关键表型特征，如茎直径和叶尺寸？
• RQ5 视角瓶颈损失是否能在无监督条件下有效捕捉植物点云中的结构语义？

主要发现

• 弱监督模型在茎和叶分割任务中达到95.1%的精确率、96.6%的召回率和92.2%的mIoU，性能接近全监督方法。
• 在叶实例分割任务中，模型达到70.3%的AP@50、62.8%的AP@25和53%的AP，显示出强大的实例级检测能力。
• 该方法在公开的Pheno4D数据集上表现出良好泛化能力，证明其在不同植物结构和成像条件下均具鲁棒性。
• 仅使用0.5%的标注点显著降低了标注成本，同时保持了高分割精度。
• 分割流程成功实现了茎直径、叶宽、叶长等表型特征的提取。
• 采用视角瓶颈损失进行自监督预训练，能有效从原始点云中学习有意义的结构表征，从而提升下游微调效率。