[论文解读] High-Throughput Phenotyping using Computer Vision and Machine Learning

Vivaan Singhvi, Langalibalele Lunga+6 more|arXiv (Cornell University)|Jan 1, 2023

Cell Image Analysis Techniques被引用 1

一句话总结

本研究开发了一套基于计算机视觉与机器学习的高通量表型分析流程，用于杨树（Populus trichocarpa）叶片的表型分析，利用标注图像实现标签提取的OCR准确率达94.31%，叶片形态分类的平均准确率为62.82%，干旱处理预测准确率为60.08%。该方法实现了自动化数据采集与表型分析，展现出在农业领域可扩展植物表型组学的强大潜力。

ABSTRACT

High-throughput phenotyping refers to the non-destructive and efficient evaluation of plant phenotypes. In recent years, it has been coupled with machine learning in order to improve the process of phenotyping plants by increasing efficiency in handling large datasets and developing methods for the extraction of specific traits. Previous studies have developed methods to advance these challenges through the application of deep neural networks in tandem with automated cameras; however, the datasets being studied often excluded physical labels. In this study, we used a dataset provided by Oak Ridge National Laboratory with 1,672 images of Populus Trichocarpa with white labels displaying treatment (control or drought), block, row, position, and genotype. Optical character recognition (OCR) was used to read these labels on the plants, image segmentation techniques in conjunction with machine learning algorithms were used for morphological classifications, machine learning models were used to predict treatment based on those classifications, and analyzed encoded EXIF tags were used for the purpose of finding leaf size and correlations between phenotypes. We found that our OCR model had an accuracy of 94.31% for non-null text extractions, allowing for the information to be accurately placed in a spreadsheet. Our classification models identified leaf shape, color, and level of brown splotches with an average accuracy of 62.82%, and plant treatment with an accuracy of 60.08%. Finally, we identified a few crucial pieces of information absent from the EXIF tags that prevented the assessment of the leaf size. There was also missing information that prevented the assessment of correlations between phenotypes and conditions. However, future studies could improve upon this to allow for the assessment of these features.

研究动机与目标

• 通过OCR技术自动提取植物图像中物理白色标签上的处理信息、基因型及实验元数据。
• 利用计算机视觉与机器学习技术，对叶片形态特征（形状、颜色、褐斑）进行分类。
• 基于形态分类结果，预测植物处理（干旱或对照）。
• 评估EXIF元数据是否可用于估算叶片大小或实现表型-环境相关性分析。
• 评估轻量化、预训练模型在低资源条件下实现可扩展表型分析流程的可行性。

提出的方法

• 采用PaddleOCR从植物图像中的白色标签中稳健提取文本，即使在旋转或部分遮挡条件下也能保持性能。
• 使用Segment Anything Model（SAM）实现零样本、高精度的叶片实例分割，无需模型微调。
• 采用预训练分类器（包括随机森林）基于分割图像特征对叶片形态进行分类。
• 提取并分析EXIF元数据（包括相机位置、焦距和方向），以推断叶片大小与空间上下文信息。
• 将OCR提取的标签与图像衍生的形态特征相结合，训练处理预测模型。
• 使用来自橡树岭国家实验室的1,672张标注图像数据集对结果进行验证，其元数据为真实值。

实验结果

研究问题

• RQ1 OCR能否准确从植物图像的物理标签中提取处理、区块、行、位置及基因型信息？
• RQ2 机器学习模型能否基于图像特征可靠地对叶片形态（形状、颜色、褐斑）进行分类？
• RQ3 基于形态分类结果，预测模型能否仅凭形态特征准确推断植物处理（干旱或对照）？
• RQ4 EXIF标签能否用于估算叶片大小或实现表型-环境相关性分析？
• RQ5 轻量化、预训练模型（如PaddleOCR和SAM）能否在极少数据特定调优的情况下，有效部署于高通量表型分析？

主要发现

• OCR模型在从标注植物图像中提取非空文本方面达到94.31%的准确率，支持自动化电子表格生成。
• 叶片形态分类模型在形状、颜色和褐斑特征上的平均准确率为62.82%。
• 基于形态特征的处理预测模型以60.08%的准确率正确分类了干旱或对照条件。
• 由于缺少焦距和距离元数据，EXIF标签不足以用于叶片大小估算，限制了几何推断能力。
• 由于缺乏土壤或气象等环境元数据，尽管模型具备潜力，仍无法开展表型-环境相关性分析。
• 使用PaddleOCR和SAM等预训练模型可实现快速、准确且鲁棒的表型分析，且仅需极少微调，展现出强大的可扩展性潜力。