[论文解读] A comprehensive review of 3D convolutional neural network-based classification techniques of diseased and defective crops using non-UAV-based hyperspectral images

Nooshin Noshiri, Michael A. Beck+2 more|arXiv (Cornell University)|Jun 15, 2023

Spectroscopy and Chemometric AnalysesChemistry参考文献 80被引用 3

一句话总结

本文综述了用于非无人机平台高光谱成像（HSI）下作物病害与缺陷分类的3D卷积神经网络（3D-CNN）技术，强调其能够同时提取空间与光谱特征，从而提升分类准确率。文章指出数据稀缺与计算复杂性等挑战，并提出迁移学习、主动学习及光谱波段选择等解决方案，以提升在有限HSI数据集上的模型效率与性能。

ABSTRACT

Hyperspectral imaging (HSI) is a non-destructive and contactless technology that provides valuable information about the structure and composition of an object. It can capture detailed information about the chemical and physical properties of agricultural crops. Due to its wide spectral range, compared with multispectral- or RGB-based imaging methods, HSI can be a more effective tool for monitoring crop health and productivity. With the advent of this imaging tool in agrotechnology, researchers can more accurately address issues related to the detection of diseased and defective crops in the agriculture industry. This allows to implement the most suitable and accurate farming solutions, such as irrigation and fertilization before crops enter a damaged and difficult-to-recover phase of growth in the field. While HSI provides valuable insights into the object under investigation, the limited number of HSI datasets for crop evaluation presently poses a bottleneck. Dealing with the curse of dimensionality presents another challenge due to the abundance of spectral and spatial information in each hyperspectral cube. State-of-the-art methods based on 1D- and 2D-CNNs struggle to efficiently extract spectral and spatial information. On the other hand, 3D-CNN-based models have shown significant promise in achieving better classification and detection results by leveraging spectral and spatial features simultaneously. Despite the apparent benefits of 3D-CNN-based models, their usage for classification purposes in this area of research has remained limited. This paper seeks to address this gap by reviewing 3D-CNN-based architectures and the typical deep learning pipeline, including preprocessing and visualization of results, for the classification of hyperspectral images of diseased and defective crops. Furthermore, we discuss open research areas and challenges when utilizing 3D-CNNs with HSI data.

研究动机与目标

• 解决尽管3D-CNN具备优越的特征提取能力，但在高光谱图像（HSI）分类中用于作物病害与缺陷检测时，其应用仍受限的问题。
• 识别基于HSI的作物分类中的关键挑战，包括数据稀缺、高维性及计算复杂性。
• 回顾专为非无人机HSI应用中的3D-CNN设计的深度学习流程——包括预处理、模型架构与可视化方法。
• 探讨迁移学习、主动学习及信息性光谱波段选择等策略，以克服数据限制并提升模型效率。
• 倡导采用轻量化模型与机器学习即服务（MLaaS）集成，以实现在农业环境中的实时、低成本部署。

提出的方法

• 采用3D-CNN架构，将高光谱立方体作为3D张量（M×N×λ）处理，以捕捉空间与光谱特征的联合表示。
• 应用迁移学习，将大型数据集上预训练的模型微调至小规模HSI作物数据集，以在数据有限的情况下提升性能。
• 采用主动学习，迭代选择最具信息量的样本进行标注，从而减少标注负担并提高模型准确率。
• 分析归一化植被指数（NDVI）与绿色度指数（GCI）等光谱指数，以增强特征表示并支持模型可解释性。
• 探索波段降维与分组为指数的方法，以降低模型复杂度、训练时间与内存占用。
• 提出利用机器学习即服务（MLaaS）平台，以降低基础设施需求，加速非专业用户模型开发。

Figure 1: The hyperspectral cube (adapted from Tarabalka et al. ( 2010 ) with modification). It is a three-dimensional array where each pixel represents a spectrum containing a range of wavelengths. This spectrum can act as a fingerprint and provides information about biophysical and biochemical cha

实验结果

研究问题

• RQ1 3D-CNN如何有效从非无人机平台的高光谱图像中提取作物病害与缺陷分类所需的空间与光谱特征？
• RQ2 在将3D-CNN应用于高光谱作物成像时，主要挑战是什么，特别是数据稀缺与计算成本问题？
• RQ3 迁移学习与主动学习在有限HSI数据集上训练的3D-CNN模型中，能在多大程度上提升其性能？
• RQ4 哪些光谱波段对检测作物病害与缺陷最具信息量？如何选择这些波段以构建轻量化、高效模型？
• RQ5 MLaaS平台与模型优化技术如何实现精准农业中实时、可现场部署的3D-CNN系统？

主要发现

• 3D-CNN在分类高光谱作物图像方面优于1D与2D-CNN，因其能联合建模空间与光谱特征，从而实现更高准确率。
• 迁移学习通过利用预训练权重，在小规模HSI数据集上显著提升模型性能，减少过拟合与训练时间。
• 主动学习通过选择最具信息量的样本进行标注，有效减少所需标注样本数量，提升学习效率。
• 光谱波段选择与分组为指数可降低模型复杂度，缩短训练时间，并维持或提升分类准确率。
• MLaaS平台可通过提供预构建的3D-CNN模型与可扩展计算基础设施，降低农业从业者的技术门槛。
• 尽管前景广阔，3D-CNN在基于HSI的作物分类中仍应用不足，主要受限于数据稀缺与计算需求，表明亟需进一步研究与工具开发。

Figure 2: A basic conceptual CNN architecture. A CNN consists of multiple layers, including convolution, detector, and pooling layers, where each layer serves as the input for the subsequent layer, enabling the extraction of low-dimensional features, learning of non-linear representations, and dimen