[Paper Review] PotatoGANs: Utilizing Generative Adversarial Networks, Instance Segmentation, and Explainable AI for Enhanced Potato Disease Identification and Classification

Fatema Tuj Johora Faria, Mukaffi Bin Moin+4 more|arXiv (Cornell University)|May 12, 2024

Smart Agriculture and AIAgricultural and Biological Sciences5 citations

TL;DR

PotatoGANs uses CycleGAN and Pix2Pix to generate diseased potato images from healthy ones, augments data for better potato disease detection, pairs XAI with multiple CNNs for interpretability, and employs Detectron2 for instance segmentation. It reports higher image quality via Inception Score and a Dice score of 0.8112 with Detectron2.

ABSTRACT

Numerous applications have resulted from the automation of agricultural disease segmentation using deep learning techniques. However, when applied to new conditions, these applications frequently face the difficulty of overfitting, resulting in lower segmentation performance. In the context of potato farming, where diseases have a large influence on yields, it is critical for the agricultural economy to quickly and properly identify these diseases. Traditional data augmentation approaches, such as rotation, flip, and translation, have limitations and frequently fail to provide strong generalization results. To address these issues, our research employs a novel approach termed as PotatoGANs. In this novel data augmentation approach, two types of Generative Adversarial Networks (GANs) are utilized to generate synthetic potato disease images from healthy potato images. This approach not only expands the dataset but also adds variety, which helps to enhance model generalization. Using the Inception score as a measure, our experiments show the better quality and realisticness of the images created by PotatoGANs, emphasizing their capacity to resemble real disease images closely. The CycleGAN model outperforms the Pix2Pix GAN model in terms of image quality, as evidenced by its higher IS scores CycleGAN achieves higher Inception scores (IS) of 1.2001 and 1.0900 for black scurf and common scab, respectively. This synthetic data can significantly improve the training of large neural networks. It also reduces data collection costs while enhancing data diversity and generalization capabilities. Our work improves interpretability by combining three gradient-based Explainable AI algorithms (GradCAM, GradCAM++, and ScoreCAM) with three distinct CNN architectures (DenseNet169, Resnet152 V2, InceptionResNet V2) for potato disease classification.

Motivation & Objective

• Address overfitting and low generalization in potato disease segmentation under new conditions.
• Develop a GAN-based data augmentation pipeline to synthesize realistic diseased potato images from healthy ones.
• Improve interpretability of classification via combined GradCAM, GradCAM++, and ScoreCAM with multiple CNN architectures.
• Leverage Detectron2 for improved potato disease segmentation and localization.

Proposed method

• Use CycleGAN and Pix2Pix to convert healthy potato images into diseased counterparts for data augmentation.
• Create a dataset with generated and manually annotated images verified by Bangladesh Agricultural Research Institute (BARI).
• Evaluate generated image quality with Fréchet Inception Distance and Inception Score.
• Combine GradCAM, GradCAM++, and ScoreCAM with DenseNet169, ResNet152 V2, and InceptionResNet V2 for interpretable disease classification.
• Apply Detectron2 with a ResNeXt-101 backbone for instance segmentation and report Dice and IoU metrics.

Figure 1: Visual Diagram of our Proposed Potato Disease Image Generation and Disease Segmentation System, meticulously designed to capture and transform specific regions, providing a comprehensive grasp of the dynamic process from the creation of images to the accurate segmentation of diseases

Experimental results

Research questions

• RQ1 Can GAN-based image generation improve generalization for potato disease classification and localization?
• RQ2 How do XAI methods visualize and validate the decision-making of CNN classifiers for potato diseases?
• RQ3 Does Detectron2-based instance segmentation enhance disease localization accuracy on generated data?
• RQ4 Which CNN architectures best benefit from the integrated XAI approach for potato disease classification?

Key findings

• CycleGAN outperforms Pix2Pix in image quality as per Inception Score for Black Scurf (IS=1.2001) and Common Scab (IS=1.0900).
• Generated synthetic data improves training of large neural networks and reduces data collection costs.
• Dice score with Detectron2 using the ResNeXt-101 backbone reaches 0.8112.
• Three gradient-based XAI methods (GradCAM, GradCAM++, ScoreCAM) combined with three CNNs (DenseNet169, ResNet152V2, InceptionResNetV2) yield deeper interpretability of model decisions.
• The extended dataset with generation and segmentation supports improved disease identification and localization in potato crops.

Figure 2: Visual Representation of Generated Realistic Potato Disease Images