[Paper Review] Explainable vision transformer enabled convolutional neural network for plant disease identification: PlantXViT

Poornima Singh Thakur, Pritee Khanna+2 more|arXiv (Cornell University)|Jul 16, 2022

Smart Agriculture and AI40 citations

TL;DR

PlantXViT is a lightweight hybrid CNN–ViT model for plant disease identification that outperforms several state-of-the-art CNNs on five public datasets and provides explainability via Grad-CAM and LIME.

ABSTRACT

Plant diseases are the primary cause of crop losses globally, with an impact on the world economy. To deal with these issues, smart agriculture solutions are evolving that combine the Internet of Things and machine learning for early disease detection and control. Many such systems use vision-based machine learning methods for real-time disease detection and diagnosis. With the advancement in deep learning techniques, new methods have emerged that employ convolutional neural networks for plant disease detection and identification. Another trend in vision-based deep learning is the use of vision transformers, which have proved to be powerful models for classification and other problems. However, vision transformers have rarely been investigated for plant pathology applications. In this study, a Vision Transformer enabled Convolutional Neural Network model called "PlantXViT" is proposed for plant disease identification. The proposed model combines the capabilities of traditional convolutional neural networks with the Vision Transformers to efficiently identify a large number of plant diseases for several crops. The proposed model has a lightweight structure with only 0.8 million trainable parameters, which makes it suitable for IoT-based smart agriculture services. The performance of PlantXViT is evaluated on five publicly available datasets. The proposed PlantXViT network performs better than five state-of-the-art methods on all five datasets. The average accuracy for recognising plant diseases is shown to exceed 93.55%, 92.59%, and 98.33% on Apple, Maize, and Rice datasets, respectively, even under challenging background conditions. The efficiency in terms of explainability of the proposed model is evaluated using gradient-weighted class activation maps and Local Interpretable Model Agnostic Explanation.

Motivation & Objective

• Motivate the need for accurate and explainable plant disease identification in smart agriculture.
• Propose a lightweight hybrid architecture combining CNNs with Vision Transformer blocks.
• Show that PlantXViT achieves high accuracy across diverse crop datasets while maintaining interpretability.

Proposed method

• Use two blocks of pre-trained VGG16, an inception-v7 block, and four transformer encoder blocks to form PlantXViT.
• Convert CNN feature maps to 5x5 patches, project them linearly, and process with a four-block Transformer encoder.
• Train with categorical cross-entropy loss using Adam optimizer; learning rate 0.0001, batch size 16.
• Evaluate explainability with Grad-CAM and LIME.
• Dataset preprocessing includes resizing to 224x224x3 and using five public plant-disease datasets.

Experimental results

Research questions

• RQ1 How well does the PlantXViT hybrid architecture perform on diverse plant-disease datasets compared to recent CNN-based methods?
• RQ2 Does integrating ViT blocks with CNN features improve both accuracy and explainability for plant disease classification?
• RQ3 How effective are Grad-CAM and LIME in explaining PlantXViT predictions across datasets?
• RQ4 What is the optimal patch size for the ViT component in PlantXViT for best performance?
• RQ5 How does PlantXViT perform under varying dataset sizes and class imbalances?

Key findings

• PlantXViT achieves high accuracy across five public datasets, outperforming five state-of-the-art CNN-based methods on all datasets.
• Patch size experimentation indicates 5x5 patches yield the best overall performance in accuracy, precision, recall, and F1 across datasets.
• Explainability analyses (Grad-CAM and LIME) provide insights into model decisions and highlight local regions contributing to predictions.
• The model is lightweight, with about 0.85 million trainable parameters, suitable for IoT-enabled smart agriculture devices.
• Across datasets, PlantXViT demonstrates strong ROC/AUC performance and competitive Cohen’s kappa scores, indicating reliable classification under varied conditions.