[Paper Review] Paddy Doctor: A Visual Image Dataset for Automated Paddy Disease Classification and Benchmarking

A Petchiammal, Briskline Kiruba S+2 more|arXiv (Cornell University)|May 23, 2022

Smart Agriculture and AI17 citations

TL;DR

This paper introduces Paddy Doctor, a large-scale, publicly available dataset of 16,225 annotated paddy leaf images across 13 classes (12 diseases and healthy leaves), collected from real fields using high-resolution smartphones. The study benchmarks the dataset using five deep learning models, with ResNet34 achieving the highest F1-score of 97.50%, demonstrating its effectiveness for automated paddy disease classification in real-world agricultural applications.

ABSTRACT

One of the critical biotic stress factors paddy farmers face is diseases caused by bacteria, fungi, and other organisms. These diseases affect plants' health severely and lead to significant crop loss. Most of these diseases can be identified by regularly observing the leaves and stems under expert supervision. In a country with vast agricultural regions and limited crop protection experts, manual identification of paddy diseases is challenging. Thus, to add a solution to this problem, it is necessary to automate the disease identification process and provide easily accessible decision support tools to enable effective crop protection measures. However, the lack of availability of public datasets with detailed disease information limits the practical implementation of accurate disease detection systems. This paper presents \emph{Paddy Doctor}, a visual image dataset for identifying paddy diseases. Our dataset contains 16,225 annotated paddy leaf images across 13 classes (12 diseases and normal leaf). We benchmarked the \emph{Paddy Doctor} dataset using a Convolutional Neural Network (CNN) and four transfer learning based models (VGG16, MobileNet, Xception, and ResNet34). The experimental results showed that ResNet34 achieved the highest F1-score of 97.50%. We release our dataset and reproducible code in the open source for community use.

Motivation & Objective

• To address the critical lack of publicly available, high-quality, and diverse paddy leaf disease image datasets for training and benchmarking automated detection systems.
• To develop a scalable, real-world dataset that supports accurate and efficient disease classification using computer vision and deep learning.
• To provide a benchmark for state-of-the-art models in paddy disease detection, reducing reliance on manual identification and excessive pesticide use.
• To enable the development of accessible, low-cost decision support tools for farmers in regions with limited access to agricultural experts.

Proposed method

• The Paddy Doctor dataset was collected using high-resolution smartphone cameras from real paddy fields across diverse geographical locations in India.
• Images were manually cleaned and annotated by an agricultural expert, ensuring accurate labeling of 13 classes: 12 paddy diseases and one normal (healthy) leaf class.
• Five deep learning models were trained and evaluated: a custom Deep Convolutional Neural Network (DCNN) and four transfer learning models—VGG16, MobileNet, Xception, and ResNet34.
• Data augmentation techniques including rotation, shear, zoom, shift, and horizontal flipping were applied to improve model generalization and robustness.
• All models were trained on a 10,384-image training set (80% of 12,980 after validation split), with 3,245 images used for testing, using 256×256 pixel input size and ImageNet-pretrained weights.
• Hyperparameters included a batch size of 32, 100 training epochs, and a base learning rate of 0.001, with training conducted on Google Colab with GPU acceleration.

Experimental results

Research questions

• RQ1 How effective is a large-scale, real-world paddy leaf image dataset in enabling accurate automated disease classification using deep learning?
• RQ2 Which deep learning architecture performs best for classifying paddy leaf diseases across diverse symptoms and pathologies?
• RQ3 To what extent does transfer learning with pre-trained models improve classification performance compared to a custom CNN on this dataset?
• RQ4 Can data augmentation and fine-tuning strategies significantly enhance model generalization and robustness on real agricultural image data?
• RQ5 How does the Paddy Doctor dataset compare to existing public datasets in terms of size, class diversity, and image quality?

Key findings

• ResNet34 achieved the highest F1-score of 97.50%, outperforming all other models tested, including Xception (96.57%), VGG16 (93.20%), MobileNet (92.39%), and the custom DCNN (88.81%).
• The Xception model demonstrated strong performance with an F1-score of 96.57%, indicating high precision and recall across all disease classes.
• The custom DCNN model achieved an F1-score of 88.81%, highlighting the limitations of non-transfer learning approaches on complex, fine-grained disease classification tasks.
• The use of data augmentation significantly improved model generalization, as evidenced by consistent performance across all evaluation metrics.
• The Paddy Doctor dataset outperforms existing public datasets in both size (16,225 images) and class diversity (13 classes), including rare and complex diseases like Bacterial Panicle Blight and Yellow Stem Borer.
• The open-sourcing of the dataset and reproducible code enables community-wide benchmarking and future research in automated paddy disease detection.