[Paper Review] Applying Data Augmentation to Handwritten Arabic Numeral Recognition Using Deep Learning Neural Networks

Akm Ashiquzzaman, Abdul Kawsar Tushar+1 more|arXiv (Cornell University)|Aug 20, 2017

Handwritten Text Recognition Techniques7 references3 citations

TL;DR

This paper proposes a deep convolutional neural network (CNN) for handwritten Arabic numeral recognition that improves accuracy to 99.4% by applying data augmentation and replacing ReLU with the ELU activation function to mitigate overfitting and vanishing gradient issues. The method enhances feature learning through augmented training data and improved activation dynamics, outperforming prior models on the CMATERDB 3.3.1 dataset.

ABSTRACT

Handwritten character recognition has been the center of research and a benchmark problem in the sector of pattern recognition and artificial intelligence, and it continues to be a challenging research topic. Due to its enormous application many works have been done in this field focusing on different languages. Arabic, being a diversified language has a huge scope of research with potential challenges. A convolutional neural network model for recognizing handwritten numerals in Arabic language is proposed in this paper, where the dataset is subject to various augmentation in order to add robustness needed for deep learning approach. The proposed method is empowered by the presence of dropout regularization to do away with the problem of data overfitting. Moreover, suitable change is introduced in activation function to overcome the problem of vanishing gradient. With these modifications, the proposed system achieves an accuracy of 99.4\% which performs better than every previous work on the dataset.

Motivation & Objective

• To improve the accuracy of handwritten Arabic numeral recognition beyond existing deep learning models.
• To address overfitting in small datasets using data augmentation and dropout regularization.
• To mitigate the vanishing gradient problem by replacing ReLU with the ELU activation function.
• To enhance feature learning and model generalization through architectural and activation function modifications.
• To achieve state-of-the-art performance on the CMATERDB 3.3.1 dataset for Arabic numeral recognition.

Proposed method

• The proposed model uses a deeper CNN architecture with two additional fully connected layers compared to prior work.
• Data augmentation is applied during training, including random shifts, zooms, and image inversion to increase training diversity.
• The ELU activation function is used instead of ReLU to improve gradient flow and reduce internal covariate shift.
• Dropout regularization is applied to prevent overfitting, especially in the fully connected layers.
• The model is trained using categorical cross-entropy loss and Adadelta optimizer over 100 epochs with a batch size of 128.
• Training is accelerated using CUDA-enabled GPU (NVIDIA GTX 625M) via Theano and Keras frameworks.

Experimental results

Research questions

• RQ1 Can data augmentation significantly improve generalization and accuracy in handwritten Arabic numeral recognition using deep learning?
• RQ2 Does replacing ReLU with ELU reduce the vanishing gradient problem and improve model convergence?
• RQ3 How does the combination of data augmentation and ELU activation affect performance compared to baseline CNNs with ReLU and no augmentation?
• RQ4 To what extent does architectural enhancement (additional fully connected layers) contribute to improved recognition accuracy?
• RQ5 Can the proposed method achieve state-of-the-art performance on the CMATERDB 3.3.1 dataset for Arabic numerals?

Key findings

• The proposed model achieves a test accuracy of 99.4% on the CMATERDB 3.3.1 dataset, surpassing previous methods.
• The model with data augmentation and ELU outperforms the baseline CNN (97.4% accuracy) and the MLP baseline (93.8% accuracy).
• The confusion matrix shows near-perfect classification with only two misclassifications in the entire test set of 500 samples.
• Class 6 had two misclassifications, with one image incorrectly predicted as class 1, indicating minor ambiguity in digit shape.
• The use of data augmentation increased the effective training data size and improved robustness to variations in digit positioning and scale.
• The ELU activation function contributed to more stable and uniform learning dynamics, reducing the risk of dying ReLU neurons.