[Paper Review] Text recognition on images using pre-trained CNN

Afgani Fajar Rizky, Novanto Yudistira+1 more|arXiv (Cornell University)|Feb 10, 2023

Computer Science and Engineering4 citations

TL;DR

This paper proposes a text recognition system using pre-trained VGG-16 CNN with data augmentation techniques, achieving 98.16% accuracy on test data and 95.62% on the IIIT-5K-Dataset. The method leverages transfer learning from ImageNet and applies rotation, scaling, and Gaussian blur to improve robustness and performance on scene text recognition tasks.

ABSTRACT

A text on an image often stores important information and directly carries high level semantics, makes it as important source of information and become a very active research topic. Many studies have shown that the use of CNN-based neural networks is quite effective and accurate for image classification which is the basis of text recognition. It can also be more enhanced by using transfer learning from pre-trained model trained on ImageNet dataset as an initial weight. In this research, the recognition is trained by using Chars74K dataset and the best model results then tested on some samples of IIIT-5K-Dataset. The research results showed that the best accuracy is the model that trained using VGG-16 architecture applied with image transformation of rotation 15°, image scale of 0.9, and the application of gaussian blur effect. The research model has an accuracy of 97.94% for validation data, 98.16% for test data, and 95.62% for the test data from IIIT-5K-Dataset. Based on these results, it can be concluded that pre-trained CNN can produce good accuracy for text recognition, and the model architecture that used in this study can be used as reference material in the development of text detection systems in the future

Motivation & Objective

• To investigate the effectiveness of pre-trained CNNs for scene text recognition.
• To evaluate the impact of data augmentation techniques such as rotation, scaling, and Gaussian blur on model performance.
• To assess the generalization capability of the model on unseen datasets like IIIT-5K-Dataset.
• To identify the optimal model architecture and hyperparameters for high-accuracy text recognition.

Proposed method

• Fine-tuning a pre-trained VGG-16 model using the Chars74K dataset for text recognition.
• Applying data augmentation via image rotation (15°), scaling (0.9), and Gaussian blur to improve robustness.
• Using transfer learning by initializing the model with ImageNet-pretrained weights to enhance feature learning.
• Training the model end-to-end with cross-entropy loss and Adam optimizer for classification.
• Evaluating performance on validation, test, and IIIT-5K-Dataset splits to measure generalization.

Experimental results

Research questions

• RQ1 Can pre-trained CNNs achieve high accuracy in scene text recognition tasks?
• RQ2 How do specific data augmentation techniques affect the performance of text recognition models?
• RQ3 What is the optimal combination of model architecture and data augmentation for maximum accuracy?
• RQ4 How well does the trained model generalize to unseen datasets like IIIT-5K-Dataset?

Key findings

• The best-performing model achieved 97.94% accuracy on validation data using VGG-16 with rotation, scaling, and Gaussian blur augmentation.
• The model reached 98.16% accuracy on the test set of the Chars74K dataset, demonstrating strong generalization.
• On the IIIT-5K-Dataset, the model maintained 95.62% accuracy, indicating effective transferability to diverse text images.
• The combination of VGG-16 architecture and specific data augmentation significantly improved recognition performance.