[Paper Review] License Plate Detection and Recognition Using Deeply Learned Convolutional Neural Networks

Syed Zain Masood, Guang Shu+2 more|arXiv (Cornell University)|Mar 21, 2017

Vehicle License Plate Recognition87 citations

TL;DR

The paper presents an end-to-end license plate detection and recognition system built from sequential deep CNNs, achieving higher accuracy than OpenALPR on US and European benchmarks. It emphasizes a unified model robust to region, pose, and lighting variations.

ABSTRACT

This work details Sighthounds fully automated license plate detection and recognition system. The core technology of the system is built using a sequence of deep Convolutional Neural Networks (CNNs) interlaced with accurate and efficient algorithms. The CNNs are trained and fine-tuned so that they are robust under different conditions (e.g. variations in pose, lighting, occlusion, etc.) and can work across a variety of license plate templates (e.g. sizes, backgrounds, fonts, etc). For quantitative analysis, we show that our system outperforms the leading license plate detection and recognition technology i.e. ALPR on several benchmarks. Our system is available to developers through the Sighthound Cloud API at https://www.sighthound.com/products/cloud

Motivation & Objective

• Develop an end-to-end pipeline for license plate detection and recognition using deep CNNs.
• Collect and annotate large-scale data for both plate detection and character recognition.
• Train specialized CNNs for plate detection, character detection, and character recognition.
• Evaluate the system on US and European benchmarks and compare with OpenALPR.
• Demonstrate robustness to variations in plate templates and imaging conditions.

Proposed method

• Collect and prepare data for license plate detection and character recognition.
• Train three CNNs: license plate detector, character detector, and character recognizer.
• Use a multi-step segmentation process for character isolation followed by binary filtering to remove non-characters.
• Recognize characters with a 35-class classifier (0-9 and A-Z excluding O).
• Evaluate with best-sequence matching against ground truth on benchmark datasets.

Experimental results

Research questions

• RQ1 Can a unified deep CNN-based pipeline detect and recognize license plates across different regions without plate-type priors?
• RQ2 How does the end-to-end system compare to OpenALPR in detection recall and recognition accuracy on USA and European datasets?
• RQ3 Is the pipeline robust to variations in lighting, pose, occlusion, and plate designs across regions?

Key findings

• License plate detection recall: USA 99.09%, Europe 99.64% (Sighthound) vs OpenALPR variants.
• License plate recognition accuracy: USA 93.44% (Sighthound) vs OpenALPR Open Source 78.36% and Cloud 84.64%; Europe 94.55% (Sighthound) vs OpenALPR Open Source 84.80% and Cloud 86.75%.
• The system delivers high accuracy across USA and Europe benchmarks and outperforms OpenALPR on both detection and recognition tasks.
• The approach uses a unified model that does not require plate-type specification (USA vs Europe).
• Qualitative results show robustness to illumination, size, blur, angle, and template variations.