[Paper Review] LPRNet: License Plate Recognition via Deep Neural Networks

S. V. Zherzdev, Alexey Gruzdev|arXiv (Cornell University)|Jun 27, 2018

Vehicle License Plate RecognitionEngineering19 references112 citations

TL;DR

LPRNet introduces an end-to-end, segmentation-free license plate recognition system using a lightweight CNN that runs in real time and achieves up to 95% accuracy on Chinese plates, without RNNs.

ABSTRACT

This paper proposes LPRNet - end-to-end method for Automatic License Plate Recognition without preliminary character segmentation. Our approach is inspired by recent breakthroughs in Deep Neural Networks, and works in real-time with recognition accuracy up to 95% for Chinese license plates: 3 ms/plate on nVIDIA GeForce GTX 1080 and 1.3 ms/plate on Intel Core i7-6700K CPU. LPRNet consists of the lightweight Convolutional Neural Network, so it can be trained in end-to-end way. To the best of our knowledge, LPRNet is the first real-time License Plate Recognition system that does not use RNNs. As a result, the LPRNet algorithm may be used to create embedded solutions for LPR that feature high level accuracy even on challenging Chinese license plates.

Motivation & Objective

• Motivate a fast, segmentation-free license plate recognition system robust to distortions, lighting, and viewpoint variations.
• Develop a lightweight CNN backbone capable of end-to-end recognition without recurrent networks.
• Enable end-to-end training with CTC loss and explore deployment on diverse hardware (CPU, GPU, FPGA).
• Demonstrate robustness on challenging Chinese license plates and perform ablation studies to isolate contributing factors.

Proposed method

• Design a lightweight CNN backbone inspired by SqueezeNet and Inception blocks for low GFLOPs per forward pass.
• Incorporate a Spatial Transformer Network (optional) for input alignment.
• Use a per-position character classification head producing a sequence of probabilities across the width of the image.
• Train with CTC loss to handle variable-length, unsegmented outputs.
• Augment with global context embedding before decoding to improve accuracy.
• Apply a post-filtering step using country-specific templates during beam search to constrain outputs.

Experimental results

Research questions

• RQ1 Can a fully convolutional, non-RNN architecture achieve competitive accuracy for license plate recognition without pre-segmentation?
• RQ2 What is the impact of global context, data augmentation, STN alignment, beam search, and post-filtering on recognition accuracy?
• RQ3 How does LPRNet perform in real-time across CPU, GPU, and FPGA hardware on challenging Chinese license plates?

Key findings

• LPRNet baseline achieves 94.1% recognition accuracy with 0.71 GFLOPs.
• LPRNet basic achieves 95.0% accuracy with 0.34 GFLOPs.
• LPRNet reduced achieves 94.0% accuracy with 0.163 GFLOPs.
• A global context augmentation yields a large accuracy gain (up to 36 percentage points over the baseline).
• Data augmentation improves accuracy by about 28.6 percentage points.
• Beam search with post-filtering provides an additional 0.4–0.6 percentage point gain.
• The network runs in real time: 3 ms per plate on GTX 1080, 1.3 ms per plate on Intel Core i7-6700K CPU, and 4 ms on an FPGA path.