[Paper Review] WeNet: Production First and Production Ready End-to-End Speech Recognition Toolkit.

Binbin Zhang, Di Wu+9 more|arXiv (Cornell University)|Feb 2, 2021

Speech Recognition and Synthesis6 references18 citations

TL;DR

WeNet is a production-first, end-to-end (E2E) speech recognition toolkit designed to bridge the gap between research and real-world deployment. It achieves low character error rate (CER) with efficient inference, demonstrating strong performance in both streaming and non-streaming scenarios on AISHELL-1, making it suitable for production use.

ABSTRACT

In this paper, we present a new open source, production first and production ready end-to-end (E2E) speech recognition toolkit named WeNet. The main motivation of WeNet is to close the gap between the research and the production of E2E speech recognition models. WeNet provides an efficient way to ship ASR applications in several real-world scenarios, which is the main difference and advantage to other open source E2E speech recognition toolkits. This paper introduces WeNet from three aspects, including model architecture, framework design and performance metrics. Our experiments on AISHELL-1 using WeNet, not only give a promising character error rate (CER) on a unified streaming and non-streaming two pass (U2) E2E model but also show reasonable RTF and latency, both of these aspects are favored for production adoption. The toolkit is publicly available at this https URL

Motivation & Objective

• Address the gap between research prototypes and production-ready E2E speech recognition systems.
• Enable efficient deployment of end-to-end ASR models in real-world applications.
• Support both streaming and non-streaming inference with a unified two-pass (U2) model architecture.
• Optimize inference efficiency and latency for production environments.
• Provide a scalable, open-source toolkit suitable for industrial-scale ASR applications.

Proposed method

• Design a unified two-pass (U2) E2E model architecture supporting both streaming and non-streaming inference.
• Implement efficient inference pipelines optimized for low latency and real-time performance.
• Leverage efficient neural network components and inference optimizations for production deployment.
• Integrate training and inference workflows into a single, production-ready framework.
• Use a single model architecture to handle both streaming and non-streaming inference, reducing complexity.
• Optimize model inference with hardware-aware optimizations for low RTF (real-time factor) and latency.

Experimental results

Research questions

• RQ1 How can end-to-end speech recognition models be made production-ready while maintaining high accuracy?
• RQ2 What architectural and engineering choices enable efficient deployment of E2E ASR in real-world systems?
• RQ3 Can a unified model achieve competitive performance in both streaming and non-streaming inference scenarios?
• RQ4 What are the latency and real-time factor (RTF) characteristics of E2E models in production-like settings?
• RQ5 How does the WeNet toolkit compare to existing open-source E2E ASR toolkits in terms of deployment readiness?

Key findings

• WeNet achieves a promising character error rate (CER) on the AISHELL-1 dataset using a unified two-pass (U2) E2E model.
• The model demonstrates reasonable real-time factor (RTF) and low latency, suitable for production deployment.
• The toolkit supports both streaming and non-streaming inference with a single model architecture.
• WeNet is designed for production use, with optimizations that ensure efficient inference in real-world scenarios.
• The open-source toolkit is publicly available and production-ready, enabling rapid deployment of E2E ASR applications.
• The framework successfully closes the gap between research prototypes and industrial-scale ASR deployment.