[Paper Review] Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling

Jonathan Shen, Patrick Nguyen+89 more|arXiv (Cornell University)|Feb 21, 2019

Algorithms and Data Compression3 references183 citations

TL;DR

Lingvo is a TensorFlow framework with modular building blocks for sequence-to-sequence modeling, featuring centralized experiment configurations, distributed training, and production-ready components such as inference and quantization.

ABSTRACT

Lingvo is a Tensorflow framework offering a complete solution for collaborative deep learning research, with a particular focus towards sequence-to-sequence models. Lingvo models are composed of modular building blocks that are flexible and easily extensible, and experiment configurations are centralized and highly customizable. Distributed training and quantized inference are supported directly within the framework, and it contains existing implementations of a large number of utilities, helper functions, and the newest research ideas. Lingvo has been used in collaboration by dozens of researchers in more than 20 papers over the last two years. This document outlines the underlying design of Lingvo and serves as an introduction to the various pieces of the framework, while also offering examples of advanced features that showcase the capabilities of the framework.

Motivation & Objective

• Provide a modular, extensible framework for rapid prototyping of sequence-to-sequence models.
• Enable reproducible, comparable experiments through centralized hyperparameter configuration and a registry-driven model setup.
• Support scalable distributed training and deployment, including inference and quantization.
• Facilitate multi-task modeling and reusability of common building blocks across tasks.

Proposed method

• Introduce modular building blocks (Layers, Inputs, Params) with uniform interfaces for easy composition.
• Utilize a hierarchical Params system to configure layers, tasks, and models with explicit defaults overridden by experiment configurations.
• Provide a model registry and decorators to register single-task and multi-task models for easy instantiation.
• Support distributed training (synchronous and asynchronous) with multiple job runners and a parameter-server style setup.
• Offer input processing pipelines with bucketed batching and support for multiple tokenizers and data formats.
• Incorporate features for inference and quantization to enable production deployment and efficiency.

Experimental results

Research questions

• RQ1 How can a modular, reusable set of building blocks accelerate development and experimentation for sequence-to-sequence models?
• RQ2 Can a unified Params-driven configuration and model registry improve reproducibility and comparability across researchers and tasks?
• RQ3 What are the practical considerations and mechanisms for distributed training and multi-task learning within such a framework?
• RQ4 How can inference and quantization be integrated into the same codebase used for training to streamline deployment?

Key findings

• Lingvo provides a cohesive design with modular layers, input processors, and a Params system that supports hierarchical configuration and easy overrides.
• Experiments can be reproduced and compared more easily due to explicit hyperparameter declarations, centralized configurations, and runtime logging.
• The framework supports both asynchronous and synchronous distributed training across multiple job runners and devices.
• Lingvo includes support for multi-task models with shared variables and configurable task weights.
• Inference and quantization are integrated, enabling export of inference graphs and device-specific implementations within the same framework.