[Paper Review] Accelerating Generalized Linear Models with MLWeaving: A One-Size-Fits-All System for Any-precision Learning (Technical Report)

Zeke Wang, Kaan Kara+4 more|arXiv (Cornell University)|Mar 8, 2019

Neural Networks and Applications132 references4 citations

TL;DR

MLWeaving accelerates generalized linear model training in databases by enabling dynamic, any-precision learning through a bit-serial, transposed memory layout and FPGA-based hardware acceleration. It achieves up to 16× speedup over low-precision CPU implementations by reducing memory traffic and enabling efficient, precision-adaptive SGD on FPGAs with a single hardware design.

ABSTRACT

Learning from the data stored in a database is an important function increasingly available in relational engines. Methods using lower precision input data are of special interest given their overall higher efficiency but, in databases, these methods have a hidden cost: the quantization of the real value into a smaller number is an expensive step. To address the issue, in this paper we present MLWeaving, a data structure and hardware acceleration technique intended to speed up learning of generalized linear models in databases. ML-Weaving provides a compact, in-memory representation enabling the retrieval of data at any level of precision. MLWeaving also takes advantage of the increasing availability of FPGA-based accelerators to provide a highly efficient implementation of stochastic gradient descent. The solution adopted in MLWeaving is more efficient than existing designs in terms of space (since it can process any resolution on the same design) and resources (via the use of bit-serial multipliers). MLWeaving also enables the runtime tuning of precision, instead of a fixed precision level during the training. We illustrate this using a simple, dynamic precision schedule. Experimental results show MLWeaving achieves up to16 performance improvement over low-precision CPU implementations of first-order methods.

Motivation & Objective

• To address the high memory overhead of low-precision data processing in database systems during machine learning training.
• To eliminate the need for pre-quantized data copies or multiple microarchitectures per precision level.
• To enable dynamic precision selection during SGD training without sacrificing performance.
• To achieve hardware-efficient, synchronous SGD on FPGAs that matches or exceeds the efficiency of asynchronous CPU approaches.
• To unify data representation and hardware acceleration for generalized linear models across all precision levels in a single system.

Proposed method

• MLWeaving uses a transposed memory layout that stores bits of each feature value across different memory segments, enabling efficient bit-level access for any precision.
• It serializes data into a bit stream format, allowing bit-serial multiplication in hardware to compute gradients efficiently across varying precision levels.
• The FPGA-based SGD accelerator processes one bit slice per cycle, reducing resource usage and increasing clock frequency compared to prior designs.
• The system supports dynamic precision tuning by reading more or fewer bits per feature per epoch, enabling adaptive precision during training.
• It employs a synchronous SGD execution model on FPGA that achieves high hardware efficiency while maintaining fast convergence.
• The design integrates with a column-store database (DoppioDB) and uses on-chip memory to minimize off-chip data movement.

Experimental results

Research questions

• RQ1 Can a single hardware accelerator efficiently support stochastic gradient descent across all precision levels without requiring separate microarchitectures per precision?
• RQ2 How can a database system minimize memory traffic and data movement overhead when training generalized linear models with low-precision inputs?
• RQ3 Can synchronous SGD on FPGAs achieve hardware efficiency comparable to asynchronous SGD on CPUs while enabling faster convergence?
• RQ4 What in-memory data layout enables efficient, dynamic retrieval of data at any precision level without pre-quantization?
• RQ5 Can bit-serial computation on FPGAs outperform traditional word-parallel approaches in terms of resource utilization and throughput for low-precision learning?

Key findings

• MLWeaving achieves up to 16× speedup over the state-of-the-art low-precision first-order CPU implementation on generalized linear model training.
• The FPGA-based implementation uses only one-third of the resources required by prior FPGA accelerators for low-precision SGD.
• The system doubles the data processing rate compared to previous FPGA designs due to improved memory bandwidth utilization and higher clock frequency.
• MLWeaving enables dynamic precision tuning during training, allowing convergence in fewer epochs than asynchronous CPU counterparts.
• The transposed bit-level memory layout reduces memory accesses proportionally to the precision level, achieving linear speedup as precision decreases.
• The system supports models with up to 32K dimensions using on-chip memory, with potential for future scaling on larger FPGAs.