[Paper Review] Efficient logic architecture in training gradient boosting decision tree for high-performance and edge computing

Takuya Tanaka, Ryosuke Kasahara+1 more|arXiv (Cornell University)|Dec 20, 2018

Machine Learning and Data Classification13 references4 citations

TL;DR

This paper proposes a custom logic architecture for accelerating gradient boosting decision tree (GBDT) training on FPGAs, achieving 26–259× faster training and 90–1,104× higher power efficiency than CPU/GPU software libraries. The design optimizes memory access and computation for binary classification, enabling high-performance and energy-efficient GBDT training suitable for edge and high-performance computing environments.

ABSTRACT

This study proposes a logic architecture for the high-speed and power efficiently training of a gradient boosting decision tree model of binary classification. We implemented the proposed logic architecture on an FPGA and compared training time and power efficiency with three general GBDT software libraries using CPU and GPU. The training speed of the logic architecture on the FPGA was 26-259 times faster than the software libraries. The power efficiency of the logic architecture was 90-1,104 times higher than the software libraries. The results show that the logic architecture suits for high-performance and edge computing.

Motivation & Objective

• To address the high computational and energy cost of training GBDT models in resource-constrained environments like edge devices.
• To reduce training latency and improve power efficiency for GBDT, especially for binary classification tasks.
• To design a domain-specific logic architecture tailored for FPGA deployment to outperform general-purpose software libraries.
• To enable high-performance GBDT training suitable for real-time and low-power edge computing applications.

Proposed method

• Design a specialized logic architecture on FPGA to accelerate key GBDT training operations, including split evaluation and gradient computation.
• Optimize memory access patterns to reduce data movement and improve bandwidth utilization during tree node splitting.
• Implement parallel processing units for efficient computation of weighted quantiles and gain metrics across features.
• Use fixed-point arithmetic and pipelined data paths to minimize latency and maximize throughput.
• Integrate a custom dataflow that minimizes redundant computation and enables efficient gradient and Hessian accumulation.
• Map the GBDT training pipeline to the FPGA fabric with minimal off-chip memory access, leveraging on-chip block RAM and DSP resources.

Experimental results

Research questions

• RQ1 Can a custom FPGA-based logic architecture significantly reduce training time for gradient boosting decision trees compared to general-purpose CPU and GPU software libraries?
• RQ2 To what extent can a hardware-optimized GBDT training architecture improve power efficiency in high-performance and edge computing scenarios?
• RQ3 How does the proposed logic architecture manage memory bandwidth and data movement to accelerate GBDT training on FPGAs?
• RQ4 What performance and energy efficiency gains are achievable by co-designing the GBDT algorithm with FPGA-specific hardware features?

Key findings

• The FPGA-based logic architecture achieved 26–259× faster training speed compared to CPU and GPU software libraries.
• The proposed architecture demonstrated 90–1,104× higher power efficiency than the evaluated software libraries.
• The performance gain was most significant on larger datasets, where the hardware acceleration fully exploited parallelism and reduced memory bottlenecks.
• The design achieved high utilization of FPGA resources, with efficient use of block RAM and DSP slices for gradient and split computation.
• The results confirm that hardware acceleration via custom logic is highly effective for GBDT training in low-latency and energy-constrained environments.
• The architecture is particularly well-suited for edge computing due to its low power consumption and high throughput.