[Paper Review] Fast Convolutional Nets With fbfft: A GPU Performance Evaluation

Nicolas Vasilache, Jeff Johnson+4 more|arXiv (Cornell University)|Dec 24, 2014

Advanced Neural Network Applications14 references251 citations

TL;DR

This paper presents fbfft, an optimized open-source FFT library for GPU-accelerated convolutional neural networks, achieving over 1.5× speedup over cuFFT and up to 23.5× over cuDNN for specific convolution layers by leveraging algorithmic optimizations tailored to deep learning workloads, including tiling, in-place transposition, and custom kernel fusion for frequency-domain convolutions.

ABSTRACT

We examine the performance profile of Convolutional Neural Network training on the current generation of NVIDIA Graphics Processing Units. We introduce two new Fast Fourier Transform convolution implementations: one based on NVIDIA's cuFFT library, and another based on a Facebook authored FFT implementation, fbfft, that provides significant speedups over cuFFT (over 1.5x) for whole CNNs. Both of these convolution implementations are available in open source, and are faster than NVIDIA's cuDNN implementation for many common convolutional layers (up to 23.5x for some synthetic kernel configurations). We discuss different performance regimes of convolutions, comparing areas where straightforward time domain convolutions outperform Fourier frequency domain convolutions. Details on algorithmic applications of NVIDIA GPU hardware specifics in the implementation of fbfft are also provided.

Motivation & Objective

• Address the performance bottleneck of convolutional neural network (CNN) training on GPUs by exploring frequency-domain convolution using Fast Fourier Transforms (FFTs).
• Overcome limitations of existing GPU FFT libraries like cuFFT and cuDNN in handling small feature planes and large mini-batches common in deep learning.
• Develop a custom, highly optimized FFT implementation (fbfft) that outperforms both cuFFT and cuDNN for a broad range of convolutional layer configurations.
• Improve GPU utilization and reduce memory pressure through algorithmic optimizations such as tiling, asynchronous FFT precomputation, and in-place memory management.

Proposed method

• Implement frequency-domain convolution using the convolution theorem: transform input and kernel to frequency domain via FFT, perform pointwise multiplication, then inverse FFT to obtain output.
• Develop fbfft, a from-scratch, open-source GPU FFT library optimized for 1D and 2D batched transforms, with support for power-of-two sizes and implicit padding to reduce memory overhead.
• Apply tiling to decompose large convolutions into smaller ones (size ~kernel size), reducing complexity from O(n log n) to O(n log w), enabling efficient use of fbfft’s high performance at small sizes (8–64).
• Optimize memory access and kernel launch overhead by reusing FFT results across layers, precomputing weight FFTs asynchronously, and eliminating redundant temporary buffers.
• Integrate custom in-place, in-register transposition and pointwise multiplication kernels to reduce memory bandwidth and kernel launch costs.
• Leverage DIF/DIT FFT variants to eliminate bit-reversal operations, improving performance on small FFT sizes.

Experimental results

Research questions

• RQ1 Can frequency-domain convolution via FFT outperform time-domain convolution in CNN training on modern GPUs for typical deep learning workloads?
• RQ2 How do the performance characteristics of cuFFT and cuDNN compare across diverse convolution layer configurations, especially for small feature planes and large mini-batches?
• RQ3 To what extent can a custom FFT implementation (fbfft) improve performance over black-box libraries like cuFFT by exploiting domain-specific patterns in deep learning?
• RQ4 What algorithmic optimizations—such as tiling, memory reuse, and kernel fusion—are most effective in minimizing latency and maximizing GPU utilization in FFT-based convolutions?
• RQ5 Can the performance advantage of fbfft at small FFT sizes (8–64) be extended to larger convolutions through tiling and memory management strategies?

Key findings

• fbfft achieves a mean speedup of 1.51× over cuFFT for convolution operations in the target problem domain, with gains exceeding 1.5× in many configurations.
• The FFT-based convolution implementation using fbfft outperforms cuDNN by up to 23.5× for synthetic kernel configurations and achieves 1.4×–14.5× speedups across common layer sizes.
• For small-sized convolutions (8–64), fbfft delivers over 1.5× speedup over cuFFT due to optimized memory access, reduced kernel launch overhead, and implicit padding.
• Tiling strategies reduce the asymptotic complexity of large convolutions from O(n log n) to O(n log w), enabling efficient use of fbfft’s high performance at small sizes.
• Memory pressure is reduced by reusing FFT results across layers and precomputing weight FFTs asynchronously, improving GPU utilization and reducing temporary buffer usage.
• The authors achieve GPU efficiency ratios exceeding 75% in certain cases, demonstrating high utilization of GPU compute resources through algorithmic and memory optimization.