[Paper Review] Butterfly-Net: Optimal Function Representation Based on Convolutional Neural Networks

Yingzhou Li, Xiuyuan Cheng+1 more|arXiv (Cornell University)|May 18, 2018

Model Reduction and Neural Networks44 references17 citations

TL;DR

This paper proposes Butterfly-net, a low-complexity convolutional neural network with structured, sparse cross-channel connections inspired by the fast butterfly algorithm. It achieves exponential error decay in approximating Fourier kernels and enables optimal function representation with model complexity O(K log N) and computational complexity O(N log N), significantly reducing parameter count while maintaining accuracy compared to standard CNNs.

ABSTRACT

Deep networks, especially convolutional neural networks (CNNs), have been successfully applied in various areas of machine learning as well as to challenging problems in other scientific and engineering fields. This paper introduces Butterfly-Net, a low-complexity CNN with structured and sparse cross-channel connections, together with a Butterfly initialization strategy for a family of networks. Theoretical analysis of the approximation power of Butterfly-Net to the Fourier representation of input data shows that the error decays exponentially as the depth increases. Combining Butterfly-Net with a fully connected neural network, a large class of problems are proved to be well approximated with network complexity depending on the effective frequency bandwidth instead of the input dimension. Regular CNN is covered as a special case in our analysis. Numerical experiments validate the analytical results on the approximation of Fourier kernels and energy functionals of Poisson's equations. Moreover, all experiments support that training from Butterfly initialization outperforms training from random initialization. Also, adding the remaining cross-channel connections, although significantly increase the parameter number, does not much improve the post-training accuracy and is more sensitive to data distribution.

Motivation & Objective

• To develop a low-complexity CNN architecture with structured sparsity that enables optimal function representation in scientific computing and signal processing.
• To analyze the approximation power of the network for Fourier kernels and related functionals, especially in the context of PDEs and integral operators.
• To demonstrate that network complexity scales with effective frequency bandwidth K rather than input dimension N, improving efficiency for high-dimensional problems.
• To evaluate the superiority of Butterfly initialization over random initialization in training deep networks for function approximation.
• To compare Butterfly-net with standard CNNs and Inflated-Butterfly-net in terms of parameter efficiency, accuracy, and generalization under domain shift.

Proposed method

• Design a CNN architecture with hierarchical, multi-scale structure based on the butterfly scheme, using convolutional and transpose-convolutional layers with recursive interpolation.
• Introduce a structured Butterfly initialization that encodes Fourier transform operations directly into the network weights, enabling accurate computation of Fourier coefficients.
• Implement sparse cross-channel connections where each channel corresponds to a specific frequency band, reducing parameter count while preserving approximation power.
• Construct an Inflated-Butterfly-net variant with dense cross-channel connections to compare parameter efficiency and performance.
• Use a task-specific head layer (e.g., square-sum or dense-dense) to represent functionals such as energy functionals of Poisson’s equation.
• Train networks from both Butterfly initialization and random initialization, and compare post-training accuracy and generalization under domain shift.

Experimental results

Research questions

• RQ1 Can a CNN with structured, sparse cross-channel connections achieve exponential error decay in approximating Fourier kernels as depth increases?
• RQ2 Does the model complexity of Butterfly-net scale with the effective frequency bandwidth K rather than the input dimension N, leading to improved sample efficiency?
• RQ3 How does Butterfly initialization compare to random initialization in terms of convergence speed and final accuracy for function approximation tasks?
• RQ4 To what extent does adding dense cross-channel connections in Inflated-Butterfly-net improve accuracy, and does it justify the increased parameter count?
• RQ5 How does Butterfly-net generalize under data distribution shifts compared to standard CNNs?

Key findings

• Butterfly-net achieves exponential decay in approximation error for Fourier kernels as network depth increases, with theoretical and numerical validation.
• The model complexity is O(K log N) and computational complexity is O(N log N), where K is the effective frequency bandwidth and N is the input length, significantly reducing parameter count compared to standard CNNs.
• Butterfly-net with Butterfly initialization outperforms random initialization in approximating discrete Fourier transforms and energy functionals of Poisson’s equations.
• Inflated-Butterfly-net, despite having orders of magnitude more parameters, shows only marginal improvement in post-training accuracy compared to Butterfly-net.
• Butterfly-net generalizes better than Inflated-Butterfly-net in transfer learning settings with domain shift, indicating improved robustness.
• The square-sum layer in the task head enables exact representation of certain functionals, leading to more accurate approximations than the dense-dense layer.