[Paper Review] ACNet: Strengthening the Kernel Skeletons for Powerful CNN via Asymmetric Convolution Blocks

Xiaohan Ding, Yuchen Guo+2 more|arXiv (Cornell University)|Aug 11, 2019

Advanced Neural Network ApplicationsComputer Science35 references87 citations

TL;DR

ACNet introduces Asymmetric Convolution Blocks (ACB) to enrich off-the-shelf CNNs during training and then fuse back to the original architecture for no extra inference cost, improving accuracy and robustness to rotational distortions.

ABSTRACT

As designing appropriate Convolutional Neural Network (CNN) architecture in the context of a given application usually involves heavy human works or numerous GPU hours, the research community is soliciting the architecture-neutral CNN structures, which can be easily plugged into multiple mature architectures to improve the performance on our real-world applications. We propose Asymmetric Convolution Block (ACB), an architecture-neutral structure as a CNN building block, which uses 1D asymmetric convolutions to strengthen the square convolution kernels. For an off-the-shelf architecture, we replace the standard square-kernel convolutional layers with ACBs to construct an Asymmetric Convolutional Network (ACNet), which can be trained to reach a higher level of accuracy. After training, we equivalently convert the ACNet into the same original architecture, thus requiring no extra computations anymore. We have observed that ACNet can improve the performance of various models on CIFAR and ImageNet by a clear margin. Through further experiments, we attribute the effectiveness of ACB to its capability of enhancing the model's robustness to rotational distortions and strengthening the central skeleton parts of square convolution kernels.

Motivation & Objective

• Motivate architecture-neutral CNN improvements that can plug into existing models without manual redesign or hyper-parameter tuning.
• Propose Asymmetric Convolution Block (ACB) to enrich square kernels using parallel 3×3, 1×3, and 3×1 convolutions.
• Demonstrate that ACNet achieves higher accuracy on CIFAR and ImageNet and can be converted back to the original architecture with equivalent outputs.
• Investigate why ACBs improve performance, focusing on skeleton (central) kernel weights and robustness to rotational distortions.

Proposed method

• Replace each 3×3 convolution with an ACB containing 3×3, 1×3, and 3×1 branches whose outputs are summed.
• Train the ACNet end-to-end; fuse BN and branches to obtain an equivalent standard convolution for inference without extra cost.
• BN fusion and branch fusion steps to combine BN parameters into kernels and sum fused kernels into a single standard layer.
• Empirically validate across CIFAR-10/100 and ImageNet with baseline architectures (Plain, ResNet, DenseNet, AlexNet) across multiple models.

Experimental results

Research questions

• RQ1 Does replacing standard square-kernel layers with ACBs improve accuracy on common CNN benchmarks?
• RQ2 Can an ACNet be converted back to the original architecture without changing the inference cost or results?
• RQ3 To what extent do horizontal/vertical asymmetries and BN placement contribute to performance gains and robustness?
• RQ4 Are the gains attributable to strengthening the kernel skeletons, and how do perturbations to kernel locations affect performance?
• RQ5 Does ACNet improve robustness to rotational distortions (e.g., rotations, flips) in practice?

Key findings

• ACNet consistently improves Top-1 accuracy across CIFAR-10: e.g., Cifar-quick from 83.13 to 84.24, ResNet-56 from 94.31 to 95.09, WRN-16-8 from 95.56 to 96.15, DenseNet-40 from 94.29 to 94.84.
• On CIFAR-100, ACNet yields gains across several baselines (e.g., ResNet-56 from 73.58 to 74.04).
• On ImageNet, ACNet improves AlexNet by 1.52% (Top-1) and ResNet-18 by 0.78% (Top-1); DenseNet-121 gains 0.67% (Top-1).
• ABlation studies show removing any of horizontal/vertical BN in branches or BN placement reduces performance; horizontal kernels improve robustness to rotational distortions (e.g., 180° rotation) in ResNet-18.
• Skeleton weights (central kernel positions) are more critical than corner weights; ACNet further strengthens skeletons, contributing to performance gains.
• ACNet introduces no hyper-parameters, adds no inference-time cost, and can be implemented in PyTorch/TensorFlow; the training flow differs due to branch fusion dynamics.