[Paper Review] A Branching and Merging Convolutional Network with Homogeneous Filter Capsules

Adam Byerly, Tatiana Kalganova|arXiv (Cornell University)|Jan 24, 2020

Music and Audio Processing26 references47 citations

TL;DR

This paper proposes a novel convolutional neural network with branching and merging pathways that transform final convolutional filters into homogeneous vector capsules, enabling improved feature extraction across varying receptive fields and abstraction levels. The method achieves a new state of the art on MNIST with 99.84% accuracy using an ensemble and 99.79% for a single model, while reducing parameters and training epochs by 75% compared to prior capsule networks.

ABSTRACT

We present a convolutional neural network design with additional branches after certain convolutions so that we can extract features with differing effective receptive fields and levels of abstraction. From each branch, we transform each of the final filters into a pair of homogeneous vector capsules. As the capsules are formed from entire filters, we refer to them as filter capsules. We then compare three methods for merging the branches--merging with equal weight and merging with learned weights, with two different weight initialization strategies. This design, in combination with a domain-specific set of randomly applied augmentation techniques, establishes a new state of the art for the MNIST dataset with an accuracy of 99.84% for an ensemble of these models, as well as establishing a new state of the art for a single model (99.79% accurate). These accuracies were achieved with a 75% reduction in both the number of parameters and the number of epochs of training relative to the previously best performing capsule network on MNIST. All training was performed using the Adam optimizer and experienced no overfitting.

Motivation & Objective

• To improve feature representation in convolutional networks by introducing branching pathways after key convolutions.
• To enable multi-scale and multi-level abstraction through filter capsules derived from entire convolutional filters.
• To reduce model complexity and training cost while maintaining or improving performance on MNIST.
• To evaluate different merging strategies—equal-weighted and learned-weighted—with distinct initialization methods for optimal performance.

Proposed method

• The architecture introduces additional branches after specific convolutional layers to extract features with differing effective receptive fields.
• Each branch's final convolutional filter is transformed into a pair of homogeneous vector capsules, referred to as filter capsules.
• Three merging strategies are evaluated: simple averaging (equal weights), learned weights with random initialization, and learned weights with Xavier initialization.
• The network uses the Adam optimizer and applies domain-specific data augmentation techniques to prevent overfitting.
• The capsule transformation preserves spatial and hierarchical information by encoding entire filter outputs as vector representations.
• The model is trained end-to-end with no overfitting observed, even with aggressive data augmentation.

Experimental results

Research questions

• RQ1 Can branching pathways after convolutional layers improve feature abstraction and model accuracy on MNIST?
• RQ2 How do different merging strategies—equal-weighted versus learned-weighted—impact performance when combining features from multiple branches?
• RQ3 To what extent can parameter count and training epochs be reduced without sacrificing accuracy using this capsule-based architecture?
• RQ4 Does the use of filter capsules derived from full convolutional filters enhance representation learning compared to standard feature maps?
• RQ5 Can domain-specific data augmentation combined with this architecture further boost performance while maintaining generalization?

Key findings

• The proposed model achieves a new state of the art on the MNIST dataset with an ensemble accuracy of 99.84%.
• A single model reaches 99.79% accuracy, setting a new SOTA for individual models on MNIST.
• The method reduces both the number of parameters and the number of training epochs by 75% compared to the previous best capsule network on MNIST.
• The model shows no signs of overfitting despite extensive data augmentation and high capacity.
• Learned-weight merging with Xavier initialization outperformed other merging strategies, indicating the importance of proper weight initialization in capsule fusion.
• The use of filter capsules enables effective hierarchical feature representation while maintaining computational efficiency.