[論文レビュー] Group Equivariant Convolutional Networks

Taco Cohen, Max Welling|arXiv (Cornell University)|Feb 24, 2016

Advanced Neural Network Applications参考文献 41被引用数 584

ひとこと要約

この論文はCNNを、離散対称群（p4 や p4m のような群）に対するグループ同変性畳み込み（G-畳み込み）へ一般化し、追加パラメータなしで性能を改善し、回転させた MNIST および CIFAR-10 において最先端の結果を達成します（最小限のオーバーヘッド）。

ABSTRACT

We introduce Group equivariant Convolutional Neural Networks (G-CNNs), a natural generalization of convolutional neural networks that reduces sample complexity by exploiting symmetries. G-CNNs use G-convolutions, a new type of layer that enjoys a substantially higher degree of weight sharing than regular convolution layers. G-convolutions increase the expressive capacity of the network without increasing the number of parameters. Group convolution layers are easy to use and can be implemented with negligible computational overhead for discrete groups generated by translations, reflections and rotations. G-CNNs achieve state of the art results on CIFAR10 and rotated MNIST.

研究の動機と目的

• Motivate and formalize the use of symmetry groups to improve data efficiency in deep networks.
• Introduce G-convolutions and show they share more weight‑sharing than standard convolutions while preserving expressivity.
• Demonstrate that G-CNNs achieve state-of-the-art results on rotated MNIST and CIFAR-10 with negligible computational overhead.
• Provide practical implementation guidance for G-convolutions and pooling for discrete groups.
• Discuss extensions and future directions for structured representations in neural networks.

提案手法

• Define group actions and functions on groups to model feature maps as G-spaces.
• Introduce G-convolution (G-correlation) that operates over a group G, ensuring equivariance under G transformations.
• Show that nonlinearities and pooling can be designed to preserve equivariance (including coset pooling for G/H).
• Provide efficient implementation via filter transformation and planar convolution, enabling fast G-convolutions for split groups like p4 and p4m.
• Demonstrate that standard network components (batch norm, residual blocks) remain equivariant within G-CNNs.

実験結果

リサーチクエスチョン

• RQ1 Can CNNs be generalized to be equivariant to larger symmetry groups beyond translations (e.g., rotations and reflections) without increasing parameter count?
• RQ2 Do G-convolutions, G-pooling, and nonlinearities preserve equivariance across layers, enabling deep G-CNNs?
• RQ3 Do G-CNNs yield practical performance gains on rotated or augmented datasets (MNIST-rot, CIFAR-10) compared to standard CNNs?
• RQ4 What are the practical considerations and efficiency costs when implementing G-convolutions for groups like p4 and p4m?
• RQ5 What extensions and future directions exist for applying G-CNNs to other groups or higher dimensions?

主な発見

• G-convolutions for discrete groups (p4, p4m) share substantially more weight across transformations, increasing expressive capacity without adding parameters.
• G-CNNs achieve state-of-the-art results on rotated MNIST: P4CNN reaches 2.28% test error, outperforming prior methods.
• On CIFAR-10 and augmented CIFAR-10+, G-CNNs with p4m yield strong results: ResNet44 with p4m achieves 6.46% and 4.94% respectively, and a wider 26-layer p4m network reaches 4.19% with augmentation (compared to 5.27% planar).
• Replacing planar convolutions with G-convolutions provides consistent improvements without extensive tuning.
• G-CNNs benefit from data augmentation similarly to standard CNNs, with improvements observed even when the dataset is not perfectly symmetric.