[Paper Review] Batch-Instance Normalization for Adaptively Style-Invariant Neural Networks

Hyeonseob Nam, Hyoeun Kim|arXiv (Cornell University)|May 21, 2018

Generative Adversarial Networks and Image Synthesis27 references124 citations

TL;DR

BIN adaptively balances batch and instance normalization to selectively preserve useful styles and normalize disturbing ones, improving recognition across classification, multi-domain learning, and style transfer.

ABSTRACT

Real-world image recognition is often challenged by the variability of visual styles including object textures, lighting conditions, filter effects, etc. Although these variations have been deemed to be implicitly handled by more training data and deeper networks, recent advances in image style transfer suggest that it is also possible to explicitly manipulate the style information. Extending this idea to general visual recognition problems, we present Batch-Instance Normalization (BIN) to explicitly normalize unnecessary styles from images. Considering certain style features play an essential role in discriminative tasks, BIN learns to selectively normalize only disturbing styles while preserving useful styles. The proposed normalization module is easily incorporated into existing network architectures such as Residual Networks, and surprisingly improves the recognition performance in various scenarios. Furthermore, experiments verify that BIN effectively adapts to completely different tasks like object classification and style transfer, by controlling the trade-off between preserving and removing style variations. BIN can be implemented with only a few lines of code using popular deep learning frameworks.

Motivation & Objective

• Motivate the problem of style variations impairing real-world recognition tasks.
• Propose Batch-Instance Normalization (BIN) to selectively normalize styles per feature map.
• Show BIN improves performance on classification, multi-domain learning, and style transfer.
• Demonstrate BIN’s scalability across architectures and tasks.

Proposed method

• Define BIN as a weighted combination of BN and IN with a learnable gate vector ρ ∈ [0,1]^C per channel.
• Compute BIN output y = (ρ ⊙ x̂(B) + (1−ρ) ⊙ x̂(I)) γ + β, where x̂(B) and x̂(I) are BN and IN normalized features.
• Train ρ with gradient updates and clip to [0,1], often with a higher learning rate to amplify the BN−IN difference.
• Initialize ρ = 1 and learn γ, β for affine transformation.
• Demonstrate BIN can replace BN/IN in existing architectures with minimal added parameters.
• Evaluate BIN on object classification, multi-domain learning, and image style transfer.

Experimental results

Research questions

• RQ1 Can BIN outperform standard BN on generic object classification datasets (CIFAR-10/100, ImageNet) by selectively normalizing styles?
• RQ2 Does BIN improve performance in multi-domain learning and domain adaptation settings?
• RQ3 Is BIN effective as an alternative to IN for image style transfer while preserving useful styles?
• RQ4 How do learned ρ gates distribute across layers and tasks, and how does BIN scale across architectures?

Key findings

• BIN outperforms BN on CIFAR-10/100 and ImageNet (Top-1 accuracy: CIFAR-10 93.72→94.29, CIFAR-100 74.26→75.88, ImageNet 69.89→70.68).
• BIN’s gate values tend to be bimodal, with many channels leaning toward BN (ρ≈1) in upper layers and IN (ρ≈0) in lower layers for classification.
• BIN consistently improves performance across diverse architectures (AlexNet, VGG, ResNet, ResNeXt, DenseNet) on CIFAR-100.
• In Office-Home multi-domain classification, BIN improves average accuracy over BN (80.08 vs 78.95).
• In domain-adaptation with DANN on Office-Home, BIN matches or slightly outperforms BN on most transfer tasks (11/12).
• For style transfer, BIN achieves better content preservation than BN+IN and comparable/style-transfer quality to IN, while mitigating loss of desired styles.