[Paper Review] APAC: Augmented PAttern Classification with Neural Networks

Ikuro Sato, Hiroki Nishimura+1 more|arXiv (Cornell University)|May 13, 2015

Advanced Neural Network ApplicationsComputer Science19 references108 citations

TL;DR

This paper proposes APAC (Augmented PAttern Classification), a novel decision rule for neural network classifiers trained with data augmentation. Instead of single-feedforward prediction, APAC aggregates softmax outputs from multiple augmented versions of a test sample, significantly improving generalization. It achieves state-of-the-art performance, including 0.23% test error on MNIST (best non-ensemble result) and surpasses some CNNs on CIFAR-10 using only a multilayer perceptron.

ABSTRACT

Deep neural networks have been exhibiting splendid accuracies in many of visual pattern classification problems. Many of the state-of-the-art methods employ a technique known as data augmentation at the training stage. This paper addresses an issue of decision rule for classifiers trained with augmented data. Our method is named as APAC: the Augmented PAttern Classification, which is a way of classification using the optimal decision rule for augmented data learning. Discussion of methods of data augmentation is not our primary focus. We show clear evidences that APAC gives far better generalization performance than the traditional way of class prediction in several experiments. Our convolutional neural network model with APAC achieved a state-of-the-art accuracy on the MNIST dataset among non-ensemble classifiers. Even our multilayer perceptron model beats some of the convolutional models with recently invented stochastic regularization techniques on the CIFAR-10 dataset.

Motivation & Objective

• To address the suboptimal decision rule used in standard classification after data augmentation training.
• To improve generalization performance by rethinking how predictions are made at inference time for augmented data.
• To demonstrate that optimal decision-making under data augmentation requires expectation maximization over virtual samples, not single-sample inference.
• To show that even simple models like MLPs can achieve high performance when combined with the proposed APAC inference rule.

Proposed method

• APAC computes the product of softmax probabilities across multiple data-augmented versions of a single test input.
• Each augmented version is generated using the same deformation functions applied during training (e.g., elastic distortions, color shifts).
• The final prediction is made by selecting the class with the highest product of probabilities across all augmented samples.
• This approach approximates the expectation of the loss function over deformation parameters, aligning inference with the training objective.
• The method is applied to both CNN and MLP architectures without architectural changes, only modifying the inference step.
• The use of product (not sum) of probabilities is justified as maximizing the joint likelihood of class predictions across augmented instances.

Experimental results

Research questions

• RQ1 Does using a single feedforward pass for test prediction lead to suboptimal performance when models are trained with data augmentation?
• RQ2 Can improving the decision rule at inference time significantly enhance generalization without architectural changes?
• RQ3 Is the product of softmax outputs across augmented samples more effective than sum or max for decision-making?
• RQ4 Can a simple MLP outperform complex CNNs when using the proposed APAC inference rule?
• RQ5 How does APAC affect the learned feature representations in the network weights?

Key findings

• APAC achieved a test error rate of 0.23% on the MNIST dataset, the best result among non-ensemble classifiers.
• The top-2 prediction error rate using APAC on MNIST was only 0.01%, indicating high confidence in predictions.
• On CIFAR-10, the MLP model using APAC outperformed several CNN models trained with advanced stochastic regularization techniques.
• Weight maps from APAC-trained models showed enhanced local-feature sensitivity, especially in MLPs, indicating improved invariance and reduced overfitting.
• The product of softmax outputs across augmented samples yielded better performance than the sum in all cases except one (CIFAR-10-MLP), supporting its use for joint probability maximization.
• APAC improved generalization even when the model architecture was not designed for local feature extraction, such as in standard MLPs.