[Paper Review] Mixup Regularization for Region Proposal based Object Detectors

Shahine Bouabid, Vincent Delaitre|arXiv (Cornell University)|Mar 4, 2020

Advanced Neural Network Applications43 references4 citations

TL;DR

This paper proposes a novel mixup regularization method for region proposal-based object detectors by applying linear interpolation of anchors and their associated labels across image pairs, enabling improved robustness and generalization with minimal computational overhead. The approach enhances detection performance, particularly on small objects, and demonstrates flatter, more stable feature representations in deep networks.

ABSTRACT

Mixup - a neural network regularization technique based on linear interpolation of labeled sample pairs - has stood out by its capacity to improve model's robustness and generalizability through a surprisingly simple formalism. However, its extension to the field of object detection remains unclear as the interpolation of bounding boxes cannot be naively defined. In this paper, we propose to leverage the inherent region mapping structure of anchors to introduce a mixup-driven training regularization for region proposal based object detectors. The proposed method is benchmarked on standard datasets with challenging detection settings. Our experiments show an enhanced robustness to image alterations along with an ability to decontextualize detections, resulting in an improved generalization power.

Motivation & Objective

• To extend mixup regularization—proven effective in image classification—to object detection, where bounding box interpolation is non-trivial.
• To address the challenge of defining meaningful mixups between object detectors' region proposals (anchors) with varying numbers, locations, and labels.
• To improve model robustness against adversarial perturbations, image alterations, and distribution shifts through a structured mixup strategy.
• To validate the method’s effectiveness on standard benchmarks (Pascal VOC, MS COCO) using the SSD architecture.
• To analyze how mixup affects feature space geometry, particularly through principal component analysis of hidden representations.

Proposed method

• The method performs mixup at the anchor level by linearly interpolating anchor coordinates and their corresponding labels (classification and regression) between two image pairs.
• For each training batch, a mixed image is created via weighted averaging of two input images, with the mixing coefficient λ drawn from a Beta(α, α) distribution.
• The model is trained using a combined loss: standard detection loss on original anchors and mixup-regularized loss on interpolated anchors, with backpropagation applied only to the mixed anchors' logits.
• The approach leverages the predefined anchor grid structure to ensure consistent mapping between mixed and original anchors, enabling valid regression and classification.
• Ablation studies isolate the effect of mixup by applying it only to specific anchor scales (e.g., only A₁), allowing analysis of scale-dependent performance.
• Feature space analysis uses PCA on penultimate layer features to compare explained variance between mixup and baseline models, assessing representation flattening.

Experimental results

Research questions

• RQ1 Can mixup regularization be meaningfully adapted to region proposal-based object detectors despite the non-linear and variable nature of bounding boxes?
• RQ2 Does anchor-wise mixup improve detection robustness to image-level perturbations and distribution shifts?
• RQ3 How does mixup affect the geometry of learned feature representations, particularly in terms of principal component variance?
• RQ4 Does mixup improve performance on small object detection, a known weakness of SSD?
• RQ5 Is the benefit of mixup consistent across different anchor scales, or is it more effective on certain feature levels?

Key findings

• The proposed mixup method improves mean average precision (mAP) by 0.7–1.3 points on Pascal VOC07, with a final mAP of 66.5% when applied to small anchors (A₁) only.
• Mixup significantly enhances robustness to image alterations and adversarial patch attacks, reducing model sensitivity to localized perturbations.
• PCA analysis shows that mixup flattens feature representations—especially at larger anchor scales (A₄–A₆)—by reducing the proportion of variance explained by the first principal component.
• When applied only to the smallest anchor level (A₁), mixup increases mAP by 0.7 points over baseline, indicating improved small-object detection capability.
• Applying mixup to larger anchors (A₄–A₆) degrades performance, suggesting that regularization effects are scale-dependent and must be carefully tuned.
• The method achieves improved generalization with negligible computational cost, as mixup is applied during training only and does not affect inference.