[Paper Review] ME R-CNN: Multi-Expert Region-based CNN for Object Detection.

Hyungtae Lee, Sungmin Eum+1 more|arXiv (Cornell University)|Apr 4, 2017

Advanced Neural Network Applications17 citations

TL;DR

ME R-CNN proposes a multi-expert Region-based CNN for object detection that uses three specialized experts for horizontally elongated, square-like, and vertically elongated objects, leveraging a shared convolutional backbone and augmented training via exhaustive search to improve detection accuracy by capturing shape-specific priors and appearance variations.

ABSTRACT

Recent CNN-based object detection methods have drastically improved their performances but still use a single classifier as opposed to in categorizing objects. The main motivation of introducing multi-experts is twofold: i) to allow different experts to specialize in different fundamental object shape priors and ii) to better capture the appearance variations caused by different poses and viewing angles. The proposed approach, referred to as multi-expert Region-based CNN (ME R-CNN), consists of three experts each responsible for objects with particular shapes: horizontally elongated, square-like, and vertically elongated. Each expert is a network with multiple fully connected layers and all the experts are preceded by a shared network which consists of multiple convolutional layers. On top of using selective search which provides a compact, yet effective set of region of interests (RoIs) for object detection, we augmented the set by also employing the exhaustive search for training. Incorporating the exhaustive search can provide complementary advantages: i) it captures the multitude of neighboring RoIs missed by the selective search, and thus ii) provide significantly larger amount of training examples to achieve the enhanced accuracy.

Motivation & Objective

• To address the limitation of single-classifier object detectors in handling diverse object shapes and appearance variations.
• To improve detection performance by enabling specialized experts to focus on distinct shape priors: horizontally elongated, square-like, and vertically elongated objects.
• To enhance training data diversity and coverage by combining selective search with exhaustive search for region proposal generation.

Proposed method

• The model employs a shared convolutional network to extract features from region proposals, followed by three dedicated experts, each tailored to a specific shape category.
• Each expert network consists of multiple fully connected layers to classify objects based on shape-specific features.
• Selective search is used to generate initial region proposals, which are augmented with exhaustive search to capture additional neighboring regions.
• The exhaustive search provides complementary training examples by including regions missed by selective search, increasing training data diversity.
• The multi-expert design allows specialization, improving the model's ability to generalize across different object poses and viewing angles.
• Training is performed end-to-end with joint optimization across all experts and the shared backbone.

Experimental results

Research questions

• RQ1 Can specialized experts for distinct object shape priors improve object detection performance compared to a single unified classifier?
• RQ2 How does combining selective search with exhaustive search for region proposal generation affect detection accuracy and training data coverage?
• RQ3 To what extent do shape-specialized experts reduce errors in detecting objects with varying aspect ratios and poses?

Key findings

• The multi-expert design significantly improves detection accuracy by enabling specialization for different object shapes.
• Incorporating exhaustive search alongside selective search increases the number of training examples and enhances model generalization.
• The model achieves better performance on objects with challenging poses and viewing angles due to shape-specific feature learning.
• The combination of shape-specialized experts and augmented region proposals leads to improved detection mAP compared to single-expert baselines.