[Paper Review] Weakly-supervised Discovery of Visual Pattern Configurations

Hyun Oh Song, Yong Jae Lee+2 more|arXiv (Cornell University)|Jun 25, 2014

Advanced Image and Video Retrieval TechniquesComputer Science31 references103 citations

TL;DR

This paper proposes a weakly-supervised method that discovers discriminative visual pattern configurations—combinations of non-overlapping, part-based patches—using constrained submodular optimization. By identifying co-occurring, spatially meaningful configurations of discriminative parts, the approach improves object detection accuracy on PASCAL VOC, achieving state-of-the-art results by generating informative hard negatives and reducing localization errors without strong annotations.

ABSTRACT

The increasing prominence of weakly labeled data nurtures a growing demand for object detection methods that can cope with minimal supervision. We propose an approach that automatically identifies discriminative configurations of visual patterns that are characteristic of a given object class. We formulate the problem as a constrained submodular optimization problem and demonstrate the benefits of the discovered configurations in remedying mislocalizations and finding informative positive and negative training examples. Together, these lead to state-of-the-art weakly-supervised detection results on the challenging PASCAL VOC dataset.

Motivation & Objective

• To address mislocalization in weakly-supervised object detection caused by reliance on single, often part-based, discriminative patches.
• To improve detection robustness by identifying frequent, spatially coherent configurations of visual patterns rather than isolated regions.
• To generate high-quality hard negative examples from detected configurations to refine detector training.
• To achieve state-of-the-art performance on PASCAL VOC using only image-level annotations, without instance-level bounding box supervision.

Proposed method

• Formulates the discovery of discriminative visual patterns as a constrained submodular optimization problem maximizing a submodular function under a matroid intersection constraint.
• Uses a discriminative covering formulation to identify patches that occur frequently in positive images but rarely in negative ones.
• Applies an independence constraint to avoid selecting overlapping or redundant patches, ensuring diverse, non-reduundant part configurations.
• Employs a binning scheme in transform space (translation, scale, aspect ratio) to group similar patches and identify co-occurring configurations.
• Constructs a graph $\mathcal{G}_P$ where nodes represent detected parts and edges represent co-occurrence frequency, with high-degree nodes indicating salient configurations.
• Uses the discovered configurations to generate hard negative examples by selecting regions with low IoU to the configuration, improving detector generalization.

Experimental results

Research questions

• RQ1 Can we improve weakly-supervised object detection by discovering configurations of multiple discriminative visual patterns instead of relying on single patches?
• RQ2 How can we ensure that selected visual patterns are non-overlapping and non-redundant while still being spatially coherent?
• RQ3 To what extent do discovered configurations lead to better spatial coverage of full objects compared to single-patch detectors?
• RQ4 Can configurations be used to generate more informative hard negative examples than conventional methods?
• RQ5 Does the proposed method achieve state-of-the-art performance on PASCAL VOC using only image-level supervision?

Key findings

• The method achieves state-of-the-art mean average precision (mAP) of 24.6% on the PASCAL VOC 2007 test set using a linear SVM, outperforming prior weakly-supervised methods.
• On the person class, the method achieves 21.2% mAP with SVM and 14.8% with LSVM, showing significant improvement over the previous best of 9.1%.
• The inclusion of discovered hard negatives improves mAP by 0.9% (SVM) and 0.9% (LSVM) over using neighboring negative regions, demonstrating their informativeness.
• The method consistently discovers meaningful configurations such as face-torso for people, wheel-body for bicycles, and window-frame for cars, as shown in qualitative results.
• The approach reduces mislocalization by covering more of the full object through combined, non-overlapping part configurations.
• The matroid intersection constraint effectively prevents redundant patch selection, enabling discovery of diverse, spatially meaningful part combinations.