[Paper Review] Towards Automatic Construction of Diverse, High-quality Image Dataset

Yazhou Yao, Jian Zhang+5 more|arXiv (Cornell University)|Aug 22, 2017

Advanced Image and Video Retrieval Techniques43 references3 citations

TL;DR

This paper proposes a web-based, automatic framework for constructing diverse, high-quality image datasets using multiple textual queries and multi-view, multi-instance learning to filter noisy images and queries. It achieves superior performance in image classification, cross-dataset generalization, and object detection compared to existing weakly and web-supervised methods, with a publicly released 100-category dataset (WSID-100).

ABSTRACT

The availability of labeled image datasets has been shown critical for high-level image understanding, which continuously drives the progress of feature designing and models developing. However, constructing labeled image datasets is laborious and monotonous. To eliminate manual annotation, in this work, we propose a novel image dataset construction framework by employing multiple textual queries. We aim at collecting diverse and accurate images for given queries from the Web. Specifically, we formulate noisy textual queries removing and noisy images filtering as a multi-view and multi-instance learning problem separately. Our proposed approach not only improves the accuracy but also enhances the diversity of the selected images. To verify the effectiveness of our proposed approach, we construct an image dataset with 100 categories. The experiments show significant performance gains by using the generated data of our approach on several tasks, such as image classification, cross-dataset generalization, and object detection. The proposed method also consistently outperforms existing weakly supervised and web-supervised approaches.

Motivation & Objective

• Address the limitations of manual dataset construction, which is labor-intensive and slow to scale.
• Overcome the scalability, accuracy, and diversity challenges in web-based image dataset creation using image search engines.
• Reduce dependency on manual annotation by leveraging multiple textual queries to improve image collection diversity and quality.
• Develop a unified framework that jointly filters inter-class and intra-class noisy images while preserving dataset diversity.
• Create a benchmark dataset (WSID-100) to evaluate weakly supervised and web-supervised learning algorithms in computer vision.

Proposed method

• Use multiple textual queries per category to enhance image collection scalability and diversity, reducing dataset bias.
• Apply a multi-view, multi-instance learning (MIL) formulation to jointly filter inter-class and intra-class noisy images through a linear programming approach.
• Classify and filter noisy textual queries by identifying semantically rich, relevant queries and removing visually non-salient or irrelevant ones.
• Categorize image noise into three types—inter-class, intra-class, and query-specific—and apply tailored filtering strategies for each.
• Leverage visual clustering and instance-level classification within bags (clusters) to improve image re-ranking and noise suppression.
• Use 3-fold cross-validation to tune key parameters (e.g., $S_i$, $ abla$) for optimal recall and precision in noise filtering.

Experimental results

Research questions

• RQ1 Can multiple textual queries significantly improve the diversity and scalability of automatically collected image datasets compared to single-query methods?
• RQ2 How effective is a joint multi-view and multi-instance learning approach in filtering both inter-class and intra-class noisy images while preserving dataset diversity?
• RQ3 To what extent does the proposed framework outperform existing weakly supervised and web-supervised methods in downstream tasks like image classification and object detection?
• RQ4 How robust is the framework to hyperparameter variations, particularly in the noise filtering and query selection stages?
• RQ5 Can the resulting dataset (WSID-100) serve as a reliable benchmark for evaluating weakly supervised learning algorithms in computer vision?

Key findings

• The proposed framework significantly improves image classification accuracy and cross-dataset generalization performance compared to manually labeled and web-supervised baselines.
• The WSID-100 dataset, constructed using the proposed method, achieves state-of-the-art performance on object detection tasks when used for pre-training on the VOC 2007 dataset.
• The method achieves 98.2% average accuracy in inter-class noise filtering when $ abla = 10^0$, demonstrating high robustness and effectiveness.
• The framework maintains high recall (0.6) and acceptable precision by selecting $S_i = 0.6$, balancing coverage and quality in image selection.
• The dataset is robust to parameter variations, with filtering accuracy remaining above 96% across a wide range of $ abla$ values.
• The framework outperforms iterative methods by jointly optimizing diversity and accuracy, avoiding the diversity loss common in sequential filtering approaches.