[Paper Review] Data-centric Artificial Intelligence: A Survey

Daochen Zha, Zaid Pervaiz Bhat+5 more|arXiv (Cornell University)|Mar 17, 2023

Data Quality and Management98 citations

TL;DR

A comprehensive survey defining data-centric AI, proposing a three-goal taxonomy (training data development, inference data development, data maintenance), and analyzing automation and human collaboration across tasks and benchmarks.

ABSTRACT

Artificial Intelligence (AI) is making a profound impact in almost every domain. A vital enabler of its great success is the availability of abundant and high-quality data for building machine learning models. Recently, the role of data in AI has been significantly magnified, giving rise to the emerging concept of data-centric AI. The attention of researchers and practitioners has gradually shifted from advancing model design to enhancing the quality and quantity of the data. In this survey, we discuss the necessity of data-centric AI, followed by a holistic view of three general data-centric goals (training data development, inference data development, and data maintenance) and the representative methods. We also organize the existing literature from automation and collaboration perspectives, discuss the challenges, and tabulate the benchmarks for various tasks. We believe this is the first comprehensive survey that provides a global view of a spectrum of tasks across various stages of the data lifecycle. We hope it can help the readers efficiently grasp a broad picture of this field, and equip them with the techniques and further research ideas to systematically engineer data for building AI systems. A companion list of data-centric AI resources will be regularly updated on https://github.com/daochenzha/data-centric-AI

Motivation & Objective

• Define data-centric AI and justify its need.
• Present a goal-driven taxonomy for data-centric AI tasks.
• Organize literature by automation level and human participation.
• Discuss challenges, benchmarks, and future opportunities in data-centric AI.

Proposed method

• Propose a goal-driven taxonomy organizing tasks into training data development, inference data development, and data maintenance.
• Categorize papers by automation vs. collaboration and assign automation levels or human participation degrees.
• Summarize representative tasks and methods for each sub-goal (e.g., data collection, labeling, preparation, reduction, augmentation, etc.).
• Analyze benchmarks and provide a global view across data lifecycle stages.
• Discuss future directions and open challenges in data-centric AI.

Experimental results

Research questions

• RQ1 RQ1: What are the necessary tasks to make AI data-centric?
• RQ2 RQ2: Why is automation significant for developing and maintaining data?
• RQ3 RQ3: In which cases and why is human participation essential in data-centric AI?
• RQ4 RQ4: What is the current progress of data-centric AI?

Key findings

• Provides a comprehensive overview of data-centric AI concepts, tasks, algorithms, challenges, and benchmarks.
• Introduces a goal-driven taxonomy aligning tasks with training data development, inference data development, and data maintenance.
• Introduces automation- and collaboration-oriented categorizations to map methods to human involvement.
• Discusses the need for data-centric approaches alongside model-centric methods as complementary.
• Covers a broad range of tasks from data collection to data maintenance and pipeline search.