[Paper Review] Feature Engineering for Knowledge Base Construction.

Christopher Ré, Amir Abbas Sadeghian+5 more|arXiv (Cornell University)|Jan 1, 2014

Data Quality and Management39 references25 citations

TL;DR

This paper proposes a feature engineering-driven approach to knowledge base construction (KBC) using joint probabilistic inference and learning, enabling high-quality knowledge bases to be built efficiently—often by a single graduate student in less time than human experts require. The DeepDive system supports this by abstracting away algorithmic complexity, letting users focus on feature design, which the authors identify as a critical but understudied factor in end-to-end system quality.

ABSTRACT

Knowledge base construction (KBC) is the process of populating a knowledge base, i.e., a relational database together with inference rules, with information extracted from documents and structured sources. KBC blurs the distinction between two traditional database problems, information extraction and information integration. For the last several years, our group has been building knowledge bases with scientific collaborators. Using our approach, we have built knowledge bases that have comparable and sometimes better quality than those constructed by human volunteers. In contrast to these knowledge bases, which took experts a decade or more human years to construct, many of our projects are constructed by a single graduate student. Our approach to KBC is based on joint probabilistic inference and learning, but we do not see inference as either a panacea or a magic bullet: inference is a tool that allows us to be systematic in how we construct, debug, and improve the quality of such systems. In addition, inference allows us to construct these systems in a more loosely coupled way than traditional approaches. To support this idea, we have built the DeepDive system, which has the design goal of letting the user “think about features— not algorithms.” We think of DeepDive as declarative in that one specifies what they want but not how to get it. We describe our approach with a focus on feature engineering, which we argue is an understudied problem relative to its importance to end-to-end quality.

Motivation & Objective

• To address the underappreciated role of feature engineering in end-to-end knowledge base construction quality.
• To reduce the time and expertise required to build high-quality knowledge bases, compared to traditional human-curated approaches.
• To enable systematic, loosely coupled development of KBC systems through joint inference and learning.
• To shift the focus from algorithm design to feature specification, improving maintainability and quality.
• To demonstrate that feature engineering, not just inference, is central to achieving high performance in KBC.

Proposed method

• The approach uses joint probabilistic inference and learning to systematically construct and improve knowledge bases.
• The DeepDive system is designed to be declarative, allowing users to specify desired outputs without detailing algorithms.
• Feature engineering is prioritized as the primary mechanism for controlling system behavior and quality.
• The system supports loosely coupled component integration, reducing tight coupling between modules.
• Inference is used not as a magic solution but as a tool for debugging, improving, and validating system components.
• The framework abstracts away low-level algorithmic concerns, allowing developers to focus on feature design.

Experimental results

Research questions

• RQ1 How can feature engineering be leveraged to improve the quality of knowledge base construction systems?
• RQ2 To what extent can high-quality knowledge bases be built with minimal human expertise and time?
• RQ3 Can a declarative system that emphasizes features over algorithms achieve competitive performance in KBC?
• RQ4 How does feature engineering compare in importance to inference and learning in end-to-end KBC quality?
• RQ5 What role does systematic inference play in enabling scalable and maintainable KBC development?

Key findings

• Knowledge bases built using the proposed feature engineering approach achieve quality comparable to or better than those built by human volunteers.
• Many projects were constructed by a single graduate student, significantly reducing the human effort required compared to decade-long expert efforts.
• The DeepDive system successfully abstracts algorithmic complexity, allowing users to focus on features rather than implementation details.
• Joint probabilistic inference enables systematic development, debugging, and improvement of KBC systems.
• Feature engineering is identified as a critical but understudied component in KBC, with substantial impact on end-to-end system quality.
• The approach supports a loosely coupled architecture, improving maintainability and scalability of knowledge base construction pipelines.