[Paper Review] Cardinal Virtues: Extracting Relation Cardinalities from Text

Paramita Mirza, Simon Razniewski+2 more|arXiv (Cornell University)|Apr 14, 2017

Topic Modeling13 references4 citations

TL;DR

This paper introduces the novel task of extracting relation cardinalities—such as how many children a person has or how many medals an athlete won—from text using distant supervision with conditional random fields (CRF). The method achieves precision between 3% and 55%, depending on relation complexity, and addresses challenges like linguistic variance, compositionality, and incomplete knowledge base training data.

ABSTRACT

Information extraction (IE) from text has largely focused on relations between individual entities, such as who has won which award. However, some facts are never fully mentioned, and no IE method has perfect recall. Thus, it is beneficial to also tap contents about the cardinalities of these relations, for example, how many awards someone has won. We introduce this novel problem of extracting cardinalities and discusses the specific challenges that set it apart from standard IE. We present a distant supervision method using conditional random fields. A preliminary evaluation results in precision between 3% and 55%, depending on the difficulty of relations.

Motivation & Objective

• To address the gap in information extraction (IE) by introducing the task of extracting relation cardinalities, such as the number of children or medals associated with a subject.
• To improve knowledge base (KB) completeness and accuracy by capturing numeric counts of relations that standard IE methods overlook.
• To overcome challenges in training data quality due to incomplete KBs, compositionality of multiple cardinality mentions, and linguistic variance in number expressions.
• To develop a robust method that handles not only cardinal numbers but also ordinals, negations, and quantifiers (e.g., 'twins', 'only child') as proxies for cardinality.
• To enhance KB curation and question answering by enabling systems to reason about quantities like 'most medals' or 'only child' from unstructured text.

Proposed method

• Uses distant supervision by leveraging triple counts from existing knowledge bases (e.g., Wikidata) as weak supervision signals for training.
• Applies conditional random fields (CRF) to model sequential patterns of numeric expressions and their syntactic context for cardinality extraction.
• Implements filtering strategies to improve training data quality, such as using only high-popularity entities to reduce KB incompleteness bias.
• Proposes a multi-step approach to handle compositionality: aggregating multiple cardinal numbers (e.g., sons + daughters) and learning composition rules (e.g., children = sons + daughters).
• Introduces linguistic transformation techniques to convert non-numerical expressions (e.g., 'twins', 'only child', 'never married') into numeric cardinalities.
• Explores lower-bound inference from ordinal expressions (e.g., 'his fourth child') to estimate minimum cardinalities when exact counts are not stated.

Experimental results

Research questions

• RQ1 Can relation cardinalities be reliably extracted from text using distant supervision, despite incomplete or inaccurate knowledge base counts?
• RQ2 How do linguistic variations—such as ordinals, negations, or quantifiers like 'twins'—affect the accuracy of cardinality extraction?
• RQ3 To what extent can compositionality (e.g., summing multiple mentions like 'two sons and three daughters') be modeled to improve cardinality inference?
• RQ4 Can training data quality be improved by filtering for high-popularity entities or relaxing equality constraints in distant supervision?
• RQ5 How effective are linguistic transformations (e.g., converting 'only child' to '1 child') in boosting recall for cardinalities 0 and 1?

Key findings

• The CRF-based method achieves a precision range of 3% to 55%, depending on the complexity and linguistic expression of the relation cardinality.
• Manual evaluation shows that Wikidata’s triple count for the 'hasChild' relation is 46% lower than the actual number asserted in text, indicating significant KB incompleteness.
• Filtering training data to include only high-popularity entities significantly improves model performance, suggesting that data quality outweighs quantity.
• Approximately 16% of false positives in child cardinality extraction stem from failure to handle compositional mentions (e.g., 'two sons and one daughter'), highlighting the need for aggregation techniques.
• Only 4% of spouse relation cardinalities are expressed using cardinal numbers; instead, ordinals ('first wife') or indefinite articles ('a wife') are more common, indicating the need for lower-bound inference.
• Linguistic transformation of non-numerical expressions (e.g., 'twins' → '2', 'never married' → '0') is a viable path to improve recall for cardinalities 0 and 1, though not yet fully evaluated in the current setup.