[Paper Review] TransOMCS: From Linguistic Graphs to Commonsense Knowledge

Hongming Zhang, Daniel Khashabi+2 more|arXiv (Cornell University)|May 1, 2020

Topic Modeling23 references4 citations

TL;DR

This paper proposes TransOMCS, a method to mine high-quality commonsense knowledge from linguistic graphs using pattern matching and ranking, resulting in a knowledge graph two orders of magnitude larger than OMCS. The approach transfers linguistic patterns from ASER into ConceptNet-style tuples, achieving superior coverage and novelty in downstream NLP tasks while maintaining strong performance despite increased noise.

ABSTRACT

Commonsense knowledge acquisition is a key problem for artificial intelligence. Conventional methods of acquiring commonsense knowledge generally require laborious and costly human annotations, which are not feasible on a large scale. In this paper, we explore a practical way of mining commonsense knowledge from linguistic graphs, with the goal of transferring cheap knowledge obtained with linguistic patterns into expensive commonsense knowledge. The result is a conversion of ASER [Zhang et al., 2020], a large-scale selectional preference knowledge resource, into TransOMCS, of the same representation as ConceptNet [Liu and Singh, 2004] but two orders of magnitude larger. Experimental results demonstrate the transferability of linguistic knowledge to commonsense knowledge and the effectiveness of the proposed approach in terms of quantity, novelty, and quality. TransOMCS is publicly available at: https://github.com/HKUST-KnowComp/TransOMCS.

Motivation & Objective

• To address the scalability bottleneck of traditional commonsense knowledge acquisition, which relies on costly human annotations.
• To explore the transferability of linguistic patterns—derived from dependency and discourse relations—into high-quality commonsense knowledge.
• To construct a large-scale commonsense knowledge graph in the format of ConceptNet/OMCS, with improved coverage and novelty.
• To evaluate the effectiveness of the mined knowledge in intrinsic and extrinsic NLP tasks.

Proposed method

• The method uses a seed commonsense knowledge set (ConceptNet 5.5 subset) and linguistic graphs from ASER to identify overlapping terms and relations.
• For each seed tuple (h, r, t), supporting linguistic graphs containing h and t are selected, and linguistic patterns are extracted from their structural relations.
• A pattern filtering module selects high-precision patterns based on overlap with known commonsense facts and linguistic consistency.
• A discriminative ranking model evaluates and scores the quality of extracted commonsense tuples to filter low-quality or incomplete assertions.
• The final knowledge set, TransOMCS, is constructed in the same format as OMCS and ConceptNet, with 2 million+ edges.
• The framework leverages dependency parsing and discourse connectives to extract 37.9 million eventualities and 10 million discourse relations from raw text.

Experimental results

Research questions

• RQ1 Can linguistic patterns derived from dependency and discourse relations be effectively used to infer commonsense knowledge?
• RQ2 To what extent can linguistic knowledge transfer to improve the scale and novelty of commonsense knowledge resources?
• RQ3 How does the quality and coverage of TransOMCS compare to existing resources like OMCS, COMET, and LAMA?
• RQ4 Does the inclusion of TransOMCS significantly improve performance in downstream NLP tasks?

Key findings

• TransOMCS contains 2 million+ edges, representing a two-orders-of-magnitude increase over the original OMCS (600K edges).
• TransOMCS contributes 3.36 supporting tuples on average per example in dialogue generation, compared to 1.46 for OMCS, indicating significantly greater coverage and novelty.
• In commonsense reading comprehension, TransOMCS contributes 0.37 accuracy gain with the top 1% of knowledge, outperforming OMCS (0.21) and other methods like COMET and LAMA.
• The dialogue generation model using TransOMCS achieves higher BLEU scores, while adding COMET or LAMA provides no meaningful improvement, indicating TransOMCS offers more valuable novel knowledge.
• TransOMCS generates multi-token concepts and novel knowledge such as 'human' being capable of 'creating life', which other models fail to produce.
• Despite being noisier than OMCS, TransOMCS demonstrates strong utility in downstream tasks due to its scale and coverage, validating the transferability of linguistic patterns to commonsense knowledge.