[Paper Review] Prepositions in Context

Hongyu Gong, Jiaqi Mu+2 more|arXiv (Cornell University)|Feb 5, 2017

Natural Language Processing Techniques33 references3 citations

TL;DR

This paper proposes a resource-independent preposition sense disambiguation (PSD) method that leverages word vector geometry around prepositions to cluster context features, achieving state-of-the-art performance on SemEval 2007 and OEC datasets. It further learns sense-specific preposition embeddings that capture semantic relations and enable paraphrasing of phrasal verbs via simple vector operations.

ABSTRACT

Prepositions are highly polysemous, and their variegated senses encode significant semantic information. In this paper we match each preposition's complement and attachment and their interplay crucially to the geometry of the word vectors to the left and right of the preposition. Extracting such features from the vast number of instances of each preposition and clustering them makes for an efficient preposition sense disambigution (PSD) algorithm, which is comparable to and better than state-of-the-art on two benchmark datasets. Our reliance on no external linguistic resource allows us to scale the PSD algorithm to a large WikiCorpus and learn sense-specific preposition representations -- which we show to encode semantic relations and paraphrasing of verb particle compounds, via simple vector operations.

Motivation & Objective

• To develop a preposition sense disambiguation (PSD) algorithm that does not rely on external linguistic tools like dependency parsers or POS taggers.
• To explore whether context-dependent features derived from word vector geometry can effectively capture prepositional polysemy and enable sense clustering.
• To learn sense-specific preposition representations from large-scale corpora using the disambiguation results, and evaluate their utility in capturing semantic relations.
• To investigate whether sense-specific preposition embeddings can improve paraphrasing of phrasal verbs through simple vector operations.
• To validate the effectiveness of sense-specific representations in intrinsic evaluation tasks and compare them with monosemous global representations.

Proposed method

• Extract context features around each preposition instance using the left and right word vectors, forming a feature vector (ℓ, r) for each occurrence.
• Use k-means clustering on the extracted (ℓ, r) features to group instances into sense clusters, enabling unsupervised PSD.
• For supervised learning, train an SVM or MLP on the (ℓ, r) features with sense labels, using weighted k-NN to analyze feature importance.
• Leverage the disambiguated sense labels to retrain preposition embeddings using Word2Vec, but with a reduced context window centered on the preposition.
• Apply a simple additive composition model (e.g., 'in' + 'America' ≈ 'American') to test whether sense-specific embeddings encode semantic relations.
• Use the VPC (verb particle compound) dataset to evaluate paraphrasing performance by comparing vector similarity between phrasal verbs and their paraphrases.

Experimental results

Research questions

• RQ1 Can preposition sense disambiguation be effectively achieved using only word vector geometry and no external linguistic resources?
• RQ2 How do the left and right context vectors of a preposition contribute to sense discrimination, and which combinations (ℓ, r, ℓ+r, etc.) yield optimal performance?
• RQ3 Do sense-specific preposition embeddings capture semantic relations better than global embeddings, as evidenced by analogical reasoning tasks?
• RQ4 Can sense-specific preposition embeddings be used to generate paraphrases of phrasal verbs through simple vector addition?
• RQ5 What is the impact of sense granularity and training data quality on the performance of sense-specific preposition representations?

Key findings

• The proposed unsupervised PSD method achieved an accuracy of 0.584 on the SemEval 2007 dataset, surpassing the state-of-the-art (0.56) and outperforming prior methods that rely on linguistic tools.
• On the OEC dataset, the supervised model using (ℓ, r, i) features achieved an accuracy of 0.351 with an MLP, significantly outperforming baseline models and demonstrating the value of integrated context features.
• Sense-specific preposition embeddings enabled the analogical relationship 'in' + 'America' ≈ 'American', a relation that global preposition embeddings failed to capture.
• Using sense-specific 'for' embeddings, the model generated 'defend' as a top-3 paraphrase of 'fight for', a result not achieved with global or simplex representations.
• The weighted k-NN analysis revealed that prepositions like 'behind', 'to', and 'with' are more influenced by the verb they attach to, while 'during' and 'on' are more dependent on their complement.
• Sense-specific representations significantly outperformed monosemous representations in intrinsic evaluations, confirming their ability to encode richer semantic content.