[Paper Review] Explaining away ambiguity: Learning verb selectional preference with Bayesian networks

Massimiliano Ciaramita, Mark Johnson|ArXiv.org|Aug 22, 2000

Topic Modeling6 references5 citations

TL;DR

This paper proposes a Bayesian network model that learns verb selectional preferences by leveraging WordNet's lexical hierarchy and corpus-based verb-object pairs, using 'explaining away' to resolve word sense ambiguity. The model outperforms state-of-the-art unsupervised systems on word sense disambiguation, achieving 51.4% accuracy—surpassing Resnik's 44.3% and Abney and Light's 42.3%—by naturally modeling uncertainty through probabilistic inference.

ABSTRACT

This paper presents a Bayesian model for unsupervised learning of verb selectional preferences. For each verb the model creates a Bayesian network whose architecture is determined by the lexical hierarchy of Wordnet and whose parameters are estimated from a list of verb-object pairs found from a corpus. ``Explaining away'', a well-known property of Bayesian networks, helps the model deal in a natural fashion with word sense ambiguity in the training data. On a word sense disambiguation test our model performed better than other state of the art systems for unsupervised learning of selectional preferences. Computational complexity problems, ways of improving this approach and methods for implementing ``explaining away'' in other graphical frameworks are discussed.

Motivation & Objective

• To develop an unsupervised model for learning verb selectional preferences from untagged corpora.
• To address word sense ambiguity in training data by leveraging the 'explaining away' property of Bayesian networks.
• To improve selectional preference estimation by integrating lexical hierarchy (WordNet) with probabilistic inference.
• To evaluate the model on word sense disambiguation as a proxy for selectional preference accuracy.
• To explore the feasibility of implementing 'explaining away' in other graphical models beyond Bayesian networks.

Proposed method

• The model constructs a Bayesian network per verb, with nodes for semantic classes (synsets) from WordNet and evidence nodes for observed verb-object pairs.
• The network's structure is derived from WordNet's hyponymy hierarchy, ensuring semantic coherence in the model architecture.
• Parameters are estimated from corpus data using a modified frequency estimator that accounts for word sense ambiguity via Equation (3): $\hat{P}(c|p,r) \approx \sum_{w \in c} \frac{count(p,r,w)}{classes(w)}$.
• The 'explaining away' mechanism allows the model to reduce posterior probability of alternative senses when one sense is strongly supported by evidence, naturally resolving ambiguity.
• The model uses marginal probability inference to determine which semantic classes a verb selects for, based on observed object words and their WordNet senses.
• The system was evaluated using Resnik’s benchmark test set from the Brown corpus, with manual sense annotations.

Experimental results

Research questions

• RQ1 Can Bayesian networks effectively model verb selectional preferences in an unsupervised manner using untagged corpora?
• RQ2 How does the 'explaining away' property of Bayesian networks help resolve word sense ambiguity in selectional preference learning?
• RQ3 Does the proposed model outperform existing unsupervised systems on word sense disambiguation, a proxy for selectional preference accuracy?
• RQ4 To what extent does the model benefit from distributional information in the training data?
• RQ5 Can the 'explaining away' mechanism be generalized to other graphical models beyond Bayesian networks?

Key findings

• The model achieved a word sense disambiguation accuracy of 51.4% on the Resnik test set, outperforming Resnik’s 44.3% and Abney and Light’s 42.3%.
• The model performed better than state-of-the-art unsupervised systems despite using only binary presence/absence of words in the training data, not frequency counts.
• The model’s performance remained stable when distributional information (frequency counts) was fully integrated, indicating robustness to data sparsity.
• The model’s performance dropped to 35.8% on the San Jose Mercury Corpus, likely due to higher data noise and richer distributional statistics.
• The 'explaining away' mechanism was effective in resolving ambiguity by reducing posterior probabilities of competing senses when one sense was strongly supported.
• The authors suggest that 'explaining away' is a general property of Bayesian inference and may be implementable in other models, such as HMMs with priors.