[Paper Review] Unsupervised Transfer of Semantic Role Models from Verbal to Nominal Domain

Yanpeng Zhao, Ivan Titov|arXiv (Cornell University)|May 1, 2020

Natural Language Processing Techniques36 references4 citations

TL;DR

This paper proposes a variational autoencoder (VAE)-based method to transfer semantic role labeling from verbal to nominal domains without requiring labeled nominal data. By leveraging shared selectional preferences between verbal and nominal predicates, the model uses a joint encoder-decoder framework with shared decoder parameters to predict roles in nominal constructions, achieving state-of-the-art performance on CoNLL-2009, outperforming supervised and unsupervised baselines by up to 4.27% F1.

ABSTRACT

Semantic role labeling (SRL) is an NLP task involving the assignment of predicate arguments to types, called semantic roles. Though research on SRL has primarily focused on verbal predicates and many resources available for SRL provide annotations only for verbs, semantic relations are often triggered by other linguistic constructions, e.g., nominalizations. In this work, we investigate a transfer scenario where we assume role-annotated data for the source verbal domain but only unlabeled data for the target nominal domain. Our key assumption, enabling the transfer between the two domains, is that selectional preferences of a role (i.e., preferences or constraints on the admissible arguments) do not strongly depend on whether the relation is triggered by a verb or a noun. For example, the same set of arguments can fill the Acquirer role for the verbal predicate `acquire' and its nominal form `acquisition'. We approach the transfer task from the variational autoencoding perspective. The labeler serves as an encoder (predicting role labels given a sentence), whereas selectional preferences are captured in the decoder component (generating arguments for the predicting roles). Nominal roles are not labeled in the training data, and the learning objective instead pushes the labeler to assign roles predictive of the arguments. Sharing the decoder parameters across the domains encourages consistency between labels predicted for both domains and facilitates the transfer. The method substantially outperforms baselines, such as unsupervised and `direct transfer' methods, on the English CoNLL-2009 dataset.

Motivation & Objective

• To address the scarcity of annotated semantic role data for nominal predicates, which are common in NLP applications like event extraction and summarization.
• To enable transfer of semantic role models from labeled verbal domains to unlabeled nominal domains, overcoming the lack of annotated nominal data.
• To exploit the hypothesis that selectional preferences of semantic roles remain consistent across verbal and nominal forms (e.g., 'acquire' and 'acquisition').
• To develop a semi-supervised VAE framework that jointly models role prediction and argument generation, with shared decoder parameters across domains to enforce consistency.
• To outperform existing unsupervised and direct-transfer baselines on nominal SRL using the CoNLL-2009 dataset.

Proposed method

• Uses a variational autoencoder (VAE) framework where the labeler acts as an encoder predicting semantic roles from input sentences.
• Employs a shared decoder component that generates arguments based on predicted roles, capturing selectional preferences across domains.
• Leverages the evidence lower bound (ELBO) objective to train the model, with the labeler optimized to predict roles that are predictive of observed arguments in the nominal domain.
• Introduces data augmentation via synthetic verbal examples to improve generalization, especially when labeled verbal data is limited.
• Applies reparameterization tricks to stabilize training and optimize the VAE objective using stochastic gradient descent.
• Shares decoder parameters between verbal and nominal domains to enforce consistency in selectional preferences and enable effective domain transfer.

Experimental results

Research questions

• RQ1 Can semantic role labeling models trained on verbal predicates be effectively transferred to nominal predicates without any labeled nominal data?
• RQ2 To what extent do selectional preferences of semantic roles remain consistent between verbal and nominal constructions?
• RQ3 Does a shared VAE-based architecture with joint modeling of roles and arguments improve unsupervised nominal SRL compared to baseline methods?
• RQ4 How do components like the latent code, joint argument modeling, and data augmentation affect model performance and training stability?
• RQ5 Can the model generalize to low-resource settings where only limited labeled verbal data is available?

Key findings

• The proposed VAE-based model achieves 74.80% F1 on the CoNLL-2009 nominal development set, outperforming the strongest baseline (Most-frequent) by 4.27% F1.
• The ablation study confirms that the latent code, joint argument modeling, and data augmentation are all essential for optimal performance, with the full model showing the most stable training.
• Model accuracy increases consistently with more labeled verbal examples, indicating scalability and data efficiency.
• The model significantly outperforms unsupervised baselines such as SyntFun and Arg2vec, and surpasses even the supervised Factorization baseline in the unsupervised setting.
• The direct transfer method, which applies verbal models directly to nominal data, performs worse than the proposed method, highlighting the importance of domain-specific adaptation.
• The results validate the core assumption that selectional preferences of semantic roles are largely invariant across verbal and nominal forms, enabling effective transfer.