[Paper Review] Chinese Event Extraction Using DeepNeural Network with Word Embedding

Yandi Xia, Yang Liu|arXiv (Cornell University)|Oct 4, 2016

Natural Language Processing Techniques3 citations

TL;DR

This paper proposes a deep neural network (DNN) model with word embeddings for Chinese event trigger identification, eliminating the need for manual feature engineering. By leveraging large-scale unlabeled text to pre-train word embeddings, the model achieves superior performance—especially in recall—outperforming traditional feature-based systems, even when using only automatically generated named entities.

ABSTRACT

A lot of prior work on event extraction has exploited a variety of features to represent events. Such methods have several drawbacks: 1) the features are often specific for a particular domain and do not generalize well; 2) the features are derived from various linguistic analyses and are error-prone; and 3) some features may be expensive and require domain expert. In this paper, we develop a Chinese event extraction system that uses word embedding vectors to represent language, and deep neural networks to learn the abstract feature representation in order to greatly reduce the effort of feature engineering. In addition, in this framework, we leverage large amount of unlabeled data, which can address the problem of limited labeled corpus for this task. Our experiments show that our proposed method performs better compared to the system using rich language features, and using unlabeled data benefits the word embeddings. This study suggests the potential of DNN and word embedding for the event extraction task.

Motivation & Objective

• To investigate whether deep neural networks and word embeddings can reduce reliance on handcrafted linguistic features in Chinese event trigger identification.
• To evaluate whether word embeddings provide a more effective representation than traditional, domain-specific features for this task.
• To examine the impact of pre-training with large amounts of unlabeled data on model performance in low-resource event extraction scenarios.

Proposed method

• The model uses character-level word embeddings as input features, replacing manually engineered linguistic features such as POS tags, parsing structures, and named entities.
• A deep neural network is trained end-to-end using backpropagation with stochastic gradient descent to learn abstract, distributed representations of input characters.
• Pre-training of word embeddings is performed on 100,000 unlabeled Chinese newswire documents using a skip-gram RNN-based word2vec framework before fine-tuning on the labeled event extraction task.
• The final prediction uses a hybrid decoding strategy: DNN outputs are treated as emission probabilities, combined with transition probabilities from a CRF model to improve sequence labeling accuracy.
• The system is implemented using Theano, a Python-based deep learning framework, enabling efficient computation and backpropagation.

Experimental results

Research questions

• RQ1 Can a deep neural network with word embeddings effectively replace manual feature engineering in Chinese event trigger identification?
• RQ2 Does word embedding provide a more robust and generalizable representation than traditional linguistic features for this task?
• RQ3 To what extent does pre-training on large-scale unlabeled data improve performance in low-resource event extraction?

Key findings

• The DNN model with word embeddings achieved an F1 score of 64.29, significantly outperforming the best baseline ME classifier using automatically generated named entities (F1: 61.44).
• Pre-training on unlabeled data improved the DNN’s F1 score by 12.92 percentage points compared to the non-pretrained version (from 58.06 to 64.29), demonstrating the value of unsupervised pre-training.
• The model achieved higher recall (61.14%) than the best baseline (52.17%), indicating that word embeddings effectively capture semantic similarity and help identify novel or rare trigger words.
• The optimal word embedding dimension was found to be around 100, as smaller sizes lacked representational capacity and larger sizes led to overfitting and reduced generalization.
• Even without using gold-standard named entity recognition, the DNN model outperformed the ME classifier that relied on ground-truth NER, showing the robustness of the embedding-based approach.