[Paper Review] Double Embeddings and CNN-based Sequence Labeling for Aspect Extraction

Hu Xu, Bing Liu+2 more|arXiv (Cornell University)|May 11, 2018

Sentiment Analysis and Opinion MiningComputer Science38 references53 citations

TL;DR

DE-CNN uses a novel double-embedding approach (general and domain-specific) with a pure CNN for sequence labeling to extract product aspects, achieving state-of-the-art results without extra supervision.

ABSTRACT

One key task of fine-grained sentiment analysis of product reviews is to extract product aspects or features that users have expressed opinions on. This paper focuses on supervised aspect extraction using deep learning. Unlike other highly sophisticated supervised deep learning models, this paper proposes a novel and yet simple CNN model employing two types of pre-trained embeddings for aspect extraction: general-purpose embeddings and domain-specific embeddings. Without using any additional supervision, this model achieves surprisingly good results, outperforming state-of-the-art sophisticated existing methods. To our knowledge, this paper is the first to report such double embeddings based CNN model for aspect extraction and achieve very good results.

Motivation & Objective

• Motivate fine-grained sentiment analysis by accurately extracting product aspects.
• Propose a simple yet effective CNN-based sequence labeling model.
• Show that combining general and domain-specific embeddings improves performance.
• Demonstrate competitive results without handcrafted features or extra supervision.

Proposed method

• Concatenate two fixed (non-tunable) embeddings per word: general GloVe and domain-specific fastText embeddings.
• Apply a 4-layer CNN stack with carefully chosen kernel sizes to produce word-aligned representations for sequence labeling.
• Use a shared fully-connected layer and a softmax classifier to predict B/I/O labels per position.
• Avoid max-pooling to preserve position alignment; apply dropout after embeddings and activations.
• Optionally compare with a CRF layer to assess its impact on labeling dependencies.

Experimental results

Research questions

• RQ1 Can a CNN-based model with double embeddings sufficiently capture fine-grained aspect information without extra supervision?
• RQ2 How does in-domain vs. general embeddings affect aspect extraction performance?
• RQ3 Is max-pooling detrimental for sequence labeling in aspect extraction tasks?
• RQ4 Does a CRF layer provide additional gains over a pure softmax in this setting?

Key findings

• DE-CNN achieves the best F1 scores on SemEval laptop and restaurant datasets compared to the listed baselines.
• Using both general and domain embeddings improves performance over using either alone.
• In-domain embeddings are particularly beneficial in the laptop domain, where domain-specific aspects are frequent.
• Max-pooling degrades sequence labeling performance in this task.
• CRF layer offers limited gains and is not consistently beneficial across domains.