[Paper Review] Sentence Similarity Learning by Lexical Decomposition and Composition

Zhiguo Wang, Haitao Mi+1 more|arXiv (Cornell University)|Feb 23, 2016

Topic Modeling13 references163 citations

TL;DR

The paper proposes a model that decomposes word vectors into similar and dissimilar parts using semantic matching, then composes them with a two-channel CNN to predict sentence similarity, achieving state-of-the-art results on answer sentence selection and competitive results on paraphrase identification.

ABSTRACT

Most conventional sentence similarity methods only focus on similar parts of two input sentences, and simply ignore the dissimilar parts, which usually give us some clues and semantic meanings about the sentences. In this work, we propose a model to take into account both the similarities and dissimilarities by decomposing and composing lexical semantics over sentences. The model represents each word as a vector, and calculates a semantic matching vector for each word based on all words in the other sentence. Then, each word vector is decomposed into a similar component and a dissimilar component based on the semantic matching vector. After this, a two-channel CNN model is employed to capture features by composing the similar and dissimilar components. Finally, a similarity score is estimated over the composed feature vectors. Experimental results show that our model gets the state-of-the-art performance on the answer sentence selection task, and achieves a comparable result on the paraphrase identification task.

Motivation & Objective

• Bridge lexical gaps by representing words with context-aware embeddings.
• Capture similarity at multiple granularity levels (word, phrase, syntax) by decomposing semantic components.
• Explicitly model dissimilar parts of sentence pairs to improve similarity assessment.
• Combine similar and dissimilar components via CNN-based feature composition to predict similarity.

Proposed method

• Represent each word as a 300-dim vector using pre-trained word2vec embeddings.
• Compute semantic matching vectors for each word by aggregating similarities to words in the other sentence (global, local, or max-based) as in the matching functions.
• Decompose each word vector into similar and dissimilar components using rigid, linear, or orthogonal schemes.
• Feed the similar and dissimilar component matrices into a two-channel CNN with unigram, bigram, and trigram filters to produce feature vectors.
• Concatenate the two feature vectors and apply a linear-sigmoid similarity predictor.
• Train end-to-end by maximizing likelihood with Adam optimization.

Experimental results

Research questions

• RQ1 How can both similarities and dissimilarities between two sentences be effectively modeled?
• RQ2 Can lexical gap be bridged by context-aware word representations and semantic matching?
• RQ3 Does decomposing words into similar and dissimilar parts improve sentence similarity tasks over single-channel representations?
• RQ4 Do multi-granularity CNN features on decomposed components outperform existing models on QA and paraphrase datasets?

Key findings

• Achieves state-of-the-art MAP on the QASent dataset for answer sentence selection.
• Attains competitive MRR on QASent and strong performance on WikiQA compared to CNN-based and attention-based models.
• Outperforms several baselines by incorporating both similarity and dissimilarity via decomposition and two-channel CNN.
• Demonstrates that local semantic matching (local-w) and orthogonal decomposition yield robust performance.
• Shows that higher-order n-grams beyond trigram do not always improve performance, with trigram filters providing best results in their setup.