[Paper Review] Text Matching as Image Recognition

Liang Pang, Yanyan Lan+4 more|arXiv (Cornell University)|Feb 20, 2016

Topic ModelingComputer Science24 references176 citations

TL;DR

The paper introduces MatchPyramid, a CNN-based approach that models text matching as image recognition by constructing a word-level similarity matching matrix and learning hierarchical matching patterns from words to sentences.

ABSTRACT

Matching two texts is a fundamental problem in many natural language processing tasks. An effective way is to extract meaningful matching patterns from words, phrases, and sentences to produce the matching score. Inspired by the success of convolutional neural network in image recognition, where neurons can capture many complicated patterns based on the extracted elementary visual patterns such as oriented edges and corners, we propose to model text matching as the problem of image recognition. Firstly, a matching matrix whose entries represent the similarities between words is constructed and viewed as an image. Then a convolutional neural network is utilized to capture rich matching patterns in a layer-by-layer way. We show that by resembling the compositional hierarchies of patterns in image recognition, our model can successfully identify salient signals such as n-gram and n-term matchings. Experimental results demonstrate its superiority against the baselines.

Motivation & Objective

• Motivate the need for capturing rich, hierarchical interaction patterns in text matching from words to sentences.
• Propose a new deep architecture (MatchPyramid) that treats text matching as image recognition through a matching matrix.
• Demonstrate that hierarchical convolution can capture unigram to sentence-level matching signals.
• Compare against strong baselines on paraphrase identification and paper citation matching to establish effectiveness.

Proposed method

• Construct a matching matrix M where Mij represents similarity between word wi and vj (using indicator, cosine, or dot product based on embeddings).
• Apply a convolutional neural network on M to extract multi-level matching patterns via hierarchical convolutions and dynamic pooling.
• Use ReLU activations and train with a two-layer MLP on top to produce a final matching score with softmax and cross-entropy loss.
• Optimize with Adagrad and employ regularization techniques like early stopping and dropout.

Experimental results

Research questions

• RQ1 Can MatchPyramid outperform traditional and deep baselines on text matching tasks such as paraphrase identification and paper citation matching?
• RQ2 How do different definitions of word-level similarity (indicator, cosine, dot product) affect performance?
• RQ3 Do hierarchical CNNs capture meaningful word- to sentence-level matching patterns (e.g., n-grams, n-terms) effectively?
• RQ4 What is the impact of dynamic pooling on handling variable text lengths in matching tasks?

Key findings

• MatchPyramid with real-valued (Cosine or Dot) matrices outperforms baselines on paraphrase identification (MSRP) with MP-Dot achieving 83.01% F1.
• On paper citation matching, MP-Dot achieves the best results (Acc. 88.73%, F1 82.86%), indicating semantic-aware matching helps in this task.
• Indicator-based MP variants (MP-Ind) still outperform several baselines on MSRP, showing the importance of interaction patterns beyond exact word matches.
• TF-IDF remains a strong baseline on paper citation matching, highlighting domain-specific differences in dataset transfer.
• The model visually learns interpretable matching patterns corresponding to n-gram and n-term matches, similar to edge detectors in image CNNs.
• Overall, MatchPyramid outperforms Arc-I/Arc-II and other deep models on the studied tasks, demonstrating the benefit of a word-to-sentence hierarchical matching approach.