[Paper Review] Convolutional Neural Network Architectures for Matching Natural Language Sentences

Baotian Hu, Zhengdong Lu+2 more|arXiv (Cornell University)|Mar 11, 2015

Topic ModelingComputer Science28 references970 citations

TL;DR

This paper proposes novel convolutional neural network (CNN) architectures—Arc-I and Arc-II—for matching natural language sentences by jointly modeling hierarchical sentence structures and cross-sentence matching patterns. The models use multi-level convolution and pooling to capture local and global semantic interactions, achieving state-of-the-art performance on multiple sentence matching tasks, including paraphrase identification and response matching, with Arc-II outperforming competitors significantly on large-scale datasets.

ABSTRACT

Semantic matching is of central importance to many natural language tasks \cite{bordes2014semantic,RetrievalQA}. A successful matching algorithm needs to adequately model the internal structures of language objects and the interaction between them. As a step toward this goal, we propose convolutional neural network models for matching two sentences, by adapting the convolutional strategy in vision and speech. The proposed models not only nicely represent the hierarchical structures of sentences with their layer-by-layer composition and pooling, but also capture the rich matching patterns at different levels. Our models are rather generic, requiring no prior knowledge on language, and can hence be applied to matching tasks of different nature and in different languages. The empirical study on a variety of matching tasks demonstrates the efficacy of the proposed model on a variety of matching tasks and its superiority to competitor models.

Motivation & Objective

• To develop a generic deep learning framework for matching natural language sentences without requiring linguistic prior knowledge.
• To jointly model hierarchical sentence composition and cross-sentence matching patterns using a unified convolutional architecture.
• To evaluate the effectiveness of the proposed models across diverse matching tasks with varying data scales and linguistic complexity.
• To investigate the role of local and global matching patterns in sentence similarity modeling.
• To demonstrate the superiority of CNN-based architectures over traditional bag-of-words and RNN-based models in sentence matching tasks.

Proposed method

• The model uses a deep convolutional architecture with shared-weight filters and ReLU activation to process word embeddings in sliding windows of fixed size.
• Max-pooling is applied after each convolutional layer to reduce dimensionality and filter out less relevant word compositions, enabling robust representation across variable-length sentences.
• The architecture supports hierarchical composition: lower layers capture local n-gram patterns, while deeper layers build increasingly abstract representations through stacking and pooling.
• Arc-II extends this by introducing a second convolutional layer on the matching feature map, allowing richer modeling of interaction patterns between sentence pairs.
• The final sentence representations are obtained via global max-pooling over the last convolutional layer, producing fixed-length vectors for matching.
• The models are trained end-to-end using stochastic gradient descent with early stopping, using margin-based ranking loss for pairwise matching.

Experimental results

Research questions

• RQ1 Can a unified CNN architecture effectively model both hierarchical sentence composition and cross-sentence matching patterns?
• RQ2 How does the performance of the proposed CNN models compare to existing methods on diverse sentence matching tasks?
• RQ3 Does the inclusion of deeper matching layers (as in Arc-II) significantly improve performance on large-scale datasets?
• RQ4 To what extent can the model learn meaningful sentence representations without linguistic priors such as parse trees?
• RQ5 How do local matching patterns (e.g., synonymy, paraphrase) influence model performance compared to global structural alignment?

Key findings

• Arc-II achieved the highest performance across all tasks, with a 69.9% accuracy and 80.91% F1 score on the MSRP paraphrase identification benchmark, outperforming all competitors.
• On the response matching task, Arc-II achieved 75.8% accuracy, significantly outperforming the next best model (72.4%) and demonstrating strong generalization on dialogue applications.
• On the top-k re-ranking task in machine translation, Arc-II achieved 74.2% accuracy, surpassing the second-best model by a large margin.
• The models showed robustness to variable sentence length, effectively handling inputs of diverse lengths through max-pooling and hierarchical composition.
• Even with random negative sampling, Arc-I and Arc-II achieved ~60% accuracy in detecting correct word order in sentences, indicating implicit learning of sequential structure.
• The use of Word2Vec embeddings with simple summation yielded competitive results, suggesting that distributed word representations inherently encode compositional meaning.