[Paper Review] Hungarian Layer: Logics Empowered Neural Architecture

Xiao Han, Chen, Yidong+1 more|arXiv (Cornell University)|Dec 7, 2017

Explainable Artificial Intelligence (XAI)1 references3 citations

TL;DR

This paper proposes the Hungarian Layer, a novel neural architecture that uses the Hungarian algorithm to enforce hard, exclusive alignments between sentence pairs, explicitly extracting semantically critical unmatched parts for paraphrase identification. By modeling these unmatched parts via dynamic computational graphs and cosine similarity, the method achieves state-of-the-art performance on the Quora Question Pairs dataset, outperforming prior models including L.D.C. by 0.78% accuracy with statistical significance (p = 0.003).

ABSTRACT

Neural architecture is a purely numeric framework, which fits the data as a continuous function. However, lacking of logic flow (e.g. extit{if, for, while}), traditional algorithms (e.g. extit{Hungarian algorithm, A$^*$ searching, decision tress algorithm}) could not be embedded into this paradigm, which limits the theories and applications. In this paper, we reform the calculus graph as a dynamic process, which is guided by logic flow. Within our novel methodology, traditional algorithms could empower numerical neural network. Specifically, regarding the subject of sentence matching, we reformulate this issue as the form of task-assignment, which is solved by Hungarian algorithm. First, our model applies BiLSTM to parse the sentences. Then Hungarian layer aligns the matching positions. Last, we transform the matching results for soft-max regression by another BiLSTM. Extensive experiments show that our model outperforms other state-of-the-art baselines substantially.

Motivation & Objective

• To address the limitation of attention mechanisms in modeling semantically critical unmatched parts during sentence matching.
• To propose a differentiable neural layer that enables hard, exclusive alignment between sentence pairs using the Hungarian algorithm.
• To empirically validate that aligned unmatched parts are crucial for accurate paraphrase identification.
• To achieve superior performance on benchmark datasets with a simpler, yet effective architecture compared to complex attention-based models.

Proposed method

• Apply BiLSTM to encode input sentences into dense hidden representations.
• Introduce the Hungarian Layer, which formulates sentence alignment as an assignment problem solved via the Hungarian algorithm for optimal, exclusive pairings.
• Dynamically construct a computational graph in the forward pass based on the Hungarian algorithm’s output to enable end-to-end backpropagation.
• Use cosine similarity on the aligned unmatched parts to compute the final matching score for paraphrase classification.
• Modify backpropagation to support both forward and backward passes through the dynamically constructed graph, ensuring differentiability.
• Train the entire model end-to-end using standard optimization with backpropagation through the Hungarian Layer’s differentiable structure.

Experimental results

Research questions

• RQ1 Can hard, exclusive alignment of sentence parts—especially unmatched pairs—improve paraphrase identification beyond soft attention mechanisms?
• RQ2 Are aligned unmatched parts semantically more informative than matched or dissimilar parts for determining paraphrase relationships?
• RQ3 Can the Hungarian algorithm be effectively integrated into a differentiable neural architecture for sequence matching tasks?
• RQ4 Does a simpler model leveraging exclusive alignment outperform complex attention-based models like L.D.C. in terms of accuracy and statistical significance?
• RQ5 Is the performance gain due to better modeling of unmatched parts, or is it attributable to architectural complexity?

Key findings

• The proposed Hungarian Layer model achieves 85.53% accuracy on the Quora Question Pairs dataset, significantly outperforming the previous state-of-the-art model L.D.C. (84.75% ± 0.42).
• The improvement over L.D.C. is statistically significant, with a p-value of 0.003 < 0.01, rejecting the null hypothesis of equivalent performance.
• The model effectively captures semantically critical unmatched parts, as validated in case studies where conflicting pairs like 'imported' vs. 'unknown' or 'love' vs. 'phone' were correctly identified as non-paraphrase.
• The Hungarian Layer enables precise, exclusive alignment, allowing the model to focus on the most discriminative unmatched components rather than relying on fuzzy attention weights.
• Despite its simplicity, the model surpasses more complex architectures like Siamese LSTM and Multi-Perspective LSTM, demonstrating strong generalization and industrial applicability.
• The case study confirms that aligned unmatched parts are indeed the key discriminative evidence: when these parts are semantically consistent (e.g., 'outside' vs. 'yard'), the model classifies as paraphrase; when conflicting (e.g., 'Austria' vs. 'Japan'), it correctly identifies as non-paraphrase.