[Paper Review] Unsupervised Sentence Textual Similarity with Compositional Phrase Semantics

Zihao Wang, Jiaheng Dou+1 more|arXiv (Cornell University)|Oct 5, 2022

Topic Modeling4 citations

TL;DR

This paper proposes Recursive Optimal Transport Similarity (ROTS), a novel unsupervised sentence textual similarity method that captures compositional phrase semantics through recursive phrase partitioning and optimal transport alignment. ROTS achieves state-of-the-art performance across 29 STS benchmarks, outperforming both traditional unsupervised methods and BERT-based approaches in both accuracy and efficiency, with linear-time complexity and minimal computational overhead.

ABSTRACT

Measuring Sentence Textual Similarity (STS) is a classic task that can be applied to many downstream NLP applications such as text generation and retrieval. In this paper, we focus on unsupervised STS that works on various domains but only requires minimal data and computational resources. Theoretically, we propose a light-weighted Expectation-Correction (EC) formulation for STS computation. EC formulation unifies unsupervised STS approaches including the cosine similarity of Additively Composed (AC) sentence embeddings, Optimal Transport (OT), and Tree Kernels (TK). Moreover, we propose the Recursive Optimal Transport Similarity (ROTS) algorithm to capture the compositional phrase semantics by composing multiple recursive EC formulations. ROTS finishes in linear time and is faster than its predecessors. ROTS is empirically more effective and scalable than previous approaches. Extensive experiments on 29 STS tasks under various settings show the clear advantage of ROTS over existing approaches. Detailed ablation studies demonstrate the effectiveness of our approaches.

Motivation & Objective

• To develop a lightweight, unsupervised STS method that effectively captures phrase-level semantics without requiring large-scale labeled data or expensive pretraining.
• To unify existing unsupervised STS approaches—additive composition, optimal transport, and tree kernels—under a common Expectation-Correction (EC) formulation.
• To design a recursive framework that enables hierarchical phrase alignment by composing finer-grained phrase embeddings through optimal transport.
• To achieve high performance and scalability in unsupervised STS with minimal computational cost and data requirements.

Proposed method

• Introduces the Expectation-Correction (EC) similarity formulation as a unifying framework for AC, OT, and TK methods, revealing their shared structural properties.
• Proposes Recursive Phrase Partitioning (RPP) to recursively decompose sentences into phrase-level constituents, enabling compositional semantics modeling.
• Develops Recursive Optimal Transport Similarity (ROTS) by applying EC similarity at each recursive partition level, using Prior Optimal Transport to guide alignment from coarser to finer granularities.
• Employs additive composition of phrase vectors and weights across recursive levels to preserve semantic hierarchy and improve alignment accuracy.
• Uses a weighted ensemble of EC similarities across multiple recursive levels, with hyperparameters easily tuned for optimal performance.
• Achieves linear-time complexity by leveraging dynamic programming and recursive decomposition, making it scalable to long texts.

Experimental results

Research questions

• RQ1 Can a unified formulation like EC similarity effectively unify and improve upon existing unsupervised STS methods such as additive composition, optimal transport, and tree kernels?
• RQ2 To what extent can recursive phrase-level alignment enhance sentence similarity modeling compared to word-level alignment alone?
• RQ3 Does incorporating compositional phrase semantics through recursive decomposition lead to measurable gains in STS performance across diverse domains?
• RQ4 How does ROTS compare in accuracy and efficiency to both classical unsupervised methods and BERT-based supervised approaches in low-resource settings?

Key findings

• ROTS achieves state-of-the-art performance on 29 diverse STS benchmarks, outperforming all prior unsupervised methods, including BERT-based models, in both effectiveness and efficiency.
• On the STS-Benchmark, ROTS achieves a Pearson correlation of 72.90 (95% CI: [71.36, 74.38]) with ParaNMT vectors, surpassing the next best method by over 0.6 points.
• With SupWord vectors, ROTS achieves 79.74 (95% CI: [77.41, 81.77]) on STS-B, demonstrating robustness across different word embedding types.
• ROTS consistently outperforms AC, WRD, and OT baselines across all 29 tasks, with average gains of 1.5–2.5 points in Pearson correlation.
• Ablation studies confirm that recursive phrase alignment and the use of Prior OT are essential components, with performance dropping significantly when removed.
• ROTS runs in linear time and is significantly faster than previous recursive or tree-based methods, enabling scalable deployment in real-world applications.