[Paper Review] Sequence Covering Similarity for Symbolic Sequence Comparison

Pierre-François Marteau|arXiv (Cornell University)|Jan 22, 2018

Natural Language Processing Techniques3 references3 citations

TL;DR

This paper introduces sequence covering similarity, a novel pairwise distance measure for symbolic sequences based on optimal subsequence covering using a reference set. It defines a semimetric that operates in O(n log n) time, outperforming Levenshtein’s O(n²) distance, and demonstrates utility in plagiarism detection with high similarity scores for lifted text passages.

ABSTRACT

This paper introduces the sequence covering similarity, that we formally define for evaluating the similarity between a symbolic sequence (string) and a set of symbolic sequences (strings). From this covering similarity we derive a pair-wise distance to compare two symbolic sequences. We show that this covering distance is a semimetric. Few examples are given to show how this string metric in $O(n \cdot log n)$ compares with the Levenshtein's distance that is in $O(n^2)$. A final example presents its application to plagiarism detection.

Motivation & Objective

• To develop a new, efficient similarity measure for symbolic sequences that complements existing methods like Levenshtein distance.
• To formalize the concept of sequence covering using a reference set of sequences as a dictionary of reusable subsequences.
• To define a pairwise distance metric derived from covering similarity that satisfies semimetric properties.
• To demonstrate practical utility in applications such as plagiarism detection using real-world text examples.
• To provide an efficient algorithm for computing optimal coverings using suffix data structures, enabling scalability.

Proposed method

• Define sequence covering as a minimal multiset of subsequences from a reference set S that fully and correctly cover a target sequence s.
• Formalize the sequence covering similarity as $ \mathscr{S}(s,S) = \frac{|s| - |c^{*}_{S}(s)| + 1}{|s|} $, where $ |c^{*}_{S}(s)| $ is the number of subsequences in the optimal covering.
• Derive a pairwise distance from the similarity measure: $ d(s_1, s_2) = 1 - \mathscr{S}(s_1, S) $, with S being a set including s₂.
• Implement an incremental greedy algorithm to compute the S-optimal covering by repeatedly selecting the longest subsequence from S_sub starting at the current position.
• Leverage suffix trees or suffix arrays for efficient subsequence lookup, enabling O(n log n) time complexity.
• Apply the method to real text pairs, including plagiarism detection, by comparing covering similarity and Levenshtein distance.

Experimental results

Research questions

• RQ1 Can a subsequence-based covering approach yield a more efficient and meaningful similarity measure than traditional edit-distance methods?
• RQ2 Is the proposed covering distance a valid semimetric on the space of symbolic sequences?
• RQ3 How does the sequence covering similarity perform in comparison to Levenshtein distance on real textual data?
• RQ4 To what extent can this method detect lifted passages in plagiarism detection scenarios?
• RQ5 Can the method be efficiently scaled using advanced string data structures like suffix arrays?

Key findings

• The sequence covering similarity achieves a time complexity of O(n log n), significantly faster than Levenshtein’s O(n²) for long sequences.
• The covering distance is formally proven to be a semimetric, satisfying non-negativity, identity of indiscernibles, and symmetry.
• For the sequence s₃ = [0,0,0,0,1,1,1,1,0,0,0,0,1,1,1,1], the similarity score was 13/16 = 0.8125, while s₄ yielded 9/16 = 0.5625, correctly reflecting structural similarity.
• In plagiarism detection, the method achieved a covering similarity of 0.801 and distance of 0.219 for a lifted passage, indicating strong alignment with source material.
• The method successfully identifies lifted text even when wordings are slightly altered, as shown by the optimal covering capturing key phrases despite minor rewording.
• The non-uniqueness of optimal coverings can lead to minor discrepancies in substrings, but post-processing can resolve these with minimal overhead.