[Paper Review] Fully dynamic data structure for LCE queries in compressed space

Hideo Bannai, I, Tomohiro+3 more|arXiv (Cornell University)|May 5, 2016

Algorithms and Data Compression22 references21 citations

TL;DR

This paper presents the first fully dynamic Longest Common Extension (LCE) data structure that operates in compressed space using signature encoding G of size w = O(min(z log N log∗M, N)), supporting LCE queries in O(log N + log ℓ log∗M) time and updates (insertions/deletions) in O((y + log N log∗M)fA) time, where z is the LZ77 factorization size, ℓ is the LCE result, and fA is the predecessor query time. The structure enables efficient grammar-compressed string processing with improved construction and query performance.

ABSTRACT

A Longest Common Extension (LCE) query on a text $T$ of length $N$ asks for the length of the longest common prefix of suffixes starting at given two positions. We show that the signature encoding $\mathcal{G}$ of size $w = O(\min(z \log N \log^* M, N))$ [Mehlhorn et al., Algorithmica 17(2):183-198, 1997] of $T$, which can be seen as a compressed representation of $T$, has a capability to support LCE queries in $O(\log N + \log \ell \log^* M)$ time, where $\ell$ is the answer to the query, $z$ is the size of the Lempel-Ziv77 (LZ77) factorization of $T$, and $M \geq 4N$ is an integer that can be handled in constant time under word RAM model. In compressed space, this is the fastest deterministic LCE data structure in many cases. Moreover, $\mathcal{G}$ can be enhanced to support efficient update operations: After processing $\mathcal{G}$ in $O(w f_{\mathcal{A}})$ time, we can insert/delete any (sub)string of length $y$ into/from an arbitrary position of $T$ in $O((y+ \log N\log^* M) f_{\mathcal{A}})$ time, where $f_{\mathcal{A}} = O(\min \{ \frac{\log\log M \log\log w}{\log\log\log M}, \sqrt{\frac{\log w}{\log\log w}} \})$. This yields the first fully dynamic LCE data structure. We also present efficient construction algorithms from various types of inputs: We can construct $\mathcal{G}$ in $O(N f_{\mathcal{A}})$ time from uncompressed string $T$; in $O(n \log\log n \log N \log^* M)$ time from grammar-compressed string $T$ represented by a straight-line program of size $n$; and in $O(z f_{\mathcal{A}} \log N \log^* M)$ time from LZ77-compressed string $T$ with $z$ factors. On top of the above contributions, we show several applications of our data structures which improve previous best known results on grammar-compressed string processing.

Motivation & Objective

• To design a fully dynamic LCE data structure that operates in compressed space, overcoming the limitations of static or uncompressed dynamic structures.
• To support efficient LCE queries and update operations (insertions/deletions) on highly compressible strings represented via signature encoding.
• To improve construction time and space efficiency for grammar-compressed and LZ77-compressed strings by leveraging signature encoding.
• To demonstrate practical applications in compressed string processing, such as palindrome detection and pattern matching, with improved time complexity.

Proposed method

• The paper uses signature encoding G as a compressed representation of the text T, derived from LZ77 factorization or straight-line programs (SLPs), with size w = O(min(z log N log∗M, N)).
• It supports LCE queries by leveraging predecessor/successor data structures over G, enabling O(log N + log ℓ log∗M) query time via a combination of LCP and LCS queries on reversed strings.
• Updates are supported by treating insertions and deletions as edit operations on the signature encoding, using a dynamic predecessor data structure with query time fA.
• The construction process uses a combination of LCP and LCS queries on SLPs and their reverses, with preprocessing via a constant-time RMQ structure on lcp arrays.
• A compacted trie (PTree) is built from patterns to support pattern matching, augmented with level ancestor and binary search for efficient prefix queries.
• The algorithm processes T in a streaming fashion, maintaining a signature p and length l to track the longest suffix of T[1..j] that is a prefix of any pattern, using LCE with p to compare pattern prefixes.

Experimental results

Research questions

• RQ1 Can a fully dynamic LCE data structure be designed to operate in compressed space while supporting efficient queries and updates?
• RQ2 How can signature encoding be used to achieve faster deterministic LCE queries than existing randomized or uncompressed structures?
• RQ3 What is the optimal construction time for signature encoding from various compressed string representations, including SLPs and LZ77?
• RQ4 Can the signature encoding framework be extended to support dynamic pattern matching with efficient output of pattern occurrences?
• RQ5 How can garbage accumulation during dynamic updates be managed effectively in a compressed dynamic data structure?

Key findings

• The signature encoding G supports LCE queries in O(log N + log ℓ log∗M) time, which is faster than the best known randomized data structure of Bille et al. [6] when log∗M = o(log ℓ), a condition that holds in many practical cases.
• The paper presents the first fully dynamic LCE data structure working in compressed space, with update time O((y + log N log∗M)fA) for inserting/deleting a substring of length y.
• Construction from an uncompressed string T of length N takes O(NfA) time and O(w) space, significantly improving over prior methods that required O(N) time or more.
• Construction from an LZ77-compressed string of size z takes O(zfA log N log∗M) time, and from an SLP of size n takes O(n log log(n log∗M) log N log∗M) time, with O(n log∗M + w) space.
• The framework enables improved pattern matching: O(|T| log m log N log∗M + occ) time for reporting all pattern occurrences, with O(log m + occ) time per position.
• The method supports output of patterns by ending position via a reversed pattern trie (RPTree), maintaining the same time and space complexity as the original algorithm.