[Paper Review] Nearly Optimal Distinct Elements and Heavy Hitters on Sliding Windows

Vladimir Braverman, Elena Grigorescu+3 more|arXiv (Cornell University)|Jan 1, 2018

Algorithms and Data Compression4 references12 citations

TL;DR

This paper presents nearly optimal algorithms for estimating distinct elements and identifying ℓp-heavy hitters in the sliding window model, where only the most recent n elements are considered. It introduces the composable histogram—a novel variant of exponential and smooth histograms—that enables efficient tracking of frequencies and identities of key items, achieving space complexity within logarithmic factors of the information-theoretic lower bounds for both problems.

ABSTRACT

We study the distinct elements and l_p-heavy hitters problems in the sliding window model, where only the most recent n elements in the data stream form the underlying set. We first introduce the composable histogram, a simple twist on the exponential (Datar et al., SODA 2002) and smooth histograms (Braverman and Ostrovsky, FOCS 2007) that may be of independent interest. We then show that the composable histogram{} along with a careful combination of existing techniques to track either the identity or frequency of a few specific items suffices to obtain algorithms for both distinct elements and l_p-heavy hitters that are nearly optimal in both n and epsilon. Applying our new composable histogram framework, we provide an algorithm that outputs a (1+epsilon)-approximation to the number of distinct elements in the sliding window model and uses O{1/(epsilon^2) log n log (1/epsilon)log log n+ (1/epsilon) log^2 n} bits of space. For l_p-heavy hitters, we provide an algorithm using space O{(1/epsilon^p) log^2 n (log^2 log n+log 1/epsilon)} for 0<p <=2, improving upon the best-known algorithm for l_2-heavy hitters (Braverman et al., COCOON 2014), which has space complexity O{1/epsilon^4 log^3 n}. We also show complementing nearly optimal lower bounds of Omega ((1/epsilon) log^2 n+(1/epsilon^2) log n) for distinct elements and Omega ((1/epsilon^p) log^2 n) for l_p-heavy hitters, both tight up to O{log log n} and O{log 1/epsilon} factors.

Motivation & Objective

• To address the fundamental problem of estimating distinct elements and identifying heavy hitters in the sliding window model, where only the most recent n elements are active.
• To develop space-efficient algorithms that provide (1+ǫ)-approximations for distinct elements and (ǫ, φ)-approximations for ℓp-heavy hitters.
• To close the gap between existing upper bounds and known lower bounds by designing a new data structure—composable histogram—that improves space efficiency.
• To establish nearly tight lower bounds for both problems, showing that the proposed algorithms are optimal up to logarithmic factors.

Proposed method

• Introduces the composable histogram, a modified exponential histogram that enables efficient composition of sketches across sliding windows.
• Leverages existing streaming techniques to track identities and frequencies of a small set of candidate items, reducing space overhead.
• Combines composable histograms with frequency estimation and heavy hitter detection algorithms to maintain accuracy under sliding window constraints.
• Employs a recursive embedding of communication complexity problems (e.g., GapHamming, AugmentedIndex) to prove lower bounds.
• Uses amplification techniques to boost success probability from 2/3 to 1−δ with only logarithmic blowup in space.
• Applies careful union bound analysis over polylogarithmic instances to correct earlier errors in prior work, ensuring correctness in space complexity.

Experimental results

Research questions

• RQ1 Can we achieve nearly optimal space complexity for distinct elements estimation in the sliding window model?
• RQ2 Can we design an efficient algorithm for ℓp-heavy hitters in the sliding window model that improves upon prior space bounds?
• RQ3 Is the composable histogram framework sufficient to unify and improve upon existing exponential and smooth histogram approaches?
• RQ4 What are the tightest possible lower bounds for distinct elements and ℓp-heavy hitters in the sliding window model?
• RQ5 Can we correct and improve upon prior incorrect space complexity claims in the literature, particularly for ℓ2-heavy hitters?

Key findings

• The paper presents an algorithm for distinct elements with space complexity O(1/ǫ² log n log(1/ǫ) log log n + 1/ǫ log² n), which is nearly optimal up to log log n and log(1/ǫ) factors.
• For ℓp-heavy hitters with 0 < p ≤ 2, the algorithm uses O(1/ǫ^p log³ n (log log n + log(1/ǫ))) bits, improving upon the prior O(1/ǫ⁴ log³ n) bound for ℓ2-heavy hitters.
• The authors establish a lower bound of Ω(1/ǫ log² n + 1/ǫ² log n) for distinct elements, showing their algorithm is nearly optimal in both ǫ and n.
• For ℓp-heavy hitters, a lower bound of Ω(1/ǫ^p log² n) is proven, matching the upper bound up to log(1/ǫ) factors.
• The paper corrects a previous error in the analysis of ℓ2-heavy hitters, replacing an incorrect O(1/ǫ² log² n (log² log n + log(1/ǫ))) bound with the correct O(1/ǫ² log³ n (log log n + log(1/ǫ))) complexity.
• The composable histogram framework is shown to be a general and powerful tool, potentially of independent interest beyond this work.