[Paper Review] Succinct Sampling on Streams

Vladimir Braverman, Rafail Ostrovsky+1 more|ArXiv.org|Feb 25, 2007

Advanced Database Systems and Queries47 references3 citations

TL;DR

This paper introduces S³ (Succinct Sampling on Streams) algorithms that provide provably optimal, worst-case memory guarantees for uniform sampling over sliding windows in data streams—both with and without replacement, and in both one-at-a-time and bursty arrival models. The method enables the first memory-optimal solutions for frequency moments, triangle counting, entropy estimation, and density estimation under sliding window constraints, using a novel sampling technique that ensures independence across non-overlapping windows and avoids floating-point operations.

ABSTRACT

A streaming model is one where data items arrive over long period of time, either one item at a time or in bursts. Typical tasks include computing various statistics over a sliding window of some fixed time-horizon. What makes the streaming model interesting is that as the time progresses, old items expire and new ones arrive. One of the simplest and central tasks in this model is sampling. That is, the task of maintaining up to $k$ uniformly distributed items from a current time-window as old items expire and new ones arrive. We call sampling algorithms {\bf succinct} if they use provably optimal (up to constant factors) {\bf worst-case} memory to maintain $k$ items (either with or without replacement). We stress that in many applications structures that have {\em expected} succinct representation as the time progresses are not sufficient, as small probability events eventually happen with probability 1. Thus, in this paper we ask the following question: are Succinct Sampling on Streams (or $S^3$-algorithms)possible, and if so for what models? Perhaps somewhat surprisingly, we show that $S^3$-algorithms are possible for {\em all} variants of the problem mentioned above, i.e. both with and without replacement and both for one-at-a-time and bursty arrival models. Finally, we use $S^3$ algorithms to solve various problems in sliding windows model, including frequency moments, counting triangles, entropy and density estimations. For these problems we present \emph{first} solutions with provable worst-case memory guarantees.

Motivation & Objective

• To address the lack of worst-case memory guarantees in streaming sampling, especially for sliding window models where old data expires and new data arrives continuously.
• To develop sampling algorithms that are both memory-optimal and deterministic, avoiding expected-memory solutions that fail under rare but inevitable edge cases.
• To extend existing sampling-based algorithms for frequency moments, triangle counting, entropy, and density estimation to sliding window models with provable memory bounds.
• To eliminate reliance on floating-point arithmetic in bursty arrival models, improving practicality and determinism.
• To ensure that samples from non-overlapping windows are statistically independent, enhancing reusability and query flexibility.

Proposed method

• Proposes a novel sampling method that maintains uniform random samples of size $k$ over a sliding window using $O(k)$ memory, matching known lower bounds.
• Uses a reservoir-like structure but modifies selection probabilities to ensure worst-case memory bounds and independence across non-overlapping windows.
• Applies the S³ framework to replace prior sampling methods (e.g., priority sampling, reservoir sampling) in existing algorithms for frequency moments, triangle counting, entropy, and density estimation.
• Adapts the algorithm to both sequence-based and timestamp-based sliding window models, supporting both with- and without-replacement sampling.
• Avoids floating-point arithmetic in bursty models by using integer-based probability thresholds, improving determinism and efficiency.
• Combines samples from multiple windows of varying lengths using step-biased functions to support non-uniform sampling in dynamic settings.

Experimental results

Research questions

• RQ1 Can we design streaming sampling algorithms with provably optimal worst-case memory usage for sliding windows?
• RQ2 Is it possible to achieve both memory optimality and independence of samples across non-overlapping windows in the one-at-a-time model?
• RQ3 Can we eliminate floating-point operations in bursty arrival models while maintaining sampling correctness and memory efficiency?
• RQ4 Can S³ be used to build the first memory-optimal algorithms for frequency moments, triangle counting, and entropy estimation in sliding window settings?
• RQ5 Can S³ be extended to support biased sampling and other non-uniform distributions in dynamic streaming environments?

Key findings

• The paper presents the first $S^3$ algorithm with matching upper and lower bounds, achieving optimal $O(k)$ memory usage for uniform sampling with or without replacement.
• For frequency moments, the method enables an $\tilde{O}(m^{1-1/k})$-bit memory algorithm for all $k>2$, improving upon prior sampling-based approaches.
• An $\tilde{O}(m^{1-2/k})$-bit algorithm is obtained for frequency moments, matching the best known bounds, though with a $k^k$ multiplicative overhead that limits practicality for large $k$.
• For triangle counting, the method yields an $(\epsilon,\delta)$-approximation algorithm using $O(1 + \frac{\log|E_W|}{|E_W|} \cdot \frac{1}{\epsilon^2} \cdot \frac{|T_1| + 2|T_2| + 3|T_3|}{|T_3|} \log \frac{2}{\delta})$ memory in sequence-based windows, and a $\log n$ factor in timestamp-based windows.
• For entropy estimation, the method achieves $O(\epsilon^{-2} \log \delta^{-1} \log n)$ memory, improving upon prior methods that lack worst-case guarantees in sliding windows.
• The S³ framework enables the first memory-optimal solutions for density estimation and biased sampling in sliding window models by replacing prior sampling primitives with $S^3$.