[Paper Review] Universal Streaming of Subset Norms

Vladimir Braverman, Robert Krauthgamer+1 more|arXiv (Cornell University)|Dec 1, 2018

Semantic Web and Ontologies46 references4 citations

TL;DR

This paper introduces a universal streaming algorithm that approximates subset-ℓ₀-norms for any family of subsets 𝒮 ⊆ 2^[n] in the insertion-only model, using space characterized by the newly defined 'heavy-hitter dimension' of 𝒮. The key result is a near-tight characterization of space complexity as Θ̃(HHdim(𝒮)) for ℓ₀ and ℓ₁ norms, with separation from more general models like turnstile and sliding window.

ABSTRACT

Most known algorithms in the streaming model of computation aim to approximate a single function such as an $\ell_p$-norm. In 2009, Nelson [\url{https://sublinear.info}, Open Problem 30] asked if it possible to design \emph{universal algorithms}, that simultaneously approximate multiple functions of the stream. In this paper we answer the question of Nelson for the class of \emph{subset $\ell_0$-norms} in the insertion-only frequency-vector model. Given a family of subsets $\mathcal{S}\subset 2^{[n]}$, we provide a single streaming algorithm that can $(1\pm ε)$-approximate the subset-norm for every $S\in\mathcal{S}$. Here, the subset-$\ell_p$-norm of $v\in \mathbb{R}^n$ with respect to set $S\subseteq [n]$ is the $\ell_p$-norm of vector $v_{|S}$ (which denotes restricting $v$ to $S$, by zeroing all other coordinates). Our main result is a near-tight characterization of the space complexity of every family $\mathcal{S}\subset 2^{[n]}$ of subset-$\ell_0$-norms in insertion-only streams, expressed in terms of the "heavy-hitter dimension" of $\mathcal{S}$, a new combinatorial quantity that is related to the VC-dimension of $\mathcal{S}$. In contrast, we show that the more general turnstile and sliding-window models require a much larger space usage. All these results easily extend to $\ell_1$. In addition, we design algorithms for two other subset-$\ell_p$-norm variants. These can be compared to the Priority Sampling algorithm of Duffield, Lund and Thorup [JACM 2007], which achieves additive approximation $ε\|{v}\|$ for all possible subsets ($\mathcal{S}=2^{[n]}$) in the entry-wise update model. One of our algorithms extends this algorithm to handle turnstile updates, and another one achieves multiplicative approximation given a family $\mathcal{S}$.

Motivation & Objective

• Address Nelson's 2009 open problem on universal streaming for multiple functions by focusing on subset-ℓp-norms.
• Characterize the space complexity of universally approximating subset-ℓ₀-norms for any family 𝒮 of subsets in the insertion-only model.
• Introduce and formalize the 'heavy-hitter dimension' as a combinatorial measure that captures the complexity of such universal approximation.
• Establish separations between insertion-only and more general streaming models (turnstile, sliding window) in terms of space requirements.
• Extend results to ℓ₁-norms and explore variants for entry-wise and insertion-only update models, including connections to Priority Sampling.

Proposed method

• Define the subset-ℓp-norm of a vector v ∈ ℝⁿ with respect to S ⊆ [n] as the ℓp-norm of v restricted to S.
• Introduce the 'heavy-hitter dimension' of a set family 𝒮 as the maximum number of distinct heavy-hitters (coordinates with large magnitude) across all subsets S ∈ 𝒮, for any input vector v.
• Design a streaming algorithm that maintains a summary of size Õ(HHdim(𝒮)) to (1±ε)-approximate the subset-ℓ₀-norm for every S ∈ 𝒮 in the insertion-only model.
• Use probabilistic analysis and norm comparisons (e.g., ‖v‖₂ ≤ ‖v‖_p^{1/2−1/p}) to bound tail contributions and ensure concentration via Markov’s inequality.
• Prove that the space complexity is tightly characterized by HHdim(𝒮) for ℓ₀ and ℓ₁ norms in insertion-only and entry-wise models.
• Demonstrate that the characterization fails in turnstile and sliding-window models, showing a strict separation in space complexity.

Experimental results

Research questions

• RQ1 Can a single streaming algorithm universally approximate all subset-ℓ₀-norms for a given family 𝒮 ⊆ 2^[n] in the insertion-only model?
• RQ2 What combinatorial property of 𝒮 determines the minimal space required for such universal approximation?
• RQ3 How does the space complexity of universal subset-ℓp-norm approximation differ between insertion-only, turnstile, and sliding-window models?
• RQ4 Can the results for ℓ₀ and ℓ₁ be extended to general ℓp-norms with p ≠ 0,1?
• RQ5 How do the proposed algorithms relate to or generalize existing methods like Priority Sampling for ℓ₁-norms?

Key findings

• The space complexity of universal streaming for subset-ℓ₀-norms in the insertion-only model is characterized by the heavy-hitter dimension of 𝒮, up to logarithmic factors.
• For ℓ₁-norms, the same characterization holds: space complexity is Õ(HHdim(𝒮)) in the insertion-only model.
• The characterization breaks down in the turnstile and sliding-window models, where significantly more space is required, indicating a strict separation between models.
• For entry-wise updates, the space complexity for general ℓp-norms (p ≠ 1) is also characterized by HHdim(𝒮), with a near-tight bound of Õ(HHdim(𝒮)).
• The algorithm achieves (1±ε)-multiplicative approximation for all S ∈ 𝒮 using a summary of size Õ(HHdim(𝒮)) in insertion-only and entry-wise models.
• A variant of the algorithm extends Priority Sampling to handle turnstile updates while maintaining multiplicative approximation for ℓ₁-norms over subsets.