[Paper Review] A Near-Optimal Algorithm for L1-Difference

Jelani Nelson, David P. Woodruff|ArXiv.org|Apr 13, 2009

graph theory and CDMA systems24 references3 citations

TL;DR

This paper presents a near-optimal $L_1$-difference sketching algorithm that achieves $O(\varepsilon^{-2}\log(1/\varepsilon)\log(nM))$-bit sketches in $O(n\log^2(nM))$ time, independent of $\varepsilon$, resolving the first open problem from the 2006 IITK Workshop on Data Streams. It combines pairwise independent hashing with a tailored estimation procedure to enable near-linear time sketching while maintaining optimal sketch size.

ABSTRACT

We give the first L_1-sketching algorithm for integer vectors which produces nearly optimal sized sketches in nearly linear time. This answers the first open problem in the list of open problems from the 2006 IITK Workshop on Algorithms for Data Streams. Specifically, suppose Alice receives a vector x in {-M,...,M}^n and Bob receives y in {-M,...,M}^n, and the two parties share randomness. Each party must output a short sketch of their vector such that a third party can later quickly recover a (1 +/- eps)-approximation to ||x-y||_1 with 2/3 probability given only the sketches. We give a sketching algorithm which produces O(eps^{-2}log(1/eps)log(nM))-bit sketches in O(n*log^2(nM)) time, independent of eps. The previous best known sketching algorithm for L_1 is due to [Feigenbaum et al., SICOMP 2002], which achieved the optimal sketch length of O(eps^{-2}log(nM)) bits but had a running time of O(n*log(nM)/eps^2). Notice that our running time is near-linear for every eps, whereas for sufficiently small values of eps, the running time of the previous algorithm can be as large as quadratic. Like their algorithm, our sketching procedure also yields a small-space, one-pass streaming algorithm which works even if the entries of x,y are given in arbitrary order.

Motivation & Objective

• To address the open problem from the 2006 IITK Workshop on Algorithms for Data Streams concerning efficient $L_1$-difference computation in streaming settings.
• To design a sketching algorithm that achieves optimal sketch size while significantly improving time complexity over prior work, especially for small $\varepsilon$.
• To enable near-linear time sketch generation for integer vectors in $\{-M, \ldots, M\}^n$ with $(1\pm\varepsilon)$-approximation of $\|x-y\|_1$.
• To support one-pass, streaming processing of vectors in arbitrary order, crucial for network monitoring and distributed data processing.

Proposed method

• Uses pairwise independent hashing to project vector differences into a smaller space, enabling efficient estimation of $L_1$-norm.
• Employs a hierarchical estimation framework that partitions the vector space into dyadic intervals and estimates the $L_1$-norm per interval using randomized counters.
• Applies Chebyshev’s inequality to bound estimation error in each interval, ensuring concentration around the true value with high probability.
• Integrates a tailored sketching procedure based on the TLE (Threshold Estimation) algorithm to estimate the $L_1$-norm of each interval with controlled error.
• Combines sketches from Alice and Bob via subtraction of counters, allowing a third party to estimate $\|x-y\|_1$ using post-processing on the combined sketch.
• Uses the Nisan-Zuckerman pseudorandom generator to compress the required shared randomness to a seed of length linear in the sketch size, enabling practical deployment.

Experimental results

Research questions

• RQ1 Can an $L_1$-difference sketching algorithm achieve optimal sketch size while maintaining near-linear time complexity, independent of $\varepsilon$?
• RQ2 Is it possible to reduce the $\varepsilon^{-2}$ dependence in the running time of existing $L_1$-sketching algorithms without sacrificing sketch size?
• RQ3 Can a one-pass, streaming algorithm for $L_1$-difference be designed that works under arbitrary update order and supports efficient sketch combination?
• RQ4 What is the minimal amount of shared randomness required to achieve a $(1\pm\varepsilon)$-approximation to $\|x-y\|_1$ with high probability?

Key findings

• The algorithm produces $O(\varepsilon^{-2}\log(1/\varepsilon)\log(nM))$-bit sketches, matching the optimal sketch size of prior work but with significantly improved time complexity.
• The running time for sketch generation is $O(n\log^2(nM))$, which is independent of $\varepsilon$, resolving the $\varepsilon^{-2}$ bottleneck in prior algorithms.
• The third party can recover a $(1\pm\varepsilon)$-approximation to $\|x-y\|_1$ with probability at least $2/3$ in $O(\varepsilon^{-2}(\log(\varepsilon^2 nM) + \log(1/\varepsilon)\log\log(1/\varepsilon)\log\log\log(1/\varepsilon)))$ time.
• The sketching procedure supports one-pass, streaming processing of vectors in arbitrary order, making it suitable for network monitoring and distributed systems.
• The required shared randomness is reduced from $\mathrm{polylog}(nM)$ to a seed of length linear in the sketch size via the Nisan-Zuckerman pseudorandom generator, enabling practical deployment.