[Paper Review] Weighted Random Sampling over Data Streams

Pavlos S. Efraimidis|arXiv (Cornell University)|Dec 1, 2010

Data Stream Mining Techniques13 references7 citations

TL;DR

This paper presents two efficient algorithms—A-Chao and A-ES—for weighted random sampling over data streams, enabling one-pass selection of weighted items with or without replacement. It establishes that both approaches are scalable and suitable for real-time applications, with A-Chao optimizing for final inclusion probabilities and A-ES supporting ordered sampling via key-based selection.

ABSTRACT

In this work, we present a comprehensive treatment of weighted random sampling (WRS) over data streams. More precisely, we examine two natural interpretations of the item weights, describe an existing algorithm for each case ([2, 4]), discuss sampling with and without replacement and show adaptations of the algorithms for several WRS problems and evolving data streams.

Motivation & Objective

• To provide a comprehensive treatment of weighted random sampling (WRS) over data streams, addressing key design choices like replacement, weight semantics, and stream processing.
• To analyze and compare two fundamental WRS algorithms—A-Chao and A-ES—under different interpretations of item weights and sampling constraints.
• To demonstrate the feasibility and efficiency of one-pass WRS over evolving data streams, resolving an open problem on reservoir maintenance with arbitrary bias.
• To propose a reusable abstract class and prototype implementations in Java for integration into standard software libraries.

Proposed method

• The paper introduces two core algorithms: A-Chao, which uses a key-based selection method with weights to ensure correct final inclusion probabilities; and A-ES, which uses exponential variates to simulate sequential selection with replacement.
• It distinguishes between two interpretations of weights: WRS-P (final selection probability proportional to weight) and WRS-W (selection probability per step proportional to weight), each requiring different algorithmic treatment.
• The algorithms are adapted for sampling with and without replacement, and extended using the 'jumps' technique to skip over large numbers of low-weight items efficiently.
• An abstract base class `StreamSampler` is designed with `feedItem()` and `getSample()` methods, supporting extensible implementation of WRS algorithms in object-oriented frameworks.
• Prototype implementations are developed in Java for A-Chao and A-ES, both with and without jumps, and evaluated experimentally on large synthetic data streams.
• Theoretical analysis confirms that both algorithms maintain correct sampling distributions under their respective weight semantics and scale linearly with input size.

Experimental results

Research questions

• RQ1 Can weighted random sampling be efficiently performed in a single pass over an unknown data stream, and if so, what are the algorithmic foundations?
• RQ2 How do different interpretations of item weights—final inclusion probability vs. per-step selection probability—affect the design and correctness of WRS algorithms?
• RQ3 What are the performance characteristics and scalability of A-Chao and A-ES under varying data stream sizes and sample sizes?
• RQ4 How can the sampling process be optimized using the 'jumps' technique to skip over low-weight items without bias?
• RQ5 Can a standardized, extensible software interface be defined for WRS over data streams to support integration into general-purpose libraries?

Key findings

• The A-Chao algorithm correctly implements WRS-P, ensuring that each item’s final inclusion probability is proportional to its weight, and is suitable for applications requiring unbiased final samples.
• The A-ES algorithm supports WRS-W, where selection probability per step is proportional to weight, and enables ordered sampling by key, useful in applications like ranked advertising slots.
• Both algorithms scale linearly with the number of items processed and are efficient even for large data streams, as confirmed by experimental measurements on the Sun Java 1.6 platform.
• The use of the 'jumps' technique significantly improves performance by skipping over large numbers of low-weight items without altering sampling distribution.
• Prototype implementations of A-Chao and A-ES with and without jumps demonstrate consistent linear time complexity and are suitable for integration into standard class libraries.
• For large populations, the distinction between WRS-P and WRS-W becomes negligible, as the impact of item removal diminishes and both converge to uniform sampling without replacement.