[Paper Review] Bloomier Filters: A second look

Denis Charles, Kumar Chellapilla|ArXiv.org|Jul 6, 2008

Caching and Content Delivery11 references5 citations

TL;DR

This paper presents a simplified, linear-time construction of Bloomier filters that compactly store functions over static sets with constant-time query performance. By leveraging random acyclic graphs and modular arithmetic, the method achieves faster construction (O(n) vs. O(n log n)) than prior work while maintaining similar space efficiency and O(1) query time, with a trade-off in update efficiency.

ABSTRACT

A Bloom filter is a space efficient structure for storing static sets, where the space efficiency is gained at the expense of a small probability of false-positives. A Bloomier filter generalizes a Bloom filter to compactly store a function with a static support. In this article we give a simple construction of a Bloomier filter. The construction is linear in space and requires constant time to evaluate. The creation of our Bloomier filter takes linear time which is faster than the existing construction. We show how one can improve the space utilization further at the cost of increasing the time for creating the data structure.

Motivation & Objective

• To develop a simpler and faster construction of Bloomier filters that compactly encode functions over static sets.
• To reduce the creation time of Bloomier filters from O(n log n) to O(n), improving upon prior methods.
• To maintain O(1) query time and space efficiency comparable to existing approaches.
• To explore trade-offs between construction time, space usage, and support for dynamic updates.
• To evaluate the practical performance of the new construction on real-world data, such as URL in-degree information.

Proposed method

• Construct a random acyclic graph using two independent hash functions h₁ and h₂ mapping elements to vertices, ensuring the graph has O(n) vertices and is acyclic with constant probability.
• For a 1-bit function, solve a system of linear equations modulo 2 over the graph’s vertices using back-substitution, storing values in a lookup table g: V → {0,1}.
• Generalize to k-bit functions by using a ring ℤ/mℤ with m ≥ 2 and an additional hash function h₃ to encode function values via f(x) ≡ g(h₁(x)) + g(h₂(x)) + h₃(x) (mod m).
• Use back-substitution on the acyclic graph to solve the system of equations in O(n) time due to the tree-like structure.
• Apply bucketing techniques to reduce the O(n³) creation time of a more space-efficient variant to O(n log^{O(1)} n), improving practicality.
• Use the same graph construction and equation-solving framework for k-bit functions, with error resilience via modular arithmetic and probabilistic guarantees.

Experimental results

Research questions

• RQ1 Can a Bloomier filter be constructed in O(n) time while maintaining O(1) query time and O(n) space usage?
• RQ2 How does the use of acyclic random graphs enable efficient and deterministic solving of the function encoding equations?
• RQ3 What is the trade-off between space efficiency and construction time in Bloomier filter designs?
• RQ4 Can the construction time of a space-optimized Bloomier filter be reduced from O(n³) to near-linear time using bucketing?
• RQ5 How does the proposed method compare in practice to the prior O(n log n) construction on real-world workloads like URL in-degree graphs?

Key findings

• The proposed construction achieves O(n) creation time, which is 3 to 5 times faster than the prior O(n log n) method on real-world URL in-degree data.
• The memory footprint of the Bloomier filter is approximately 20 GB for storing in-degree information for 1.1 billion URLs, matching the space usage of the prior method.
• The number of trials required to find an acyclic graph (for the new method) or a lossless expander (for the prior method) is comparable, indicating similar probabilistic efficiency.
• The method supports O(1) query time with only 2 memory accesses and 3 hash evaluations, slightly faster than the prior scheme requiring r+1 ≥ 3 memory accesses.
• The space-optimized variant using bucketing reduces construction time from O(n³) to O(n log^{O(1)} n), making it practical while achieving (1+δ)n(log(1/ε)+k) space for any δ>0.
• The method trades off dynamic updates, requiring O(log n) time to change f(x), compared to O(1) in the prior scheme.