[论文解读] Joins on Samples: A Theoretical Guide for Practitioners

Dawei Huang, Dong Young Yoon+2 more|arXiv (Cornell University)|Dec 7, 2019

Data Management and Algorithms参考文献 56被引用 4

一句话总结

本文为使用离线样本近似连接（equi-joins）提供了理论基础，表明最优采样策略——特别是结合分层采样、全集采样和伯努利采样三者的混合方案——可将方差控制在信息论下限的常数倍以内。本文还提出一种去中心化变体，适用于实际部署，实现准确、低开销的连接估算，而无需进行全量数据重分布。

ABSTRACT

Despite decades of research on approximate query processing (AQP), our understanding of sample-based joins has remained limited and, to some extent, even superficial. The common belief in the community is that joining random samples is futile. This belief is largely based on an early result showing that the join of two uniform samples is not an independent sample of the original join, and that it leads to quadratically fewer output tuples. However, unfortunately, this result has little applicability to the key questions practitioners face. For example, the success metric is often the final approximation's accuracy, rather than output cardinality. Moreover, there are many non-uniform sampling strategies that one can employ. Is sampling for joins still futile in all of these settings? If not, what is the best sampling strategy in each case? To the best of our knowledge, there is no formal study answering these questions. This paper aims to improve our understanding of sample-based joins and offer a guideline for practitioners building and using real-world AQP systems. We study limitations of offline samples in approximating join queries: given an offline sampling budget, how well can one approximate the join of two tables? We answer this question for two success metrics: output size and estimator variance. We show that maximizing output size is easy, while there is an information-theoretical lower bound on the lowest variance achievable by any sampling strategy. We then define a hybrid sampling scheme that captures all combinations of stratified, universe, and Bernoulli sampling, and show that this scheme with our optimal parameters achieves the theoretical lower bound within a constant factor. Since computing these optimal parameters requires shuffling statistics across the network, we also propose a decentralized variant where each node acts autonomously using minimal statistics.

研究动机与目标

• 为解决长期以来关于对均匀样本进行连接在近似查询处理（AQP）中本质上无用的误解。
• 正式分析在两种关键成功指标下——输出大小与估计器方差——离线采样在近似等值连接时的局限性。
• 推导出信息论最优的采样策略，以最小化方差，实现性能接近理论下限的常数倍。
• 设计一种去中心化采样变体，实现最小化网络通信开销的实用部署。
• 在真实世界的SQL与AQP引擎上对理论发现进行实证验证，证明其在实际应用中的准确性与效率。

提出的方法

• 提出一种混合采样方案，将分层采样、全集采样与伯努利采样统一于单一框架中，通过各组件的采样率参数化。
• 推导出任何离线采样策略在等值连接上可实现的最小方差的信息论下限。
• 识别出混合方案的最优采样参数，使其方差在理论下限的常数倍以内。
• 引入一种去中心化变体，其中每个节点仅使用本地统计信息即可计算其自身的最优采样参数，避免全局协调。
• 采用变分子采样与统计估计技术，确保对连接结果上的聚合函数（如COUNT、SUM）实现准确近似。
• 使用真实AQP引擎（如VerdictDB、Quickr）与合成工作负载对方法进行验证，以衡量准确率与加速比。

实验结果

研究问题

• RQ1 对均匀样本进行连接是否真的对近似等值连接毫无用处？还是非均匀采样策略可实现高精度？
• RQ2 任何离线采样策略在等值连接上可实现的理论最小方差是多少？该下限在实际中是否可被逼近？
• RQ3 统一的采样框架能否同时捕捉分层采样、全集采样与伯努利采样的优势，同时最小化方差？
• RQ4 如何在无需全网协调的情况下，以高效方式计算最优采样参数？
• RQ5 理论发现能在多大程度上转化为真实AQP系统中的实际精度提升？

主要发现

• 本文建立了任何离线采样策略在等值连接上可实现方差的信息论下限，为最优性能提供了基准。
• 所提出的混合采样方案在参数最优调优下，其方差处于该理论下限的常数倍以内。
• 该采样方案的去中心化变体使每个节点仅使用本地统计信息即可计算自身最优采样参数，显著降低通信开销。
• 在SQL与AQP引擎上的实证评估表明，所提方法在实现高精度的同时获得显著加速，优于均匀采样与全集采样。
• 结果表明，当连接列分布呈偏态时，使用最优非均匀采样策略进行离线样本连接并非徒劳，反而能显著提升精度。
• 该框架通过提供更具代表性的连接结果样本，提升了数据库学习与选择度估计的准确性。