[论文解读] BLOCKBENCH: A Framework for Analyzing Private Blockchains

Tien Tuan Anh Dinh, Ji Wang+4 more|arXiv (Cornell University)|Mar 12, 2017

Blockchain Technology Applications and Security被引用 210

一句话总结

Blockbench 提供了首个基准框架，用于比较允许权限的区块链平台（Ethereum、Parity、Hyperledger）在共识、数据和执行层面的性能，通过真实与综合负载来测量吞吐量、时延、扩展性和容错性。

ABSTRACT

Blockchain technologies are taking the world by storm. Public blockchains, such as Bitcoin and Ethereum, enable secure peer-to-peer applications like crypto-currency or smart contracts. Their security and performance are well studied. This paper concerns recent private blockchain systems designed with stronger security (trust) assumption and performance requirement. These systems target and aim to disrupt applications which have so far been implemented on top of database systems, for example banking, finance applications. Multiple platforms for private blockchains are being actively developed and fine tuned. However, there is a clear lack of a systematic framework with which different systems can be analyzed and compared against each other. Such a framework can be used to assess blockchains' viability as another distributed data processing platform, while helping developers to identify bottlenecks and accordingly improve their platforms. In this paper, we first describe BlockBench, the first evaluation framework for analyzing private blockchains. It serves as a fair means of comparison for different platforms and enables deeper understanding of different system design choices. Any private blockchain can be integrated to BlockBench via simple APIs and benchmarked against workloads that are based on real and synthetic smart contracts. BlockBench measures overall and component-wise performance in terms of throughput, latency, scalability and fault-tolerance. Next, we use BlockBench to conduct comprehensive evaluation of three major private blockchains: Ethereum, Parity and Hyperledger Fabric. The results demonstrate that these systems are still far from displacing current database systems in traditional data processing workloads. Furthermore, there are gaps in performance among the three systems which are attributed to the design choices at different layers of the software stack.

研究动机与目标

• 介绍 Blockbench 作为一个公平、可扩展的评估私有区块链的框架。
• 通过简单的 API 实现对任意私有区块链的集成，以针对现实工作负载进行基准测试。
• 评估共识、数据模型和执行层的性能，以识别瓶颈和设计权衡。
• 为私有区块链在传统数据处理工作负载中的可行性提供洞见。

提出的方法

• 定义一个四层区块链抽象（共识、数据模型、执行、应用）及相应的工作负载。
• 为 Ethereum、Parity 和 Hyperledger 实现 Blockbench 后端，以便使用通用 API 进行基准测试。
• 开发宏基准工作负载（YCSB-like 键值存储、Smallbank OLTP、EtherId、Doubler、WavesPresale）和微基准（DoNothing、IOHeavy、CPUHeavy、Analytics）。
• 在模拟网络故障和分区攻击的条件下，衡量吞吐量、时延、可扩展性、容错性和安全性。
• 提供一个驱动程序，处理异步交易提交和轮询，以在各后端收集区块确认数据。

实验结果

研究问题

• RQ1 以私有区块链工作负载来看，Ethereum、Parity 与 Hyperledger 在吞吐量、时延和可扩展性方面有何比较？
• RQ2 在不同平台上，共识、数据和执行层会出现哪些瓶颈？
• RQ3 容错性和安全性（分叉、分区攻击）如何影响私有区块链的性能？
• RQ4 私有区块链能否相对于传统数据库高效地支持数据库型工作负载和分析？

主要发现

• Hyperledger 在七项基准测试中始终优于另外两个系统。
• 在传统数据处理工作负载方面，区块链性能仍远低于最先进的数据库。
• 共识协议在很大程度上解释了 Ethereum 和 Hyperledger 的应用层性能差距。
• Parity 在执行阶段存在处理瓶颈，而 Ethereum 和 Parity 在执行和数据层上均存在瓶颈。
• 存在可扩展性极限，在评估场景中 Hyperledger 的扩展性未能超过 16 个节点。
• Blockbench 提供基线并识别瓶颈，以引导对私有区块链平台的进一步优化。