[Paper Review] Performance Evaluation of the Quorum Blockchain Platform

Arati Baliga, I Subhod+2 more|arXiv (Cornell University)|Jul 19, 2018

Blockchain Technology Applications and Security2 references60 citations

TL;DR

The paper evaluates Quorum 2.0 performance, comparing RAFT and IBFT, and analyzes throughput and latency under various workloads, block times, and private vs public contracts using micro-benchmarks.

ABSTRACT

Quorum is a permissioned blockchain platform built from the Ethereum codebase with adaptations to make it a permissioned consortium platform. It is one of the key contenders in the permissioned ledger space. Quorum supports confidentiality and privacy of smart contracts and transactions, and crash and Byzantine fault tolerant consensus algorithms. In this paper, we characterize the performance features of Quorum. We study the throughput and latency characteristics of Quorum with different workloads and consensus algorithms that it supports. Through a suite of micro-benchmarks, we explore how certain transaction and smart contract parameters can affect transaction latencies.

Motivation & Objective

• Characterize the performance features of Quorum across workloads and consensus algorithms.
• Measure throughput and latency under controlled workloads and varying block times.
• Evaluate the impact of private versus public contracts on performance.
• Identify how smart contract and payload parameters affect transaction latencies.
• Provide design guidance for application developers building on Quorum.

Proposed method

• Extend Caliper with a Quorum plugin to generate controlled workloads and measure throughput and latency.
• Set up a private Quorum network with 3 (RAFT) or 4 (IBFT) peers and multiple clients on commodity hardware.
• Use workloads including write-only, null, read, and mix to drive experiments.
• Vary block time to study its impact on throughput and latency for RAFT; compare RAFT vs IBFT performance.
• Conduct micro-benchmarks varying read/write sets, key-value store size, and payload sizes to study latency effects.
• Deploy private contracts to measure performance impact and investigate cryptographic/communication overhead.

Experimental results

Research questions

• RQ1 How do RAFT and IBFT consensus algorithms affect Quorum's throughput and latency across workloads?
• RQ2 How does changing the block time parameter influence throughput and latency under realistic transaction rates?
• RQ3 What is the performance impact of private contracts versus public contracts on Quorum?
• RQ4 How do payload sizes and key-value store sizes affect end-to-end transaction latency?
• RQ5 What are the scalability limits of Quorum under higher transaction rates and private contract usage?

Key findings

• Under RAFT, throughput remains roughly constant across block times, while latency increases with longer block times.
• IBFT yields slightly higher throughput than RAFT up to about 1500 tx/sec, but latencies with IBFT are significantly higher (often 2x or more) than with RAFT.
• All workloads scale linearly with load up to at least 1650 tx/sec (with 3 clients), showing good horizontal scalability.
• Read workloads incur the lowest latency due to local lookups; write and null workloads are dominated by block time and consensus processing.
• Private contracts reduce throughput at higher loads due to additional encryption/communication overhead; maximum observed private-contract throughput was ~900 tx/sec, with failures beyond that point attributed to consensus issues in the Quorum code.
• Latency increases with larger read/write sets and larger transaction/event payloads; payloads from 1 KB to 30 KB raise transaction latency by about 22-25%.