[Paper Review] Shoal++: High Throughput DAG BFT Can Be Fast!

Balaji Arun, Zekun Li+3 more|arXiv (Cornell University)|May 30, 2024

Medical Imaging Techniques and ApplicationsMedicine3 citations

TL;DR

Shoal++ is a high-throughput DAG-based Byzantine Fault Tolerant (BFT) consensus protocol that reduces end-to-end transaction latency to 4.5 message delays by increasing anchor frequency, using a deterministic reputation scheme for anchor selection, and enabling parallel DAG instances. It achieves throughput comparable to state-of-the-art DAG-BFT systems while reducing latency by up to 60% compared to prior work.

ABSTRACT

Today's practical partially synchronous Byzantine Fault Tolerant (BFT) consensus protocols trade off low latency and high throughput. On the one end, traditional BFT protocols such as PBFT and its derivatives optimize for latency. They require, in fault-free executions, only 3 message exchanges to commit, the optimum for BFT consensus. However, this class of protocols typically relies on a single leader, hampering throughput scalability. On the other end, a new class of so-called DAG-BFT protocols demonstrates how to achieve highly scalable throughput by separating data dissemination from consensus, and using every replica as proposer. Unfortunately, existing DAG-BFT protocols pay a steep latency premium, requiring on average 10.5 message exchanges to commit a transactions. This work aims to soften this tension and proposes Shoal++, a novel DAG-based BFT consensus system that offers the throughput of DAGs while reducing commit latency to an average of 4.5 message exchanges. Our empirical findings are encouraging, showing that Shoal++ achieves throughput comparable to state-of-the-art DAG BFT solutions while reducing latency by up to 60%.

Motivation & Objective

• To close the latency gap between high-throughput DAG-BFT protocols and low-latency traditional BFT protocols like PBFT.
• To address the high end-to-end latency (10.5 message delays) in existing DAG-BFT protocols despite their high throughput.
• To enable fast, scalable, and resilient consensus in partially synchronous environments using DAG structures.
• To design a system that maintains the throughput benefits of DAG-BFT while approaching the 3-message-delay optimum of traditional BFT protocols.
• To improve practicality by eliminating reliance on data fetching on the critical path, enhancing resilience to Byzantine and slow replicas.

Proposed method

• Increases anchor frequency to one per round by dynamically reinterpreting anchor schedules, reducing the number of rounds needed for commitment.
• Employs a deterministic reputation scheme to select only the fastest and best-connected replicas as anchor candidates, improving reliability and performance.
• Introduces multiple parallel DAG instances per replica to increase proposal frequency and reduce queuing latency for transactions.
• Leverages the Bullshark Direct Commit rule to allow early commitment of proposals based on DAG structure, minimizing latency in fault-free cases.
• Uses certified DAGs to ensure durability and avoid data fetching on the critical path, enhancing resilience to Byzantine and slow replicas.
• Eliminates leader/anchor timeouts in normal operation, relying only on deterministic reputation and structural guarantees for liveness.

Figure 1 . Narwhal’s round-based DAG from a validator’s local point of view. All nodes are certified.

Experimental results

Research questions

• RQ1 Can DAG-based BFT protocols achieve end-to-end latency close to the theoretical minimum of 3 message delays while maintaining high throughput?
• RQ2 What are the primary sources of latency in existing DAG-BFT protocols, and how can they be systematically reduced?
• RQ3 How does increasing anchor frequency per round impact consensus latency and system resilience?
• RQ4 To what extent can a deterministic reputation system improve anchor selection and reduce latency in DAG-BFT?
• RQ5 Can parallel DAG instances effectively reduce queuing latency without compromising consistency or increasing complexity?

Key findings

• Shoal++ reduces the average end-to-end transaction latency from 10.5 message delays (in Shoal) to 4.5 message delays in the fault-free case.
• The protocol achieves throughput comparable to state-of-the-art DAG-BFT systems while reducing latency by up to 60%.
• The use of multiple parallel DAG instances effectively reduces queuing latency by increasing proposal frequency.
• The deterministic reputation scheme ensures that only high-performing replicas are selected as anchors, improving reliability and reducing latency.
• Shoal++ avoids data fetching on the critical path by using certified DAGs, making it resilient to Byzantine and slow replicas.
• The system demonstrates that high throughput and low latency in DAG-BFT are not mutually exclusive, approaching the 3-message-delay optimum of traditional BFT protocols.

Figure 2 . Bullshark Commit example. Replica 4 observes two ( $f+1$ ) votes for anchor A1, and thus directly commits it. Replica 1 observes only one ( $<f+1$ ) vote, and thus cannot directly commit A1. Upon committing A2, however, it observes that A1 is present in its causal history and indirectly c