[Paper Review] Autobahn: Seamless high speed BFT

Neil Giridharan, Florian Suri-Payer+3 more|arXiv (Cornell University)|Jan 18, 2024

Distributed systems and fault tolerance4 citations

TL;DR

Autobahn is a novel partially synchronous Byzantine Fault Tolerant (BFT) consensus protocol that achieves both low latency during synchronous periods and seamless recovery from network blips by combining a highly parallel asynchronous data dissemination layer with a low-latency consensus mechanism. It eliminates the 'hangover' effect seen in traditional BFT protocols while matching the throughput of DAG-based protocols and halving their latency.

ABSTRACT

Today's practical, high performance Byzantine Fault Tolerant (BFT) consensus protocols operate in the partial synchrony model. However, existing protocols are inefficient when deployments are indeed partially synchronous. They deliver either low latency during fault-free, synchronous periods (good intervals) or robust recovery from events that interrupt progress (blips). At one end, traditional, view-based BFT protocols optimize for latency during good intervals, but, when blips occur, can suffer from performance degradation (hangovers) that can last beyond the return of a good interval. At the other end, modern DAG-based BFT protocols recover more gracefully from blips, but exhibit lackluster latency during good intervals. To close the gap, this work presents Autobahn, a novel high-throughput BFT protocol that offers both low latency and seamless recovery from blips. By combining a highly parallel asynchronous data dissemination layer with a low-latency, partially synchronous consensus mechanism, Autobahn (i) avoids the hangovers incurred by traditional BFT protocols and (ii) matches the throughput of state of the art DAG-based BFT protocols while cutting their latency in half, matching the latency of traditional BFT protocols.

Motivation & Objective

• To address the fundamental tradeoff in BFT protocols between low latency during good intervals and robustness to network blips.
• To eliminate the 'hangover' effect—persistent latency degradation after blips—common in traditional view-based BFT protocols.
• To achieve the high throughput of DAG-based BFT protocols while matching the low latency of traditional BFT protocols.
• To design a protocol that is both efficient under real-world partial synchrony and extensible to future algorithmic improvements.
• To enable practical deployment of BFT systems that maintain performance under intermittent network failures and dynamic conditions.

Proposed method

• Decouples data dissemination from consensus using a parallel, asynchronous lanes architecture that enables high-throughput, low-latency data propagation.
• Employs a lightweight, partially synchronous consensus mechanism that avoids reliance on multiple rounds of reliable broadcast (RB), reducing message delays.
• Introduces 'cuts'—aggregated views of multiple replica lanes—as the atomic unit of consensus, minimizing quorum sizes and dependency overhead.
• Uses a synchronization and ordering logic that allows replicas to converge on a total order without blocking on missing data, enhancing resilience.
• Designs the system to be modular, allowing existing consensus mechanisms to be enhanced with 'lanes' and cut-based synchronization without full protocol replacement.
• Leverages parallelism across n replicas to disseminate proposals efficiently, reducing end-to-end latency during good intervals.

Experimental results

Research questions

• RQ1 Can a BFT protocol achieve low latency during good intervals while also recovering seamlessly from network blips?
• RQ2 Is it possible to eliminate the 'hangover' effect—persistent latency degradation after a blip—without sacrificing throughput or latency?
• RQ3 Can the performance of DAG-based BFT protocols be matched in throughput while halving their latency?
• RQ4 How can asynchronous data dissemination be integrated with partially synchronous consensus to achieve both efficiency and resilience?
• RQ5 Can a modular architecture be designed to allow existing BFT systems to be enhanced with improved fault tolerance and performance?

Key findings

• Autobahn eliminates the hangover effect seen in traditional BFT protocols, ensuring that end-to-end latency recovers quickly after a blip.
• Autobahn achieves end-to-end latency of 1.5–2.0 ms under good conditions, matching the performance of traditional BFT protocols like PBFT and HotStuff.
• The protocol matches the throughput of state-of-the-art DAG-based BFT protocols such as Bullshark and Shoal, achieving over 100,000 transactions per second under high load.
• By replacing multiple rounds of reliable broadcast with a cut-based consensus mechanism, Autobahn reduces worst-case latency from 12 message delays (mds) in Bullshark to under 4 mds.
• The modular design allows existing BFT systems to be enhanced with 'lanes' and cut-based synchronization, enabling seamless performance improvements without full protocol migration.
• Empirical evaluation shows that Autobahn maintains low latency and high throughput even under intermittent network failures and high request loads.