[Paper Review] Byzantine-Tolerant Register in a System with Continuous Churn

Saptaparni Kumar, Jennifer L. Welch|arXiv (Cornell University)|Oct 13, 2019

Distributed systems and fault tolerance32 references4 citations

TL;DR

This paper presents the first Byzantine-tolerant atomic register emulation in a dynamic system with continuous churn, where servers and clients can join and leave continuously and no upper bound exists on system size. The algorithm ensures atomicity and safety under the condition that churn is bounded by a constant fraction α of the system size over any time interval and up to a constant number f of Byzantine servers exist, proving impossibility when f is unknown.

ABSTRACT

A shared read/write register emulation provides the illusion of shared-memory on top of message-passing models. The main hurdle with such emulations is dealing with server faults in the system. Several crash-tolerant register emulations in static systems require algorithms to replicate the value of the shared register onto a majority of servers. Majority correctness is necessary for such emulations. Byzantine faults are considered to be the worst kind of faults that can happen in any distributed system. Emulating a Byzantine-tolerant register requires replicating the register value on to more than two-thirds of the servers. Emulating a register in a dynamic system where servers and clients can enter and leave the system and be faulty is harder than in static systems. There are several crash-tolerant register emulations for dynamic systems. This paper presents the first emulation of a multi-reader multi-writer atomic register in a system that can withstand nodes continually entering and leaving, imposes no upper bound on the system size and can tolerate Byzantine servers. The algorithm works as long as the number of servers entering and leaving during a fixed time interval is at most a constant fraction of the system size at the beginning of the interval, and as long as the number of Byzantine servers in the system is at most f. Although our algorithm requires that there be a constant known upper bound on the number of Byzantine servers, this restriction is unavoidable, as we show that it is impossible to emulate an atomic register if the system size and maximum number of servers that can be Byzantine in the system is unknown to the nodes.

Motivation & Objective

• To design a Byzantine-tolerant atomic register in a dynamic system with continuous churn, no upper bound on system size, and no requirement for churn to ever stop.
• To address the challenge of maintaining consistency and safety in a system where servers and clients continuously join and leave, and where Byzantine faults may occur.
• To establish conditions under which such a register can be safely emulated, particularly bounding churn and the number of Byzantine servers.
• To prove that a uniform algorithm cannot be implemented in this model, highlighting fundamental limits of dynamic Byzantine systems.
• To demonstrate that a known upper bound on the number of Byzantine servers is necessary, as the problem becomes impossible without it.

Proposed method

• The algorithm uses a timestamp-based total ordering mechanism to ensure atomicity, assigning unique timestamps to all operations based on their start times and coordination with quorum sets.
• It employs a Byzantine quorum system (BQS) where read and write operations must interact with overlapping sets of servers to ensure consistency, even under Byzantine faults.
• The system assumes that the number of servers entering or leaving during any fixed time interval is at most α times the system size at the start of the interval, ensuring bounded churn.
• Clients and servers coordinate via a leader-based mechanism that maintains quorum integrity and ensures that all operations are ordered consistently across the system.
• The algorithm uses a two-phase protocol for writes: first, a write request is sent to a quorum of servers; second, the write is committed only after a majority of servers acknowledge.
• A total order is constructed on all operations by ordering writes by timestamp and placing reads after the write they read from, ensuring atomicity and real-time ordering.

Experimental results

Research questions

• RQ1 Can an atomic register be emulated in a dynamic system with continuous churn and no upper bound on system size, while tolerating Byzantine server faults?
• RQ2 Is it possible to achieve atomicity and safety in such a system without requiring churn to eventually stop or bounding the system size?
• RQ3 What are the necessary conditions on churn rate (α) and the number of Byzantine servers (f) for a correct and safe emulation to exist?
• RQ4 Is it possible to implement a uniform algorithm for Byzantine-tolerant register emulation in this model, or are there fundamental impossibility results?
• RQ5 Is the assumption of a known upper bound on f necessary, or can the system tolerate an unknown number of Byzantine servers?

Key findings

• The paper presents the first algorithm that emulates a Byzantine-tolerant atomic register in a dynamic system with continuous churn, no upper bound on system size, and no requirement for churn to cease.
• The algorithm ensures atomicity by constructing a total order of operations based on timestamps and quorum coordination, guaranteeing that every read returns the value of the latest preceding write.
• The system is correct as long as the churn rate is bounded by a constant fraction α of the system size over any fixed interval and the number of Byzantine servers is bounded by a known constant f.
• The paper proves that it is impossible to implement such a register if the number of Byzantine servers is unknown, establishing a fundamental lower bound on system assumptions.
• The authors show that the assumption of a known upper bound on f is unavoidable, as relaxing it renders the problem unsolvable.
• The algorithm is proven correct under the given constraints, and the correctness relies on a careful construction of operation ordering that respects real-time execution order and ensures consistency.