[Paper Review] Delay-Free Concurrency on Faulty Persistent Memory

Naama Ben-David, Guy E. Blelloch+2 more|arXiv (Cornell University)|Jun 12, 2018

Distributed systems and fault tolerance28 references4 citations

TL;DR

This paper presents a general simulator that transforms any concurrent program using reads, writes, and CAS operations into a persistent, fault-tolerant version with constant computation and recovery delays. By using capsule-based checkpointing and optimizations for local variable usage and memory barriers, it achieves high performance comparable to hand-optimized persistent data structures while ensuring correctness after crashes.

ABSTRACT

Non-volatile memory (NVM) promises persistent main memory that remains correct despite loss of power. This has sparked a line of research into algorithms that can recover from a system crash. Since caches are expected to remain volatile, concurrent data structures and algorithms must be redesigned to guarantee that they are left in a consistent state after a system crash, and that the execution can be continued upon recovery. However, the prospect of redesigning every concurrent data structure or algorithm before it can be used in NVM architectures is daunting. In this paper, we present a construction that takes any concurrent program with reads, writes and CASs to shared memory and makes it persistent, i.e., can be continued after one or more processes fault and have to restart. Importantly the converted algorithm has constant computational delay (preserves instruction counts on each process within a constant factor), as well as constant recovery delay (a process can recover from a fault in a constant number of instructions). We show this first for a simple transformation, and then present optimizations to make it more practical, allowing for a tradeoff for better constant factors in computational delay, for sometimes increased recovery delay. We also provide an optimized transformation that works for any normalized lock-free data structure, thus allowing more efficient constructions for a large class of concurrent algorithms. We experimentally evaluate our transformations by applying them to a queue.

Motivation & Objective

• Address the challenge of making concurrent programs durable and recoverable after system crashes in non-volatile memory (NVM) architectures.
• Overcome limitations of existing approaches like persistent transactional memory and universal constructions, which suffer from high overhead or lack generality.
• Provide a general-purpose transformation that preserves performance and enables fast recovery without requiring algorithm-specific redesign.
• Enable automatic, reusable porting of existing concurrent algorithms to persistent memory with minimal programmer effort.
• Optimize the transformation for practical performance, balancing computation delay and recovery delay through configurable capsule boundaries and code-level optimizations.

Proposed method

• Use a capsule-based checkpointing mechanism to enable recovery from faults by storing processor state in persistent memory at regular intervals.
• Implement a Constant-Delay Simulator that ensures both computation and recovery delays are bounded by a constant factor, regardless of program size.
• Introduce a Low-Computation-Delay Simulator that reduces computational overhead by increasing capsule size, at the cost of slightly higher recovery time.
• Apply optimizations such as eliminating redundant local variables, removing unnecessary memory fences, and leveraging cache-line write ordering to reduce flushes and improve performance.
• Design a Persistent Normalized Simulator for lock-free data structures, enabling more efficient transformations by exploiting structural properties of the algorithm.
• Integrate the simulator with a recoverable CAS primitive (from Attiya et al.) to ensure correct and efficient recovery of atomic operations.

Experimental results

Research questions

• RQ1 Can a general-purpose simulator be built that transforms any concurrent program with reads, writes, and CAS operations into a persistent, crash-recoverable version with constant delay?
• RQ2 How can the trade-off between computational overhead and recovery time be optimized in a persistent simulation framework?
• RQ3 To what extent can performance be improved through code-level optimizations like local variable reduction and fence elimination?
• RQ4 How does the performance of automatically transformed persistent data structures compare to hand-optimized or transactional memory-based alternatives?
• RQ5 Can the simulator be effectively applied to real concurrent data structures like queues while maintaining competitive performance and low recovery cost?

Key findings

• The Constant-Delay Simulator achieves both constant computation and recovery delays, ensuring predictable performance even under fault conditions.
• The Low-Computation-Delay Simulator reduces computational overhead by using larger capsules, improving performance by up to 1.28x in the General queue and 1.57x in the Normalized queue compared to non-optimized versions.
• Optimizations such as eliminating redundant local variables and removing fences followed by CAS operations significantly reduce flush count and improve throughput.
• The Normalized-Opt version of the queue outperforms RomulusLR for up to 5 threads and matches or exceeds LogQueue in performance, with 1.42x better throughput on a single thread.
• Recovery in the proposed system is efficient, relying only on loading the last capsule and recovering a recoverable CAS, with recovery time linear in the number of threads.
• The experimental results show that the performance cost of generality and fast recovery is minimal compared to the baseline non-persistent queue, with throughput degradation remaining low even under high contention.