[Paper Review] Datacenter RPCs can be General and Fast

Anuj Kalia, Michael Kaminsky+1 more|arXiv (Cornell University)|Jun 2, 2018

Interconnection Networks and Systems45 references74 citations

TL;DR

eRPC is a general-purpose RPC library that matches specialized systems in performance on commodity datacenter networks, including lossy Ethernet, without network-specific hardware.

ABSTRACT

It is commonly believed that datacenter networking software must sacrifice generality to attain high performance. The popularity of specialized distributed systems designed specifically for niche technologies such as RDMA, lossless networks, FPGAs, and programmable switches testifies to this belief. In this paper, we show that such specialization is not necessary. eRPC is a new general-purpose remote procedure call (RPC) library that offers performance comparable to specialized systems, while running on commodity CPUs in traditional datacenter networks based on either lossy Ethernet or lossless fabrics. eRPC performs well in three key metrics: message rate for small messages; bandwidth for large messages; and scalability to a large number of nodes and CPU cores. It handles packet loss, congestion, and background request execution. In microbenchmarks, one CPU core can handle up to 10 million small RPCs per second, or send large messages at 75 Gbps. We port a production-grade implementation of Raft state machine replication to eRPC without modifying the core Raft source code. We achieve 5.5 microseconds of replication latency on lossy Ethernet, which is faster than or comparable to specialized replication systems that use programmable switches, FPGAs, or RDMA.

Motivation & Objective

• Demonstrate that general-purpose RPCs can achieve state-of-the-art performance on typical datacenter networks.
• Identify design principles that deliver high small-message throughput, large-message bandwidth, and scalability.
• Show that eRPC can operate efficiently on lossy Ethernet without lossless fabrics.
• Port Raft and Masstree to eRPC without core Raft modifications to prove practicality.

Proposed method

• Design a high-performance RPC framework optimized for the common case (small messages, congestion-free network, short handlers).
• Use single-BDP flow control to prevent switch-buffer drops and achieve scalability with constant NIC memory footprint per core.
• Implement a zero-copy capable message buffer (msgbuf) layout and unsignaled transmission with a controlled flush for rare retransmissions or failures.
• Support a client-driven wire protocol with session credits to enable end-to-end flow control and scalable session management.
• Evaluate on multiple NICs and network types, including lossy Ethernet and InfiniBand, and compare against specialized systems.

Experimental results

Research questions

• RQ1 Can a general-purpose RPC library achieve performance comparable to specialized systems on commodity datacenter networks?
• RQ2 What design principles enable high message-rate for small RPCs and high bandwidth for large messages without lossless fabrics?
• RQ3 How scalable is the approach with many nodes and CPU cores, given NIC and switch constraints?
• RQ4 Can eRPC integrate with existing production-grade systems (e.g., Raft, Masstree) without core source changes?

Key findings

• One core can handle up to 10 million small RPCs per second.
• Large messages can be sent at 75 Gbps with one core.
• Median RPC latency for microbenchmarks is 2.3 microseconds.
• Three-way Raft replication latency on lossy Ethernet is 5.5 microseconds.
• eRPC achieves high-throughput and low latency without lossless fabrics and without modifying core Raft sources.