[Paper Review] CPC: programming with a massive number of lightweight threads

Gabriel Kerneis, Juliusz Chroboczek|arXiv (Cornell University)|Feb 4, 2011

Parallel Computing and Optimization Techniques8 references3 citations

TL;DR

CPC enables programming with a massive number of lightweight, cooperative threads by translating threaded code into event-driven, system-threaded execution, allowing high-performance network servers like Hekate—a BitTorrent seeder—to sustain up to 1,000 peers with minimal CPU usage. The approach combines thread-like simplicity with event-driven efficiency, reducing memory overhead and improving code readability while enabling hybrid programming with detached native threads for blocking I/O.

ABSTRACT

Threads are a convenient and modular abstraction for writing concurrent programs, but often fairly expensive. The standard alternative to threads, event-loop programming, allows much lighter units of concurrency, but leads to code that is difficult to write and even harder to understand. Continuation Passing C (CPC) is a translator that converts a program written in threaded style into a program written with events and native system threads, at the programmer's choice. Together with two undergraduate students, we taught ourselves how to program in CPC by writing Hekate, a massively concurrent network server designed to efficiently handle tens of thousands of simultaneously connected peers. In this paper, we describe a number of programming idioms that we learnt while writing Hekate; while some of these idioms are specific to CPC, many should be applicable to other programming systems with sufficiently cheap threads.

Motivation & Objective

• To address the performance and expressiveness trade-off between traditional threads and event-driven programming in concurrent systems.
• To enable a programming model that supports thousands of concurrent connections with minimal overhead and high code readability.
• To explore the feasibility of using extremely lightweight, cooperative threads for building scalable network servers like BitTorrent seeders.
• To demonstrate that hybrid programming—combining cooperative CPC threads with detached native threads—can efficiently handle blocking I/O while maintaining performance.

Proposed method

• Translating programs written in threaded style into event-driven code using a Continuation Passing Style (CPS) transformation, enabling extremely lightweight thread execution.
• Using cooperative, non-preemptive scheduling for CPC threads, where context switches occur only at well-defined points in cps functions, reducing runtime overhead.
• Implementing timeouts via short-lived, detached timeout threads that abort I/O if no activity occurs within a specified period.
• Introducing a hybrid model where cps functions can call native functions and use cpc_detached to spawn blocking native threads for I/O operations.
• Delaying buffer allocation until after I/O readiness is confirmed via cpc_io_wait, reducing memory usage compared to blocking thread models.
• Using cpc_yield and cpc_attach to switch between cooperative and native thread execution, enabling efficient handling of both non-blocking and blocking operations.

Experimental results

Research questions

• RQ1 Can a programming model with a massive number of lightweight, cooperative threads achieve performance comparable to event-driven systems while retaining the simplicity of thread-based programming?
• RQ2 How does the use of CPS transformation and cooperative scheduling affect memory usage and system overhead in concurrent network servers?
• RQ3 To what extent can hybrid programming—combining cooperative CPC threads with detached native threads—improve efficiency in handling blocking I/O operations?
• RQ4 Can such a system be deployed effectively on embedded hardware with limited resources, such as low-end routers?

Key findings

• Hekate sustained up to 1,000 connected peers and achieved a peak throughput of 10 MB/s on a standard dual-core CPU, using less than 10% of CPU capacity.
• The system ran successfully on a 266 MHz MIPS-based router with 32 MB RAM, achieving 2.9 MB/s throughput under a 1,000-client stress test with a 1.2 GB torrent.
• Code for complex protocols like the BitTorrent handshake was reduced from over 1,000 lines in event-driven systems to fewer than 50 lines in CPC, significantly improving readability.
• Buffer allocation was deferred until I/O readiness, reducing memory usage compared to traditional blocking threads, while preserving linear control flow.
• The hybrid model using cpc_detached allowed efficient handling of disk I/O, with writes completing in under 10 ms on average, minimizing the impact of race conditions.
• The CPC runtime detected when cpc_write was used in detached mode and optimized it to behave like a regular write system call, simplifying code without performance loss.