[Paper Review] Microgrid - The microthreaded many-core architecture

M. Irfan Uddin|arXiv (Cornell University)|Sep 21, 2013

Parallel Computing and Optimization Techniques28 references4 citations

TL;DR

The paper proposes the Microgrid, a many-core architecture based on the microthreading model—a hybrid von Neumann/dataflow execution model that enables fine-grained, hardware-managed concurrency. By using hardware-scheduled threads, asynchronous completion, and register-based communication, the Microgrid achieves high latency tolerance and throughput, approaching one instruction per cycle under single-issue conditions.

ABSTRACT

Traditional processors use the von Neumann execution model, some other processors in the past have used the dataflow execution model. A combination of von Neuman model and dataflow model is also tried in the past and the resultant model is referred as hybrid dataflow execution model. We describe a hybrid dataflow model known as the microthreading. It provides constructs for creation, synchronization and communication between threads in an intermediate language. The microthreading model is an abstract programming and machine model for many-core architecture. A particular instance of this model is named as the microthreaded architecture or the Microgrid. This architecture implements all the concurrency constructs of the microthreading model in the hardware with the management of these constructs in the hardware.

Motivation & Objective

• Address the limitations of traditional von Neumann and dataflow models in exploiting parallelism at scale.
• Overcome the complexity of large matching stores in generalized dataflow models by introducing a hybrid model with structured thread families.
• Design a many-core architecture that natively supports dynamic thread creation, synchronization, and communication at the hardware level.
• Enable efficient latency tolerance through hardware multi-threading and asynchronous completion of memory and floating-point operations.
• Provide a scalable, composable execution model for future many-core systems with up to 10,000 cores and 1 million threads.

Proposed method

• Adopt a hybrid execution model combining sequential instruction execution within threads (von Neumann) with dataflow-based scheduling for thread execution.
• Introduce 'families' of threads as hierarchical, composable units resembling loops or function calls, enabling structured concurrency.
• Implement all concurrency constructs—creation, synchronization, communication—directly in hardware using dedicated registers (globals, locals, shareds, dependents).
• Use a delegation network to distribute independent thread families across cores, with dynamic load balancing and context reuse.
• Employ a windowed execution model where only a limited number of threads per core are active at once, improving resource utilization.
• Design specialized I/O cores with extended instruction sets to manage I/O asynchronously, decoupled from general-purpose cores.

Experimental results

Research questions

• RQ1 How can a hybrid execution model combine the simplicity of von Neumann with the parallelism of dataflow to improve many-core scalability?
• RQ2 What is the most effective way to manage fine-grained concurrency and asynchronous thread execution in a many-core system?
• RQ3 How can hardware-managed thread creation, synchronization, and communication be efficiently implemented at scale?
• RQ4 What architectural mechanisms enable high latency tolerance without relying on software thread scheduling?
• RQ5 How can I/O operations be efficiently offloaded and managed in a decentralized, scalable many-core environment?

Key findings

• The microthreading model enables fine-grained, hardware-managed concurrency by treating threads as first-class hardware entities, reducing software overhead.
• Asynchronous completion of memory and floating-point operations allows threads to be suspended and resumed without pipeline flushes, improving latency tolerance.
• The use of register-based communication (globals, shareds, dependents) enables efficient, low-latency synchronization and data exchange between threads in a family.
• Independent thread families are distributed equally across cores with a window size limit (e.g., 5), allowing dynamic reuse of thread contexts and efficient resource utilization.
• The architecture supports a theoretical throughput of one instruction per cycle under single-issue conditions, approaching the RISC ideal through aggressive latency hiding.
• Specialized I/O cores with extended instruction sets manage I/O operations independently, decoupling I/O from general computation and improving system scalability.