[Paper Review] An Open-Source Platform for High-Performance Non-Coherent On-Chip Communication

Andreas Kurth, Wolfgang Rönninger+5 more|arXiv (Cornell University)|Sep 11, 2020

Interconnection Networks and Systems30 references33 citations

TL;DR

This paper presents an open-source, modular, and topology-agnostic on-chip communication platform based on the industry-standard AMBA AXI5 protocol, enabling high-performance, non-coherent communication in heterogeneous many-core and accelerator-rich SoCs. The platform supports up to 1024-bit data width and 2.5 GHz operation, achieving 32 TB/s cross-sectional bandwidth with only 24 ns round-trip latency in a 1024-core ML training accelerator implemented in 22 nm technology.

ABSTRACT

On-chip communication infrastructure is a central component of modern systems-on-chip (SoCs), and it continues to gain importance as the number of cores, the heterogeneity of components, and the on-chip and off-chip bandwidth continue to grow. Decades of research on on-chip networks enabled cache-coherent shared-memory multiprocessors. However, communication fabrics that meet the needs of heterogeneous many-cores and accelerator-rich SoCs, which are not, or only partially, coherent, are a much less mature research area. In this work, we present a modular, topology-agnostic, high-performance on-chip communication platform. The platform includes components to build and link subnetworks with customizable bandwidth and concurrency properties and adheres to a state-of-the-art, industry-standard protocol. We discuss microarchitectural trade-offs and timing/area characteristics of our modules and show that they can be composed to build high-bandwidth (e.g., 2.5 GHz and 1024 bit data width) end-to-end on-chip communication fabrics (not only network switches but also DMA engines and memory controllers) with high degrees of concurrency. We design and implement a state-of-the-art ML training accelerator, where our communication fabric scales to 1024 cores on a die, providing 32 TB/s cross-sectional bandwidth at only 24 ns round-trip latency between any two cores.

Motivation & Objective

• To address the lack of open, extensible, and standardized platforms for non-coherent on-chip communication in heterogeneous many-core and accelerator-rich SoCs.
• To provide a modular, parametrizable, and topology-agnostic platform that supports high bandwidth, high concurrency, and seamless integration of diverse IP blocks (e.g., DMA engines, memory controllers).
• To enable end-to-end design of high-performance on-chip fabrics with verifiable timing, area, and microarchitectural trade-offs.
• To support research and industrial development through full RTL open-sourcing under a permissive license, including bridges, converters, and system-level components.

Proposed method

• The platform is built on AMBA AXI5 protocol, using valid-ready flow control and transaction ordering rules (O1–O3) to ensure correctness and support for multiple outstanding transactions.
• Core components include network (de)multiplexers, ID width converters, data width converters, and specialized modules like DMA engines and memory controllers, all implemented in synthesizable SystemVerilog.
• The design emphasizes modularity and separation of concerns—routing, buffering, and flow control are decoupled to enable flexible topologies and independent optimization.
• The platform supports customizable data widths (up to 1024 bits), concurrency (up to 256 concurrent transactions), and is fully compatible with industry-standard AXI5.
• A complete end-to-end communication fabric is demonstrated by integrating all components into a 1024-core machine learning training accelerator in 22 nm technology.
• Performance and area characterization are conducted via topographical synthesis, enabling detailed analysis of timing, area, and trade-offs across different configurations.

Experimental results

Research questions

• RQ1 Can a fully open-source, modular, and topology-agnostic on-chip communication platform be built using an industry-standard protocol to support high-bandwidth, high-concurrency communication in heterogeneous SoCs?
• RQ2 How do microarchitectural choices in network components (e.g., multiplexers, converters) affect performance, area, and timing in real-world designs?
• RQ3 To what extent can such a platform scale to support extreme bandwidth and low latency in large-scale many-core accelerators?
• RQ4 How does the platform’s open-source nature enable new research and development in on-chip network design, especially in co-designing communication fabrics with accelerators?
• RQ5 Can the platform be used to implement a production-grade, high-performance communication fabric that matches or exceeds commercial offerings in performance and flexibility?

Key findings

• The platform successfully implements a 1024-core machine learning training accelerator in 22 nm technology, achieving 32 TB/s cross-sectional bandwidth between any two cores.
• The on-chip communication fabric achieves a round-trip latency of only 24 ns between any two cores in the 1024-core configuration.
• The platform supports up to 1024-bit data width and 2.5 GHz operation, enabling high-throughput data movement across the fabric.
• The design achieves high concurrency with support for up to 256 independent transactions, enabling efficient utilization of wide data paths.
• The open-sourced RTL code enables full transparency, verification, and customization, supporting research and industrial deployment beyond the scope of proprietary IP.
• The platform’s modular components (e.g., multiplexers, converters) are shown to be scalable and composable, enabling flexible topologies and system-level integration without protocol complexity.