[Paper Review] OnePiece: A Large-Scale Distributed Inference System with RDMA for Complex AI-Generated Content (AIGC) Workflows

June Chen, Neal Xu+3 more|arXiv (Cornell University)|Jan 28, 2026

Cell Image Analysis Techniques0 citations

TL;DR

OnePiece is a large-scale distributed inference system that uses one-sided RDMA and microservice decomposition to optimize multi-stage AIGC workflows, with a novel double-ring buffer deadlock resolution and dynamic Node Manager for elastic resource allocation.

ABSTRACT

The rapid growth of AI-generated content (AIGC) has enabled high-quality creative production across diverse domains, yet existing systems face critical inefficiencies in throughput, resource utilization, and scalability under concurrent workloads. This paper introduces OnePiece, a large-scale distributed inference system with RDMA optimized for multi-stage AIGC workflows. By decomposing pipelines into fine-grained microservices and leveraging one-sided RDMA communication, OnePiece significantly reduces inter-node latency and CPU overhead while improving GPU utilization. The system incorporates a novel double-ring buffer design to resolve deadlocks in RDMA-aware memory access without CPU involvement. Additionally, a dynamic Node Manager allocates resources elastically across workflow stages in response to real-time load. Experimental results demonstrate that OnePiece reduces GPU resource consumption by 16x in Wan2.1 image-to-video generation compared to monolithic inference pipelines, offering a scalable, fault-tolerant, and efficient solution for production AIGC environments.

Motivation & Objective

• Motivate improvements in throughput, resource utilization, and scalability for complex AIGC workflows.
• Explore fine-grained microservice partitioning to reduce inter-node latency and GPU contention.
• Develop RDMA-based inter-service communication to bypass CPU overhead and enhance GPU utilization.
• Introduce a deadlock-free ring buffer design for RDMA message passing with dynamic, variable-sized data
• Provide elastic, real-time resource management via a centralized Node Manager across workflow stages.

Proposed method

• Decompose end-to-end AIGC pipelines into fine-grained microservices distributed across regionally autonomous workflow sets.
• Adopt one-sided RDMA for direct memory-to-memory inter-service data transfer to minimize CPU involvement.
• Implement a double-ring buffer to enable deadlock-free RDMA-based messaging with dynamically sized messages.
• Utilize a memory-centric, RAM/NVMe-backed database with replication for fault tolerance and low-latency result retrieval.
• Incorporate a Node Manager to elastically reallocate GPU resources across workflow stages based on real-time load.
• Provide two workflow scheduling modes (Individual Mode and Collaboration Mode) with corresponding TaskWorker behavior and a ResultDeliver routing strategy.
• Apply pipelining and load-monitoring theory to maintain steady end-to-end throughput under varying stage runtimes.

Figure 1 . The system architecture overview of OnePiece

Experimental results

Research questions

• RQ1 How can RDMA-based communication and microservice partitioning improve throughput and GPU utilization for multi-stage AIGC workloads?
• RQ2 What deadlock-free messaging mechanisms can support dynamic-sized RDMA messages without CPU intervention?
• RQ3 How should resources be allocated across pipeline stages to maintain steady-state throughput under bursty load?
• RQ4 What are effective scheduling strategies for diverse task granularities in AIGC workflows?

Key findings

• OnePiece achieves substantial resource efficiency gains, reporting a 16× reduction in GPU resource usage for Wan2.1 image-to-video generation compared to monolithic pipelines.
• The system reduces inter-node latency and CPU overhead by leveraging one-sided RDMA for inter-service communication.
• A novel double-ring buffer resolves RDMA-related deadlocks without CPU involvement while supporting dynamic message sizes.
• The Node Manager enables elastic GPU resource allocation across workflow stages in response to real-time load.
• A memory-centric database design with replication provides fast result storage and retrieval with transient data lifecycles.

Figure 2 . The Workflow Set in OnePiece