[Paper Review] Comparison of Middlewares in Edge-to-Edge and Edge-to-Cloud Communication for Distributed ROS2 Systems

Jiaqiang Zhang, Xianjia Yu+3 more|arXiv (Cornell University)|Sep 14, 2023

IoT and Edge/Fog Computing4 citations

TL;DR

This study evaluates DDS, MQTT, and Zenoh as networking middlewares in distributed ROS 2 systems across edge-to-edge and edge-to-cloud communication. Using latency, throughput, and real robot trajectory drift tests on Ethernet, Wi-Fi, and 4G, Zenoh outperforms others in wireless environments, showing the smallest drift (0.05 m over 96 s), while CycloneDDS excels on Ethernet. CPU and security impacts are also analyzed, with Zenoh offering low overhead and minimal performance degradation under encryption.

ABSTRACT

The increased data transmission and number of devices involved in communications among distributed systems make it challenging yet significantly necessary to have an efficient and reliable networking middleware. In robotics and autonomous systems, the wide application of ROS\,2 brings the possibility of utilizing various networking middlewares together with DDS in ROS\,2 for better communication among edge devices or between edge devices and the cloud. However, there is a lack of comprehensive communication performance comparison of integrating these networking middlewares with ROS\,2. In this study, we provide a quantitative analysis for the communication performance of utilized networking middlewares including MQTT and Zenoh alongside DDS in ROS\,2 among a multiple host system. For a complete and reliable comparison, we calculate the latency and throughput of these middlewares by sending distinct amounts and types of data through different network setups including Ethernet, Wi-Fi, and 4G. To further extend the evaluation to real-world application scenarios, we assess the drift error (the position changes) over time caused by these networking middlewares with the robot moving in an identical square-shaped path. Our results show that CycloneDDS performs better under Ethernet while Zenoh performs better under Wi-Fi and 4G. In the actual robot test, the robot moving trajectory drift error over time (96\,s) via Zenoh is the smallest. It is worth noting we have a discussion of the CPU utilization of these networking middlewares and the performance impact caused by enabling the security feature in ROS\,2 at the end of the paper.

Motivation & Objective

• To evaluate the communication performance of DDS, MQTT, and Zenoh in multi-host ROS 2 systems under diverse network conditions.
• To assess the impact of network type—Ethernet, Wi-Fi, and 4G—on latency, throughput, and real-time control accuracy.
• To measure trajectory drift error in a real-world robotic experiment using a TurtleBot 4 navigating a square path.
• To analyze CPU utilization and the performance overhead introduced by enabling security features in ROS 2.

Proposed method

• Conducted quantitative experiments measuring end-to-end latency and throughput using ROS 2 messages of varying sizes (1k to 2MB) across Ethernet, Wi-Fi, and 4G networks.
• Deployed a multi-host ROS 2 system with a central host and a TurtleBot 4 robot, using the OptiTrack MOCAP system to track position and compute drift over time.
• Evaluated middleware performance using the 'performance_test' tool to measure CPU usage under different message loads and network conditions.
• Enabled ROS 2 security features and measured their impact on latency, throughput, and CPU usage using small (1k) messages.
• Used a publish/subscribe model with ROS 2 topics to simulate real-time robotic communication, comparing native DDS with integrated MQTT and Zenoh middlewares.
• Collected and analyzed 2200 frames per middleware to compute drift error over 96 seconds of continuous robot motion.

Figure 1 : Proposed evaluation scheme in this study

Experimental results

Research questions

• RQ1 How do DDS, MQTT, and Zenoh compare in terms of latency and throughput across Ethernet, Wi-Fi, and 4G networks in a multi-host ROS 2 system?
• RQ2 Which middleware minimizes trajectory drift error when controlling a mobile robot along a square path in real-world conditions?
• RQ3 What is the CPU utilization overhead of each middleware under different message sizes and network types?
• RQ4 How does enabling ROS 2 security features affect latency, throughput, and CPU usage for each middleware?
• RQ5 Which middleware offers the best balance of performance, efficiency, and real-time control accuracy in edge-to-cloud and edge-to-edge robotic communication?

Key findings

• Under Ethernet, CycloneDDS achieved the lowest latency (72.11 ms without security) and highest throughput (9804 bytes/sec), outperforming Zenoh and MQTT.
• On Wi-Fi and 4G, Zenoh demonstrated superior performance with the lowest latency and highest throughput, especially for larger messages.
• In real robot testing, Zenoh produced the smallest trajectory drift error—0.05 m over 96 seconds—indicating superior real-time control accuracy.
• CPU usage was consistently low across all middlewares, with Zenoh-TCP and Zenoh-broker showing minimal overhead, especially for large messages.
• Enabling ROS 2 security increased mean latency by 5.24 ms (from 72.11 ms to 77.35 ms) and reduced throughput by 103 bytes/sec, but had negligible impact on CPU usage.
• Zenoh’s performance and low resource consumption align with ROS 2 community feedback and the ROS 2 TSC’s decision to consider Zenoh for inclusion in future releases.

Figure 2 : DDS architecture in ROS 2 system