[Paper Review] Sampling, Communication, and Prediction Co-Design for Synchronizing the Real-World Device and Digital Model in Metaverse

Zhen Meng, Changyang She+2 more|arXiv (Cornell University)|Jul 31, 2022

Virtual Reality Applications and Impacts4 citations

TL;DR

This paper proposes a co-design framework integrating sampling, communication, and prediction to minimize communication load while maintaining tight synchronization between a real-world robotic arm and its digital twin in the metaverse. Using a knowledge-augmented constrained deep reinforcement learning algorithm (KC-TD3), the framework dynamically optimizes sampling rate and prediction horizon, reducing average communication load by up to 87% under strict error constraints and outperforming static benchmarks in stability and tail error performance.

ABSTRACT

The metaverse has the potential to revolutionize the next generation of the Internet by supporting highly interactive services with the help of Mixed Reality (MR) technologies; still, to provide a satisfactory experience for users, the synchronization between the physical world and its digital models is crucial. This work proposes a sampling, communication and prediction co-design framework to minimize the communication load subject to a constraint on tracking the Mean Squared Error (MSE) between a real-world device and its digital model in the metaverse. To optimize the sampling rate and the prediction horizon, we exploit expert knowledge and develop a constrained Deep Reinforcement Learning (DRL) algorithm, named Knowledge-assisted Constrained Twin-Delayed Deep Deterministic (KC-TD3) policy gradient algorithm. We validate our framework on a prototype composed of a real-world robotic arm and its digital model. Compared with existing approaches: (1) When the tracking error constraint is stringent (MSE=0.002 degrees), our policy degenerates into the policy in the sampling-communication co-design framework. (2) When the tracking error constraint is mild (MSE=0.007 degrees), our policy degenerates into the policy in the prediction-communication co-design framework. (3) Our framework achieves a better trade-off between the average MSE and the average communication load compared with a communication system without sampling and prediction. For example, the average communication load can be reduced up to 87% when the track error constraint is 0.002 degrees. (4) Our policy outperforms the benchmark with the static sampling rate and prediction horizon optimized by exhaustive search, in terms of the tail probability of the tracking error. Furthermore, with the assistance of expert knowledge, the proposed algorithm KC-TD3 achieves better convergence time, stability, and final policy performance.

Motivation & Objective

• To address the challenge of synchronizing real-world devices with digital models in the metaverse with minimal communication overhead.
• To maintain low tracking error (MSE) despite network constraints such as latency and packet loss.
• To co-optimize sampling rate, communication, and prediction horizon for improved efficiency and reliability.
• To develop a reinforcement learning-based policy that adapts dynamically to system dynamics and constraints.
• To validate the framework on a real robotic arm and digital twin prototype under realistic network conditions.

Proposed method

• Proposes a co-design framework that jointly optimizes sampling, communication, and prediction for real-time synchronization in the metaverse.
• Introduces a constrained Deep Reinforcement Learning (DRL) algorithm, KC-TD3, which integrates expert knowledge to guide exploration and enforce MSE constraints.
• Uses a Markov decision process formulation where the state includes historical trajectories, predicted states, and system parameters to enable long-horizon decision-making.
• Employs a de-correlation time-based observation horizon to ensure the Markov property, enabling stable learning.
• Designs a reward function that penalizes high communication load and tracking error, with constraints enforced via a penalty-based method.
• Validates the framework on a real robotic arm and digital twin system with variable packet loss rates and MSE constraints.

Figure 1: Proposed co-design framework to synchronize a real-world device and its digital model in the metaverse.

Experimental results

Research questions

• RQ1 Can a co-design framework of sampling, communication, and prediction significantly reduce communication load while maintaining low tracking error in metaverse synchronization?
• RQ2 How does the proposed KC-TD3 algorithm perform compared to static policies optimized via exhaustive search in terms of convergence, stability, and tracking error distribution?
• RQ3 Under what conditions does the framework degenerate into pure sampling-communication co-design or prediction-communication co-design?
• RQ4 To what extent does expert knowledge improve the convergence speed and final policy performance of the DRL agent?
• RQ5 How does the system perform under varying packet loss probabilities in real-world communication channels?

Key findings

• The proposed KC-TD3 algorithm reduces average communication load by up to 87% when the average tracking error constraint is 0.002°, significantly outperforming baseline systems.
• When the MSE constraint is stringent (0.002°), the policy converges to a behavior resembling pure sampling-communication co-design, validating its adaptability.
• Under a milder constraint (0.007°), the policy shifts toward prediction-communication co-design, demonstrating its ability to exploit prediction when allowed.
• KC-TD3 achieves better tail probability performance for tracking error than exhaustive search-based static policies, indicating more consistent and reliable synchronization.
• With expert knowledge, KC-TD3 converges faster, exhibits greater stability, and achieves superior final policy performance compared to unconstrained DRL baselines.
• Even under 10% packet loss, KC-TD3 reduces normalized communication load by up to 75% compared to the baseline while maintaining an average tracking error of 0.002°.

Figure 2: The timing sequence of the proposed co-design framework (The senor belongs to the real world device for data generation and the transmitter belongs to a local server. The receiver and functions for construction and predication are deployed at the cloud server that operates the metaverse).