[Paper Review] What Went Wrong? Closing the Sim-to-Real Gap via Differentiable Causal Discovery

Peide Huang, Xilun Zhang+7 more|arXiv (Cornell University)|Jun 28, 2023

Time Series Analysis and Forecasting4 citations

TL;DR

This paper proposes COMPASS, a differentiable causal discovery framework that identifies and optimizes key simulator parameters to close the sim-to-real gap in robotic manipulation. By learning a causal graph linking environment parameters to trajectory discrepancies, COMPASS enables end-to-end, interpretable, and efficient parameter tuning, achieving significant improvements in trajectory alignment and task success rates over baselines in both simulation and real-world experiments.

ABSTRACT

Training control policies in simulation is more appealing than on real robots directly, as it allows for exploring diverse states in an efficient manner. Yet, robot simulators inevitably exhibit disparities from the real-world ebut{dynamics}, yielding inaccuracies that manifest as the dynamical simulation-to-reality (sim-to-real) gap. Existing literature has proposed to close this gap by actively modifying specific simulator parameters to align the simulated data with real-world observations. However, the set of tunable parameters is usually manually selected to reduce the search space in a case-by-case manner, which is hard to scale up for complex systems and requires extensive domain knowledge. To address the scalability issue and automate the parameter-tuning process, we introduce COMPASS, which aligns the simulator with the real world by discovering the causal relationship between the environment parameters and the sim-to-real gap. Concretely, our method learns a differentiable mapping from the environment parameters to the differences between simulated and real-world robot-object trajectories. This mapping is governed by a simultaneously learned causal graph to help prune the search space of parameters, provide better interpretability, and improve generalization on unseen parameters. We perform experiments to achieve both sim-to-sim and sim-to-real transfer, and show that our method has significant improvements in trajectory alignment and task success rate over strong baselines in several challenging manipulation tasks.

Motivation & Objective

• To address the sim-to-real gap in robotic control by identifying and tuning key simulator parameters that cause discrepancies between simulated and real-world dynamics.
• To automate and scale the parameter-tuning process beyond case-by-case manual selection, reducing reliance on domain expertise.
• To improve interpretability and generalization by learning a causal graph that prunes irrelevant parameters and reveals cause-effect relationships.
• To enable efficient, end-to-end optimization of simulator parameters using differentiable dynamics modeling.
• To demonstrate superior performance in trajectory alignment and task success rate in both sim-to-sim and sim-to-real transfer settings.

Proposed method

• COMPASS formulates the sim-to-real gap as a differentiable mapping from environment parameters to trajectory differences between simulation and real-world robot-object interactions.
• It jointly learns a causal graph that models the relationships between environment parameters and the sim-to-real gap, enabling structured search space pruning.
• The framework uses back-propagation through the differentiable causal model to optimize simulator parameters end-to-end, minimizing trajectory discrepancies.
• The causal graph is initialized as fully connected and pruned iteratively based on learned causal dependencies, improving efficiency and interpretability.
• The method is trained on real-world trajectory data and simulated trajectories, with policy performance evaluated in both simulated and real environments.
• It leverages a differentiable physics engine to enable gradient-based optimization of physical parameters such as mass, damping, and friction.

Experimental results

Research questions

• RQ1 Which simulator environment parameters most significantly contribute to the sim-to-real gap in robotic manipulation tasks?
• RQ2 How can we automatically identify and prioritize the most influential parameters without relying on manual, case-specific selection?
• RQ3 Can a differentiable causal model improve the efficiency and interpretability of sim-to-real parameter tuning?
• RQ4 To what extent does incorporating causal structure enhance generalization and performance in sim-to-real transfer?
• RQ5 How does the proposed method compare to gradient-based and gradient-free baselines in terms of trajectory alignment and task success rate?

Key findings

• COMPASS reduced the effective parameter search space from 67 dimensions to just 3 within 2 iterations in the Push-I task, identifying cube@dyna@mass, cube@dyna@damping, and cube@dyna@friction_sliding as the most influential parameters.
• The method achieved a task success rate of 0.70 ± 0.28 in real-world testing, significantly outperforming NPDR (0.40 ± 0.21) and EXI-Net (0.50 ± 0.22).
• Trajectory alignment results showed that COMPASS converged faster and achieved lower trajectory differences compared to NPDR and EXI-Net, with visually superior alignment in roll, pitch, and position trajectories.
• The causal graph learned by COMPASS effectively pruned irrelevant parameters, demonstrating that the method can identify key physical factors without prior assumptions.
• In sim-to-sim evaluation, agents trained with COMPASS showed higher success rates than nominal and baseline methods, confirming improved policy generalization.
• The framework enabled end-to-end optimization of simulator parameters, with environment parameters gradually approaching ground truth values over optimization iterations.