[Paper Review] Perspectives on Sim2Real Transfer for Robotics: A Summary of the R:SS 2020 Workshop

Sebastian Höfer, Kostas E. Bekris+13 more|arXiv (Cornell University)|Dec 7, 2020

Robot Manipulation and Learning12 references34 citations

TL;DR

This paper summarizes the debates, contributed abstracts, and panel discussions from the R:SS 2020 Sim2Real workshop, outlining practitioner guidance and open research questions for transferring robotics skills from simulation to reality.

ABSTRACT

This report presents the debates, posters, and discussions of the Sim2Real workshop held in conjunction with the 2020 edition of the "Robotics: Science and System" conference. Twelve leaders of the field took competing debate positions on the definition, viability, and importance of transferring skills from simulation to the real world in the context of robotics problems. The debaters also joined a large panel discussion, answering audience questions and outlining the future of Sim2Real in robotics. Furthermore, we invited extended abstracts to this workshop which are summarized in this report. Based on the workshop, this report concludes with directions for practitioners exploiting this technology and for researchers further exploring open problems in this area.

Motivation & Objective

• Summarize the debates on the definition, viability, and importance of Sim2Real transfer in robotics.
• Present the contributed abstracts and their themes related to bridging the reality gap.
• Provide practitioner-oriented guidance for applying Sim2Real techniques in real-world robotics tasks.
• Identify open research challenges to guide future work in Sim2Real.

Proposed method

• Organized three debates with opposing roles to explore core questions about Sim2Real.
• Summarized 18 peer-reviewed abstracts presenting methods, representations, and data-driven approaches for Sim2Real.
• Conducted a panel discussion to synthesize insights and future directions.
• Categorized insights into practitioner and researcher perspectives for actionable guidance.
• Highlighted concrete recommendations for applying Sim2Real techniques and areas needing open research.

Experimental results

Research questions

• RQ1 What is Sim2Real, and is it a distinct field or a collection of methods?
• RQ2 To what extent can simulation-based training transfer to real robotic tasks, and what limits exist?
• RQ3 What strategies (e.g., domain randomization, intermediate representations, differentiable simulators) best bridge the reality gap?
• RQ4 What are practical guidelines and open questions to advance Sim2Real toward production-level robotics?

Key findings

• Sim2Real debates highlighted cost, democratization, safety, and the reality gap, with consensus that simulation alone is not sufficient for real-world success.
• Sim2Real can be viewed both as a field of study and as a suite of techniques, but the reality gap remains a central challenge.
• Bridging approaches include domain randomization, explicit transferable abstractions, intermediate representations, and leveraging real-world data to improve simulators.
• Panel discussion emphasized the value of task-agnostic simulators with cautions about applicability, as well as promising directions like differentiable simulators and meta-learning.
• The contributed abstracts span formalization, bridging with existing simulators, data-driven model improvements, and online parameter inference, illustrating diverse approaches to Sim2Real.
• Practitioners are advised to consider bootstrapping, data efficiency, hardware-in-the-loop optimization, and careful tuning when applying Sim2Real techniques.