[Paper Review] LGSVL Simulator: A High Fidelity Simulator for Autonomous Driving

Guodong Rong, Byung Hyun Shin+15 more|arXiv (Cornell University)|May 7, 2020

Autonomous Vehicle Technology and Safety20 references48 citations

TL;DR

LGSVL Simulator is a Unity-based, open-source, high-fidelity autonomous driving simulator that integrates with Autoware and Apollo, supports customizable sensors and HD maps, and enables SIL/HIL testing and synthetic data generation.

ABSTRACT

Testing autonomous driving algorithms on real autonomous vehicles is extremely costly and many researchers and developers in the field cannot afford a real car and the corresponding sensors. Although several free and open-source autonomous driving stacks, such as Autoware and Apollo are available, choices of open-source simulators to use with them are limited. In this paper, we introduce the LGSVL Simulator which is a high fidelity simulator for autonomous driving. The simulator engine provides end-to-end, full-stack simulation which is ready to be hooked up to Autoware and Apollo. In addition, simulator tools are provided with the core simulation engine which allow users to easily customize sensors, create new types of controllable objects, replace some modules in the core simulator, and create digital twins of particular environments.

Motivation & Objective

• Demonstrate a high-fidelity, open-source simulator for autonomous driving that integrates with major AD stacks (Autoware and Apollo).
• Showcase sensor customization, digital twin environments, and HD map tooling to support research and development.
• Enable end-to-end testing, data generation, and scenario-based evaluation through a flexible, extensible framework.

Proposed method

• Use Unity game engine with HDRP for photorealistic rendering and a full-stack simulation engine.
• Provide a communication bridge to connect LGSVL with ROS, ROS2, and Cyber RT to Autoware and Apollo.
• Offer editable sensor configurations via JSON to customize intrinsic/extrinsic parameters and add plugin sensors.
• Support FMI, IPC, or shared libraries for integrating external vehicle dynamics models.
• Allow creation, editing, and exporting of HD maps in formats like Apollo HD Map, Lanelet2, and OpenDrive.
• Enable scenario scripting via Python API and deterministic physics for repeatable testing.

Experimental results

Research questions

• RQ1 How can a high-fidelity simulator be integrated with existing open-source autonomous driving stacks (Autoware and Apollo) through flexible bridges?
• RQ2 Can LGSVL simulate rich sensor suites and HD maps to support end-to-end and module-level AD testing?
• RQ3 To what extent can digital twins and scenario generation accelerate perception, planning, and control testing?
• RQ4 How effective is the simulator for SIL/HIL testing and synthetic data generation for ML training?
• RQ5 What is the role of digital twins and SCENIC-generated scenarios in identifying useful real-world test cases?

Key findings

• LGSVL provides end-to-end simulation with bridges for ROS, ROS2, and Cyber RT to enable integration with Autoware and Apollo.
• The engine supports customizable sensors, vehicle dynamics interfaces, and FMI-based co-simulation for external dynamics models.
• HD maps can be created, edited, and exported/imported in Apollo HD Map, Lanelet2, and OpenDrive formats for cross-stack compatibility.
• Digital twins of real locations (e.g., GoMentum Station) and SCENIC-driven scenario generation enable broad scenario coverage and transfer to real facilities.
• The platform supports SIL/HIL testing and synthetic data generation, including ground-truth labels for perception training and OpenAI Gym integration for reinforcement learning.