[Paper Review] CitySim: A Drone-Based Vehicle Trajectory Dataset for Safety Oriented Research and Digital Twins

Ou Zheng, Mohamed Abdel‐Aty+4 more|arXiv (Cornell University)|Aug 23, 2022

Autonomous Vehicle Technology and Safety28 citations

TL;DR

CitySim provides drone-derived vehicle trajectories from 1140 minutes of video across 12 locations, including rotated bounding boxes, to support safety research and digital twin applications, via a five-step processing pipeline.

ABSTRACT

The development of safety-oriented research and applications requires fine-grain vehicle trajectories that not only have high accuracy, but also capture substantial safety-critical events. However, it would be challenging to satisfy both these requirements using the available vehicle trajectory datasets do not have the capacity to satisfy both.This paper introduces the CitySim dataset that has the core objective of facilitating safety-oriented research and applications. CitySim has vehicle trajectories extracted from 1140 minutes of drone videos recorded at 12 locations. It covers a variety of road geometries including freeway basic segments, signalized intersections, stop-controlled intersections, and control-free intersections. CitySim was generated through a five-step procedure that ensured trajectory accuracy. The five-step procedure included video stabilization, object filtering, multi-video stitching, object detection and tracking, and enhanced error filtering. Furthermore, CitySim provides the rotated bounding box information of a vehicle, which was demonstrated to improve safety evaluations. Compared with other video-based critical events, including cut-in, merge, and diverge events, which were validated by distributions of both minimum time-to-collision and minimum post-encroachment time. In addition, CitySim had the capability to facilitate digital-twin-related research by providing relevant assets, such as the recording locations' three-dimensional base maps and signal timings.

Motivation & Objective

• Facilitate safety-oriented research and applications with fine-grained, high-accuracy vehicle trajectories.
• Cover diverse road geometries to reflect real-world safety scenarios.
• Provide data products (rotated bounding boxes, maps, and signal timings) to support digital-twin driven studies.
• Enable analysis of safety-critical events such as cut-ins, merges, and diverges using robust metrics.

Proposed method

• Capture drone videos totaling 1140 minutes from 12 locations with varied road geometries.
• Process trajectories through a five-step pipeline: video stabilization, object filtering, multi-video stitching, object detection and tracking, and enhanced error filtering.
• Provide rotated bounding box information for vehicles to improve safety evaluations.
• Validate safety events using distributions of minimum time-to-collision and minimum post-encroachment time.
• Offer assets relevant to digital-twin research, including 3D base maps of recording locations and signal timings.

Experimental results

Research questions

• RQ1 How can high-accuracy, safety-relevant vehicle trajectories be extracted from drone footage?
• RQ2 What is the impact of using rotated bounding boxes on safety evaluations compared to standard bounding boxes?
• RQ3 How do safety-critical events (e.g., cut-ins, merges, diverges) manifest in the CitySim dataset via TTC and PET distributions?
• RQ4 How can CitySim support digital-twin research through accompanying base maps and signal timing data?

Key findings

• CitySim provides rotated bounding box information that improves safety evaluations.
• Compared with other video-based critical events, CitySim analyzes events like cut-ins, merges, and diverges using TTC and PET distributions.
• Trajectories were produced from 1140 minutes of drone video across 12 locations and diverse road geometries.
• A five-step processing pipeline was used to ensure trajectory accuracy (stabilization, filtering, stitching, detection/tracking, error filtering).
• The dataset supports digital-twin research by including 3D base maps of recording locations and signal timings.