[Paper Review] Spatial-Temporal Map Vehicle Trajectory Detection Using Dynamic Mode Decomposition and Res-UNet+ Neural Networks

Tianya Zhang, Peter J. Jin|arXiv (Cornell University)|Jan 13, 2022

Anomaly Detection Techniques and Applications60 references4 citations

TL;DR

This paper proposes a novel method for vehicle trajectory detection from high-angle traffic camera videos using Dynamic Mode Decomposition (DMD) to separate spatial-temporal maps (STMap) into foreground (vehicles) and background (scene), combined with a custom Res-UNet+ deep neural network for semantic segmentation. The approach achieves high accuracy and robustness, significantly improving trajectory extraction over prior methods and producing cleaned, high-quality trajectory data for traffic flow research.

ABSTRACT

This paper presents a machine-learning-enhanced longitudinal scanline method to extract vehicle trajectories from high-angle traffic cameras. The Dynamic Mode Decomposition (DMD) method is applied to extract vehicle strands by decomposing the Spatial-Temporal Map (STMap) into the sparse foreground and low-rank background. A deep neural network named Res-UNet+ was designed for the semantic segmentation task by adapting two prevalent deep learning architectures. The Res-UNet+ neural networks significantly improve the performance of the STMap-based vehicle detection, and the DMD model provides many interesting insights for understanding the evolution of underlying spatial-temporal structures preserved by STMap. The model outputs were compared with the previous image processing model and mainstream semantic segmentation deep neural networks. After a thorough evaluation, the model is proved to be accurate and robust against many challenging factors. Last but not least, this paper fundamentally addressed many quality issues found in NGSIM trajectory data. The cleaned high-quality trajectory data are published to support future theoretical and modeling research on traffic flow and microscopic vehicle control. This method is a reliable solution for video-based trajectory extraction and has wide applicability.

Motivation & Objective

• To address the challenge of extracting accurate vehicle trajectories from high-angle traffic camera videos under complex conditions.
• To improve upon existing image processing and deep learning models in segmenting vehicles from spatial-temporal maps (STMap).
• To leverage Dynamic Mode Decomposition (DMD) for separating dynamic vehicle strands from static background structures in STMap.
• To develop a specialized deep neural network, Res-UNet+, tailored for STMap-based vehicle segmentation.
• To produce a cleaned, high-quality trajectory dataset to support future research in traffic flow modeling and microscopic vehicle control.

Proposed method

• Dynamic Mode Decomposition (DMD) is applied to decompose the Spatial-Temporal Map (STMap) into low-rank background and sparse foreground components, isolating vehicle motion patterns.
• A modified Res-UNet+ architecture is designed by adapting ResNet and U-Net structures to enhance feature learning and segmentation precision on STMap data.
• The Res-UNet+ model performs semantic segmentation on the DMD-processed STMap to detect and track vehicle trajectories.
• The method integrates DMD’s structural decomposition with deep learning to improve robustness against noise, occlusions, and varying lighting.
• The pipeline processes video frames into STMap representations, applies DMD for background subtraction, and uses Res-UNet+ for final vehicle instance segmentation.
• The final output is a set of high-accuracy, clean vehicle trajectories validated against ground truth and prior models.

Experimental results

Research questions

• RQ1 Can DMD effectively separate dynamic vehicle motion from static background in STMap representations of traffic camera videos?
• RQ2 How does the proposed Res-UNet+ model compare to mainstream semantic segmentation networks in detecting vehicles on STMap?
• RQ3 To what extent does the combined DMD-Res-UNet+ framework improve trajectory detection accuracy under real-world challenges like occlusion and low contrast?
• RQ4 Can the method produce high-quality, cleaned trajectory data suitable for traffic flow modeling and microscopic control research?
• RQ5 What insights does DMD provide into the underlying spatial-temporal dynamics of vehicle movement in traffic?

Key findings

• The DMD-based preprocessing significantly enhances the signal-to-noise ratio in STMap, enabling clearer extraction of vehicle strands.
• The Res-UNet+ model outperforms both traditional image processing and mainstream deep learning models in vehicle segmentation accuracy on STMap.
• The method demonstrates robustness against challenging conditions such as partial occlusions, varying lighting, and high vehicle density.
• The final trajectory outputs show a substantial reduction in errors compared to existing NGSIM data, addressing long-standing quality issues.
• The cleaned trajectory dataset produced is publicly released, supporting future research in traffic dynamics and vehicle control.
• The integration of DMD and deep learning provides interpretable insights into the evolution of spatial-temporal traffic structures.