[Paper Review] DDK: A Deep Koopman Approach for Dynamics Modeling and Trajectory Tracking of Autonomous Vehicles

Yongqian Xiao|arXiv (Cornell University)|Oct 27, 2021

Model Reduction and Neural Networks34 references4 citations

TL;DR

This paper proposes Deep Direct Koopman (DDK), a deep learning approach that models nonlinear vehicle dynamics as a linear time-invariant (LTI) system via trainable Koopman eigenvalues and system matrices, while preserving physical interpretability by directly incorporating original vehicle states into the Koopman eigenfunctions. The method enables high-accuracy trajectory tracking using a DDK-MPC controller, achieving sub-meter error and real-time performance on real-world data from a 60km dataset.

ABSTRACT

Autonomous driving has attracted lots of attention in recent years. An accurate vehicle dynamics is important for autonomous driving techniques, e.g. trajectory prediction, motion planning, and control of trajectory tracking. Although previous works have made some results, the strong nonlinearity, precision, and interpretability of dynamics for autonomous vehicles are open problems worth being studied. In this paper, the approach based on the Koopman operator named deep direct Koopman (DDK) is proposed to identify the model of the autonomous vehicle and the identified model is a linear time-invariant (LTI) version, which is convenient for motion planning and controller design. In the approach, the Koopman eigenvalues and system matrix are considered as trainable tensors with the original states of the autonomous vehicle being concatenated to a part of the Koopman eigenfunctions so that a physically interpretable subsystem can be extracted from the identified latent dynamics. Subsequently, the process of the identification model is trained under the proposed method based on the dataset which consists of about 60km of data collected with a real electric SUV while the effectiveness of the identified model is validated. Meanwhile, a high-fidelity vehicle dynamics is identified in CarSim with DDK, and then, a linear model predictive control (MPC) called DDK-MPC integrating DDK is designed to validate the performance for the control of trajectory tracking. Simulation results illustrate that the model of the nonlinear vehicle dynamics can be identified effectively via the proposed method and that excellent tracking performance can be obtained with the identified model under DDK-MPC.

Motivation & Objective

• To address the challenge of modeling highly nonlinear, precise, and interpretable vehicle dynamics for autonomous driving.
• To develop a data-driven method that transforms complex nonlinear dynamics into a linear, time-invariant form suitable for control design.
• To enable physically interpretable subsystem extraction by embedding original vehicle states directly into the Koopman eigenfunctions.
• To validate the effectiveness of the identified model through trajectory tracking using a linear MPC controller (DDK-MPC).
• To demonstrate real-time, high-precision control performance on real-world vehicle data collected from diverse driving scenarios.

Proposed method

• DDK treats Koopman eigenvalues and system matrix B as trainable tensors, enabling end-to-end learning of a linear time-invariant (LTI) dynamics model.
• The original vehicle states (e.g., position, velocity, yaw rate) are directly concatenated into the Koopman eigenfunctions, ensuring physical interpretability of the latent dynamics.
• The state transition matrix in DDK is constrained to be diagonal, allowing extraction of an interpretable subsystem corresponding to the physical vehicle dynamics.
• A multi-step prediction loss function with 80-step horizon is used to train the model on a 60km real-world dataset collected from an electric SUV.
• A linear Model Predictive Control (MPC) framework, DDK-MPC, is designed using the identified LTI model for point-to-point (P2P) trajectory tracking.
• The method is validated in a high-fidelity CarSim environment and on real vehicle data, comparing against Deep EDMD and linear time-varying (LTV) models.

Experimental results

Research questions

• RQ1 Can a deep learning-based Koopman approach effectively identify a linear time-invariant (LTI) approximation of nonlinear vehicle dynamics while preserving physical interpretability?
• RQ2 How well can the DDK model predict vehicle trajectories over long horizons using real-world driving data?
• RQ3 Can the identified DDK model enable high-precision, real-time trajectory tracking when integrated with a linear MPC controller?
• RQ4 How does DDK compare to existing methods like Deep EDMD and LTV models in terms of prediction accuracy and control performance?
• RQ5 To what extent does incorporating original vehicle states into the Koopman eigenfunctions improve interpretability and tracking accuracy?

Key findings

• DDK achieved a mean absolute error of 0.09 m in P2P trajectory tracking with a 30|30 MPC horizon, significantly outperforming Deep EDMD (0.97 m) and LTV (2.12 m).
• The maximum absolute error for DDK was 0.37 m in position, compared to 3.82 m for Deep EDMD and 20.80 m for LTV, demonstrating robustness across scenarios.
• On real-world data, DDK predicted 10 seconds ahead with a root mean square error (RMSE) of 0.353 m for position, showing strong long-horizon generalization.
• The DDK-MPC controller maintained real-time performance with an average computation time of 8.0 ms per step, meeting real-time control requirements.
• The model's ability to extract an interpretable subsystem was validated by aligning the first m latent states with physical vehicle states such as yaw rate and lateral velocity.
• DDK showed superior performance in complex maneuvers such as U-turns and roundabouts, where prediction errors remained low despite high nonlinearity.