[Paper Review] MSTFormer: Motion Inspired Spatial-temporal Transformer with Dynamic-aware Attention for long-term Vessel Trajectory Prediction

Huimin Qiang, Zhiyuan Guo+2 more|arXiv (Cornell University)|Mar 21, 2023

Maritime Navigation and Safety4 citations

TL;DR

MSTFormer proposes a motion-inspired spatial-temporal Transformer for long-term vessel trajectory prediction by integrating vessel dynamics into a self-attention mechanism and a knowledge-inspired loss function. It achieves state-of-the-art performance, especially on corner trajectories, with a 1.3781 DIS on challenging corner data, outperforming Transformers and LSTM-based models by over 200% in error reduction.

ABSTRACT

Incorporating the dynamics knowledge into the model is critical for achieving accurate trajectory prediction while considering the spatial and temporal characteristics of the vessel. However, existing methods rarely consider the underlying dynamics knowledge and directly use machine learning algorithms to predict the trajectories. Intuitively, the vessel's motions are following the laws of dynamics, e.g., the speed of a vessel decreases when turning a corner. Yet, it is challenging to combine dynamic knowledge and neural networks due to their inherent heterogeneity. Against this background, we propose MSTFormer, a motion inspired vessel trajectory prediction method based on Transformer. The contribution of this work is threefold. First, we design a data augmentation method to describe the spatial features and motion features of the trajectory. Second, we propose a Multi-headed Dynamic-aware Self-attention mechanism to focus on trajectory points with frequent motion transformations. Finally, we construct a knowledge-inspired loss function to further boost the performance of the model. Experimental results on real-world datasets show that our strategy not only effectively improves long-term predictive capability but also outperforms backbones on cornering data.The ablation analysis further confirms the efficacy of the proposed method. To the best of our knowledge, MSTFormer is the first neural network model for trajectory prediction fused with vessel motion dynamics, providing a worthwhile direction for future research.The source code is available at https://github.com/simple316/MSTFormer.

Motivation & Objective

• To address the lack of dynamic knowledge integration in deep learning-based vessel trajectory prediction models.
• To enhance long-term trajectory prediction accuracy by modeling spatial-temporal dependencies and motion state transitions.
• To improve performance on challenging trajectory segments, particularly sharp turns (cornering data), where existing models underperform.
• To bridge the gap between neural network learning and physical dynamics by embedding motion laws into attention and loss functions.
• To provide a novel, interpretable framework for trajectory prediction that fuses domain-specific dynamics with deep learning.

Proposed method

• Proposes an Augmented Trajectory Matrix (ATM) to encode spatial features and motion dynamics (e.g., speed and turning rate changes) from AIS data.
• Introduces a Multi-head Dynamic-aware Self-attention mechanism that emphasizes trajectory points with frequent motion transformations, such as turns.
• Designs a knowledge-inspired loss function based on geodesic correction to guide the model toward physically plausible predictions.
• Uses CNN to extract spatial and dynamic patterns from the ATM before feeding into the Transformer encoder.
• Combines dynamic-aware attention with the knowledge-inspired loss to jointly optimize for both attention focus and dynamic consistency.
• Employs a hybrid training strategy where the model learns from both raw trajectory data and physics-informed correction signals.

Experimental results

Research questions

• RQ1 Can integrating vessel motion dynamics into a Transformer architecture improve long-term trajectory prediction accuracy?
• RQ2 How does dynamic-aware attention enhance the model’s ability to focus on critical motion transitions like turning?
• RQ3 To what extent does a knowledge-inspired loss function improve prediction fidelity and generalization on complex trajectories?
• RQ4 Does the fusion of motion dynamics with deep learning outperform standard deep learning backbones on cornering and long-term predictions?
• RQ5 Why is the joint use of dynamic-aware attention and knowledge-inspired loss essential for optimal performance?

Key findings

• On the corner dataset, MSTFormer achieved a DIS of 1.3781, representing a 213.5% increase in error compared to LSTMED-attention, which had a DIS of 4.9627.
• MSTFormer reduced MSE to 2.5058×10⁻⁴ on corner data, significantly outperforming the Transformer (32.7519×10⁻⁴) and LSTMED-attention (24.6290×10⁻⁴).
• The ablation study confirmed that the combination of dynamic-aware attention and knowledge-inspired loss reduced DIS by 16.2% compared to the baseline Transformer.
• Adding CNN to extract features from the ATM reduced DIS by 8.04% and MSE by 12.7% compared to the standard Transformer.
• The model’s performance dropped when only dynamic-aware attention or only the knowledge loss was used, indicating that both components are necessary for optimal results.
• The knowledge-inspired loss function improved DIS from 2.2057 to 2.2650 when used alone, but the full model with all components achieved the lowest error at 2.2650, confirming synergistic effects.