[Paper Review] SVG-Net: An SVG-based Trajectory Prediction Model

Mohammadhossein Bahari, Vahid Zehtab+4 more|arXiv (Cornell University)|Oct 7, 2021

Autonomous Vehicle Technology and Safety23 references4 citations

TL;DR

SVG-Net proposes using Scalable Vector Graphics (SVG) as a standardized, structure-preserving input format for vehicle trajectory prediction, leveraging a Transformer-based encoder-decoder network to model scene context and agent interactions. Evaluated on the Argoverse dataset, SVG-Net achieves state-of-the-art performance with an FDE of 3.25 and ADE of 1.46, demonstrating the effectiveness of SVG as a generalizable, reusable representation for trajectory prediction.

ABSTRACT

Anticipating motions of vehicles in a scene is an essential problem for safe autonomous driving systems. To this end, the comprehension of the scene's infrastructure is often the main clue for predicting future trajectories. Most of the proposed approaches represent the scene with a rasterized format and some of the more recent approaches leverage custom vectorized formats. In contrast, we propose representing the scene's information by employing Scalable Vector Graphics (SVG). SVG is a well-established format that matches the problem of trajectory prediction better than rasterized formats while being more general than arbitrary vectorized formats. SVG has the potential to provide the convenience and generality of raster-based solutions if coupled with a powerful tool such as CNNs, for which we introduce SVG-Net. SVG-Net is a Transformer-based Neural Network that can effectively capture the scene's information from SVG inputs. Thanks to the self-attention mechanism in its Transformers, SVG-Net can also adequately apprehend relations amongst the scene and the agents. We demonstrate SVG-Net's effectiveness by evaluating its performance on the publicly available Argoverse forecasting dataset. Finally, we illustrate how, by using SVG, one can benefit from datasets and advancements in other research fronts that also utilize the same input format. Our code is available at https://vita-epfl.github.io/SVGNet/.

Motivation & Objective

• To address the limitations of rasterized and non-standard vectorized scene representations in trajectory prediction.
• To propose SVG as a standardized, structure-preserving, and general-purpose format for scene context in autonomous driving.
• To develop a neural network architecture (SVG-Net) that effectively learns from SVG inputs using self-attention mechanisms.
• To enable knowledge transfer and reuse of pre-trained models across vision tasks by adopting a common input format like SVG.
• To demonstrate that SVG-based representation supports both high performance and generalization in trajectory prediction.

Proposed method

• Representing scene context using Scalable Vector Graphics (SVG), a standardized vector format that preserves structural information.
• Designing SVG-Net as a Transformer-based encoder-decoder architecture with separate encoders for scene and agent history.
• Using a scene encoder to embed SVG paths into a latent representation, capturing geometric and spatial context.
• Employing residual MLPs to encode agent motion history into latent embeddings.
• Applying self-attention in the decoder to model complex interactions between agents and scene elements.
• Fine-tuning the model on the Argoverse forecasting dataset after pre-training on the SVG-Icons8 dataset to assess transferability.

Experimental results

Research questions

• RQ1 Can a standardized vector format like SVG outperform rasterized or proprietary vector representations in trajectory prediction?
• RQ2 How well can a Transformer-based model learn to reason about scene context and agent interactions from SVG inputs?
• RQ3 To what extent can representations learned from SVG data be transferred to downstream trajectory prediction tasks?
• RQ4 Does using SVG as input enable better generalization and reuse of pre-trained models across vision tasks?
• RQ5 Can SVG-Net achieve competitive performance while maintaining interpretability through attention visualization?

Key findings

• SVG-Net achieves an FDE of 3.25 and ADE of 1.46 on the Argoverse validation set, demonstrating strong predictive performance.
• Pre-training on the SVG-Icons8 dataset yields a transferred model with FDE 3.42 and ADE 1.51, showing strong generalization capability.
• The model performs competitively on a downstream SVG image classification task (72.2% accuracy), comparable to ResNet-18 (73.0%), indicating transferability.
• Qualitative attention visualization confirms that the model correctly attends to relevant scene elements (e.g., lanes) and interacting agents.
• The ablation study confirms the importance of both the scene encoder and the interaction modeling in the decoder for optimal performance.
• The results suggest that SVG can serve as a viable, standardized input format that enables knowledge sharing across computer vision tasks.