[Paper Review] AttnMove: History Enhanced Trajectory Recovery via Attentional Network

Xia Tong, Yunhan Qi+5 more|arXiv (Cornell University)|Jan 3, 2021

Human Mobility and Location-Based Analysis26 references4 citations

TL;DR

AttnMove proposes a transformer-based attention network to recover fine-grained trajectories from sparse mobility data by leveraging intra-trajectory attention for local mobility patterns and inter-trajectory attention to exploit long-term historical periodicity. Evaluated on real-world datasets, it outperforms state-of-the-art methods in trajectory recovery accuracy by effectively modeling sparsity and periodic behavior.

ABSTRACT

A considerable amount of mobility data has been accumulated due to the proliferation of location-based service. Nevertheless, compared with mobility data from transportation systems like the GPS module in taxis, this kind of data is commonly sparse in terms of individual trajectories in the sense that users do not access mobile services and contribute their data all the time. Consequently, the sparsity inevitably weakens the practical value of the data even it has a high user penetration rate. To solve this problem, we propose a novel attentional neural network-based model, named AttnMove, to densify individual trajectories by recovering unobserved locations at a fine-grained spatial-temporal resolution. To tackle the challenges posed by sparsity, we design various intra- and inter- trajectory attention mechanisms to better model the mobility regularity of users and fully exploit the periodical pattern from long-term history. We evaluate our model on two real-world datasets, and extensive results demonstrate the performance gain compared with the state-of-the-art methods. This also shows that, by providing high-quality mobility data, our model can benefit a variety of mobility-oriented down-stream applications.

Motivation & Objective

• To address the challenge of sparse and irregularly sampled mobility trajectories in location-based services.
• To improve trajectory recovery by modeling both intra-trajectory mobility regularity and inter-trajectory periodic patterns from long-term history.
• To overcome limitations of RNNs and sequence-based models in handling sparse, non-sequential trajectory data.
• To design a unified attention mechanism that captures spatial-temporal dependencies and historical periodicity simultaneously.
• To enable high-precision, fine-grained trajectory imputation for downstream mobility applications.

Proposed method

• The model employs an intra-trajectory attention mechanism to model mobility regularity within a single, sparse target trajectory by attending to all observed locations equally, regardless of temporal order.
• It introduces an inter-trajectory attention mechanism to align and fuse patterns from multiple historical trajectories, extracting periodic behaviors such as home-workplace cycles.
• The framework integrates long-term historical data by representing it as a sequence of daily mobility patterns, enabling the model to learn recurring spatio-temporal behaviors.
• A multi-head attention architecture is used to capture complex dependencies across time and space, replacing RNNs to better handle long-range and sparse sequences.
• The model is trained end-to-end using a reconstruction loss to minimize the difference between predicted and ground-truth locations at unobserved timestamps.
• Hyperparameter tuning is performed on embedding size, number of attention heads, and network depth to optimize performance.

Experimental results

Research questions

• RQ1 Can an attention-based model effectively recover fine-grained trajectories from highly sparse mobility data where traditional RNNs fail?
• RQ2 How can long-term historical mobility patterns be effectively distilled to improve trajectory recovery in sparse scenarios?
• RQ3 To what extent does inter-trajectory attention improve recovery by leveraging periodic behaviors across multiple users?
• RQ4 Can intra-trajectory attention outperform sequential modeling in handling non-ordered, sparse trajectory data?
• RQ5 How does the integration of intra- and inter-trajectory attention mechanisms compare to existing state-of-the-art trajectory recovery methods?

Key findings

• AttnMove significantly outperforms existing state-of-the-art methods on two real-world mobility datasets in terms of trajectory recovery accuracy.
• The model achieves substantial performance gains in sparse scenarios where traditional RNN-based or smoothing-filter methods degrade sharply.
• The ablation study confirms that both intra-trajectory and inter-trajectory attention mechanisms contribute meaningfully to the overall performance.
• The use of long-term historical data through inter-trajectory attention enables the model to capture periodic mobility patterns such as daily commuting routines.
• The model demonstrates robustness to high sparsity, effectively reducing uncertainty in missing location prediction by leveraging attention over all observed points.
• Hyperparameter tuning shows optimal performance with 8 attention heads and a specific embedding size, indicating sensitivity to architectural design.