[论文解读] Remember Intentions: Retrospective-Memory-based Trajectory Prediction

Chenxin Xu, Weibo Mao+2 more|arXiv (Cornell University)|Mar 22, 2022

Anomaly Detection Techniques and Applications被引用 9

一句话总结

该论文提出 MemoNet，一种受人类回溯性记忆启发的基于实例的轨迹预测模型，通过记忆库存储过去-未来轨迹对，并利用可训练的记忆地址机制检索语义上相似的过往实例以进行意图预测。该方法在 SDD、ETH-UCY 和 NBA 数据集上分别将 FDE 改进 20.3%、10.2% 和 28.3%，同时在推理过程中实现可解释的、基于实例的检索。

ABSTRACT

To realize trajectory prediction, most previous methods adopt the parameter-based approach, which encodes all the seen past-future instance pairs into model parameters. However, in this way, the model parameters come from all seen instances, which means a huge amount of irrelevant seen instances might also involve in predicting the current situation, disturbing the performance. To provide a more explicit link between the current situation and the seen instances, we imitate the mechanism of retrospective memory in neuropsychology and propose MemoNet, an instance-based approach that predicts the movement intentions of agents by looking for similar scenarios in the training data. In MemoNet, we design a pair of memory banks to explicitly store representative instances in the training set, acting as prefrontal cortex in the neural system, and a trainable memory addresser to adaptively search a current situation with similar instances in the memory bank, acting like basal ganglia. During prediction, MemoNet recalls previous memory by using the memory addresser to index related instances in the memory bank. We further propose a two-step trajectory prediction system, where the first step is to leverage MemoNet to predict the destination and the second step is to fulfill the whole trajectory according to the predicted destinations. Experiments show that the proposed MemoNet improves the FDE by 20.3%/10.2%/28.3% from the previous best method on SDD/ETH-UCY/NBA datasets. Experiments also show that our MemoNet has the ability to trace back to specific instances during prediction, promoting more interpretability.

研究动机与目标

• 为解决参数化轨迹预测模型中因无关训练实例导致的可解释性不足与性能下降问题。
• 在推理过程中实现对相似过往场景的显式、基于实例的检索，以提升预测准确率与可解释性。
• 构建一种记忆增强型架构，模拟人类的回溯性记忆，将当前情境与存储的过往经验相联系。

提出的方法

• 提出一种基于联合重建的特征学习架构，利用两个编码器初始化兼容的过去轨迹记忆库与意图记忆库。
• 引入一种无需训练、排列不变的记忆过滤器，从记忆库中去除冗余实例，提升效率与鲁棒性。
• 设计一种可学习的注意力机制记忆地址器，计算相似度得分并检索最相关的过往实例用于当前预测。
• 引入意图聚类机制，以增强对低频行为的预测多样性，并提升对高频行为的鲁棒性。
• 构建两步式轨迹预测系统：首先通过 MemoNet 预测目的地（意图），然后从预测的意图生成完整轨迹。
• 使用伪标签生成来监督记忆地址器的训练，确保其学习到检索最相似实例的能力。

Figure 1 : MemoNet mimics retrospective memory process. We use the memory bank to explicitly store representative instances, acting like prefrontal cortex; and the memory addresser to search similar memory instances with current situation, acting like basal ganglia.

实验结果

研究问题

• RQ1 基于实例的方法是否能通过显式记忆检索在轨迹预测中超越参数化模型？
• RQ2 神经科学中的回溯性记忆机制在深度学习中如何被有效模拟以用于轨迹预测？
• RQ3 记忆过滤与聚类在多大程度上提升了轨迹预测的多样性与鲁棒性？
• RQ4 模型是否能通过在推理过程中追溯到特定过往实例，提供可解释的预测？
• RQ5 粗粒度意图锚点的数量与记忆过滤阈值对模型性能有何影响？

主要发现

• 与先前 SOTA 方法相比，MemoNet 在 SDD 数据集上实现了 FDE 相对 20.3% 的提升。
• 在 ETH-UCY 数据集上，该模型相较之前最佳方法将 FDE 降低了 10.2%。
• NBA 数据集显示 FDE 相对提升了 28.3%，表明模型在跨领域场景中具有强大的泛化能力。
• 消融实验表明，记忆库、过滤、地址器与聚类所有组件均对性能提升有贡献。
• 模型在单张 RTX 3090 GPU 上实现 55.5 FPS 的实时推理速度，适用于实际部署。
• 可视化结果表明，模型能够追溯到特定且语义相关的过往实例，显著增强可解释性。

Figure 2 : Inference phase of MemoNet. The red agent is to-be-predicted and the blue/orange agents are neighbours. According to the past feature obtained by the social encoder, we address related memory instances in the past memory bank through an attention network, producing similarity scores. The