[论文解读] When Remembering and Planning are Worth it: Navigating under Change

Omid Madani, J. Brian Burns+2 more|arXiv (Cornell University)|Feb 17, 2026

Memory and Neural Mechanisms被引用 0

一句话总结

论文比较记忆为基础的地图策略与更简单策略在变化、不确定的网格世界中的导航表现，表明记忆驱动的规划在中等变化下可以显著提高效率。

ABSTRACT

We explore how different types and uses of memory can aid spatial navigation in changing uncertain environments. In the simple foraging task we study, every day, our agent has to find its way from its home, through barriers, to food. Moreover, the world is non-stationary: from day to day, the location of the barriers and food may change, and the agent's sensing such as its location information is uncertain and very limited. Any model construction, such as a map, and use, such as planning, needs to be robust against these challenges, and if any learning is to be useful, it needs to be adequately fast. We look at a range of strategies, from simple to sophisticated, with various uses of memory and learning. We find that an architecture that can incorporate multiple strategies is required to handle (sub)tasks of a different nature, in particular for exploration and search, when food location is not known, and for planning a good path to a remembered (likely) food location. An agent that utilizes non-stationary probability learning techniques to keep updating its (episodic) memories and that uses those memories to build maps and plan on the fly (imperfect maps, i.e. noisy and limited to the agent's experience) can be increasingly and substantially more efficient than the simpler (minimal-memory) agents, as the task difficulties such as distance to goal are raised, as long as the uncertainty, from localization and change, is not too large.

研究动机与目标

• 研究不同记忆使用与规划策略如何影响非静态环境中的空间导航。
• 评估基于记忆的地图构建与规划在不同任务难度下是否相对于更简单策略具有优势。
• 确定如何设计一个能灵活结合多种策略以应对探索与规划任务的代理体系结构。

提出的方法

• 在随机格子世界中评估从随机到记忆驱动规划的多种导航策略。
• 引入一个多策略代理，能够在逐步增加的时间预算下在策略之间切换。
• 实现若干记忆驱动策略（LeastVisited, Path-Memory, ProbMap）并与 Greedy 与 Random 基线进行比较。
• 在 ProbMap 中，维护情节记忆并学习分布以构建用于规划的概率地图。
• 允许部分观测与运动噪声以测试记忆与规划的鲁棒性。

(a)

实验结果

研究问题

• RQ1 在变化环境中，在哪些条件下基于记忆的地图策略会优于更简单的策略？
• RQ2 当食物位置为未知或已知时，代理应如何组合多种策略以处理探索与规划？

主要发现

• 更新并使用情节记忆来构建地图的记忆驱动策略在任务难度增加且不确定性中等的条件下可显著减少达到食物所需的步数。
• 要实现鲁棒性，需将多种策略结合以处理搜索与规划子任务；单纯的规划或单纯的贪婪方法可能表现不佳。
• 使用非平稳概率学习来更新记忆并使用不完美地图进行规划的做法在某些条件下比简单代理带来显著提升（步数减少最多可超过20倍）。
• 一个带有渐进时间预算和轮换切换策略的复合代理能够有效利用记忆更新来在变化下进行规划。

(b)