[论文解读] Playing hide and seek: tackling in-store picking operations while improving customer experience

Fábio Neves-Moreira, Pedro Amorim|arXiv (Cornell University)|Jan 5, 2023

Consumer Retail Behavior StudiesBusiness, Management and Accounting被引用 3

一句话总结

本文提出了动态店内拣货路径问题（diPRP），将店内在线订单拣选建模为马尔可夫决策过程（MDP），并采用混合强化学习与数学规划的方法求解。该方法学习到的策略相比最短路径策略可将顾客相遇次数减少50%以上，同时在全渠道零售环境中平衡了运营效率与店内顾客体验。

ABSTRACT

The evolution of the retail business presents new challenges and raises pivotal questions on how to reinvent stores and supply chains to meet the growing demand of the online channel. One of the recent measures adopted by omnichannel retailers is to address the growth of online sales using in-store picking, which allows serving online orders using existing assets. However, it comes with the downside of harming the offline customer experience. To achieve picking policies adapted to the dynamic customer flows of a retail store, we formalize a new problem called Dynamic In-store Picker Routing Problem (diPRP). In this relevant problem - diPRP - a picker tries to pick online orders while minimizing customer encounters. We model the problem as a Markov Decision Process (MDP) and solve it using a hybrid solution approach comprising mathematical programming and reinforcement learning components. Computational experiments on synthetic instances suggest that the algorithm converges to efficient policies. Furthermore, we apply our approach in the context of a large European retailer to assess the results of the proposed policies regarding the number of orders picked and customers encountered. Our work suggests that retailers should be able to scale the in-store picking of online orders without jeopardizing the experience of offline customers. The policies learned using the proposed solution approach reduced the number of customer encounters by more than 50% when compared to policies solely focused on picking orders. Thus, to pursue omnichannel strategies that adequately trade-off operational efficiency and customer experience, retailers cannot rely on actual simplistic picking strategies, such as choosing the shortest possible route.

研究动机与目标

• 为应对全渠道零售中日益增长的店内在线订单拣选挑战，同时最小化对店内顾客的干扰。
• 形式化提出一种新的动态路径规划问题——diPRP，以考虑实时顾客流动与拣货效率。
• 开发一种可扩展、低计算量的策略，平衡拣货性能与顾客体验。
• 通过合成实例与一家大型欧洲零售商的真实案例验证该方法。
• 证明简单最短路径策略显著增加顾客相遇次数，从而损害店内顾客体验。

提出的方法

• 将diPRP形式化为马尔可夫决策过程（MDP），其中状态由拣货员位置与目标位置定义。
• 整合数学规划以生成初始路径策略，并在动态门店环境中模拟顾客流动。
• 采用Q学习（QL）并结合状态-动作值函数近似，训练拣货员智能体以避开高密度节点。
• 使用基于弧段的距离与人群感知特征作为状态变量，引导策略学习。
• 结合基于仿真的训练与强化学习，学习最小化顾客相遇次数的策略。
• 采用混合求解框架：仿真用于环境动态建模，强化学习用于策略学习，数学规划用于路径初始化。

实验结果

研究问题

• RQ1 如何优化店内拣货路径规划，以在保持高拣货效率的同时减少顾客相遇？
• RQ2 在具有不可预测顾客流动的动态实时零售环境中，强化学习在多大程度上能学习到有效的策略？
• RQ3 与基于最短路径的策略相比，人群感知路径规划在顾客干扰与订单吞吐量方面表现如何？
• RQ4 结合仿真、数学规划与强化学习的混合方法，能否实现可扩展、低计算量的店内拣货决策？
• RQ5 在一家大型欧洲零售商的真实应用中，此类策略能提供哪些管理启示？

主要发现

• 所提出的强化学习策略在合成实例与真实案例中，相比基于最短路径的策略，将顾客相遇次数减少了50%以上。
• 基于弧段距离与人群感知特征训练的QL策略，即使路径更长，也能学会避开高流量节点，显著减少顾客互动。
• 最短路径（SP）策略虽然在距离上更高效，但访问了最多拥挤的节点，导致顾客相遇次数接近翻倍。
• CN策略（优先完全避免顾客）选择了最长路径，表明最优权衡需在效率与避让之间取得平衡。
• 混合求解方法在合理数量的训练回合内实现了有效的策略学习，计算开销低，推理速度快。
• 真实世界应用验证表明，人群感知路径规划能显著改善店内顾客体验，且不牺牲拣货性能。