[论文解读] Data Cache Prefetching with Perceptron Learning

Haoyuan Wang, Zhiwei Luo|arXiv (Cornell University)|Dec 4, 2017

Parallel Computing and Optimization Techniques参考文献 23被引用 11

一句话总结

本文提出一种两级数据缓存预取方案，结合基于表格的预取（如步长或马尔可夫）与硬件可实现的感知器学习器，以减少不必要的预取。感知器通过利用局部和全局的时间与空间历史动态分类内存访问模式，拒绝有害预取，将内存请求减少60.64%–83.84%，对IPC和缓存命中率影响极小。

ABSTRACT

Cache prefetcher greatly eliminates compulsory cache misses, by fetching data from slower memory to faster cache before it is actually required by processors. Sophisticated prefetchers predict next use cache line by repeating program's historical spatial and temporal memory access pattern. However, they are error prone and the mis-predictions lead to cache pollution and exert extra pressure on memory subsystem. In this paper, a novel scheme of data cache prefetching with perceptron learning is proposed. The key idea is a two-level prefetching mechanism. A primary decision is made by utilizing previous table-based prefetching mechanism, e.g. stride prefetching or Markov prefetching, and then, a neural network, perceptron is taken to detect and trace program memory access patterns, to help reject those unnecessary prefetching decisions. The perceptron can learn from both local and global history in time and space, and can be easily implemented by hardware. This mechanism boost execution performance by ideally mitigating cache pollution and eliminating redundant memory request issued by prefetcher. Detailed evaluation and analysis were conducted based on SPEC CPU 2006 benchmarks. The simulation results show that generally the proposed scheme yields a geometric mean of 60.64%-83.84% decrease in prefetching memory requests without loss in instruction per cycle(IPC)(floating between -2.22% and 2.55%) and cache hit rate(floating between -1.67% and 2.46%). Though it is possible that perceptron may refuse useful blocks and thus cause minor raise in cache miss rate, lower memory request count can decrease average memory access latency, which compensate for the loss, and in the meantime, enhance overall performance in multi-programmed workloads.

研究动机与目标

• 解决高性能处理器中因预取不准确导致的缓存污染问题。
• 在不降低IPC或缓存命中率的前提下，减少预取器产生的冗余内存请求。
• 利用感知器学习，通过局部和全局历史动态适应程序的内存访问模式。
• 开发一种硬件高效、与周期无关的预取机制，可与现有基于表格的预取器集成。
• 在单任务和多任务工作负载下，评估该方案在SPEC CPU 2006基准测试中的有效性。

提出的方法

• 使用一级基于表格的预取器（步长或马尔可夫）利用全局历史缓冲区（GHB）生成初始预取建议。
• 将GHB状态和历史访问模式输入单神经元感知器，用于最终决策。
• 感知器通过计算输入加权和的符号（包括局部和全局时空历史）来决定是否接受或拒绝预取建议。
• 通过历史内存访问模式在线训练感知器，实现对工作负载变化的动态适应。
• 将两级系统集成到基于SPEC CPU 2006基准测试的微架构仿真框架中。
• 利用缓存命中率和GHB更新的反馈，建立预取活动与内存访问行为之间的动态平衡模型。

实验结果

研究问题

• RQ1 基于感知器的二级过滤器能否有效减少来自基于表格预取器的冗余预取请求？
• RQ2 感知器学习在多大程度上缓解了缓存污染，同时保持或提升IPC和缓存命中率？
• RQ3 局部和全局时间与空间历史的整合如何提升预取拒绝决策的准确性？
• RQ4 减少的预取流量对多核环境下的内存子系统压力有何影响？
• RQ5 GHB更新、预取器建议与缓存命中率之间的反馈回路如何影响系统稳定性和性能？

主要发现

• 所提出的两级预取器相较于步长预取，将预取内存请求减少了60.64%的几何平均值；相较于马尔可夫预取，减少了83.84%。
• 该方案将IPC保持在基线的±2.55%以内，缓存命中率保持在±2.46%以内，表明性能退化极小。
• 尽管缓存命中率平均略有下降（步长为0.03%，马尔可夫为0.15%），但内存流量减少导致平均内存访问延迟降低，整体性能提升。
• 感知器实现了高拒绝率——拒绝了大量冗余预取建议，从而有效缓解了缓存污染和带宽压力。
• 观察到GHB更新、预取器建议与缓存命中率之间存在动态平衡，系统稳定在新的、更高效的运行点。
• 感知器能够从局部和全局内存访问模式中学习，从而实现对不规则访问模式的有效、硬件友好的自适应。