[论文解读] Hardware-Aware Machine Learning: Modeling and Optimization

Diana Marculescu, Dimitrios Stamoulis+1 more|arXiv (Cornell University)|Sep 14, 2018

Parallel Computing and Optimization Techniques被引用 6

一句话总结

本文全面综述了硬件感知机器学习，重点聚焦于深度神经网络（DNNs）的预测建模与硬件效率优化。提出了一类硬件感知超参数优化技术，可联合最小化延迟与能耗，同时保持精度，从而实现从移动设备到数据中心等各类平台的DNN与加速器协同设计。

ABSTRACT

Recent breakthroughs in Deep Learning (DL) applications have made DL models a key component in almost every modern computing system. The increased popularity of DL applications deployed on a wide-spectrum of platforms have resulted in a plethora of design challenges related to the constraints introduced by the hardware itself. What is the latency or energy cost for an inference made by a Deep Neural Network (DNN)? Is it possible to predict this latency or energy consumption before a model is trained? If yes, how can machine learners take advantage of these models to design the hardware-optimal DNN for deployment? From lengthening battery life of mobile devices to reducing the runtime requirements of DL models executing in the cloud, the answers to these questions have drawn significant attention. One cannot optimize what isn't properly modeled. Therefore, it is important to understand the hardware efficiency of DL models during serving for making an inference, before even training the model. This key observation has motivated the use of predictive models to capture the hardware performance or energy efficiency of DL applications. Furthermore, DL practitioners are challenged with the task of designing the DNN model, i.e., of tuning the hyper-parameters of the DNN architecture, while optimizing for both accuracy of the DL model and its hardware efficiency. Therefore, state-of-the-art methodologies have proposed hardware-aware hyper-parameter optimization techniques. In this paper, we provide a comprehensive assessment of state-of-the-art work and selected results on the hardware-aware modeling and optimization for DL applications. We also highlight several open questions that are poised to give rise to novel hardware-aware designs in the next few years, as DL applications continue to significantly impact associated hardware systems and platforms.

研究动机与目标

• 应对在从移动设备到数据中心系统等多样化硬件平台上部署能耗与延迟受限的DNN所面临的日益严峻挑战。
• 克服传统DNN设计的局限性，即仅关注精度而忽略超参数调优过程中的硬件效率。
• 在训练前实现对DNN运行时和能耗的早期预测，利用预测模型指导高效模型与硬件协同设计。
• 通过将硬件性能作为首要约束条件，推动自动化神经架构搜索（NAS）技术的最新发展。
• 识别硬件感知DNN设计中跨平台建模与多目标优化的开放研究方向。

提出的方法

• 综述并评估现有用于DNN硬件性能的预测模型，包括运行时与能耗估算框架，如Eyeriss、Paleo和NeuralPower。
• 提出基于贝叶斯优化与基于NAS的方法进行硬件感知超参数优化，将硬件指标（如延迟、能耗）作为优化目标。
• 提出将预测模型扩展至处理非线性DNN架构，特别是NAS中使用的具有复杂非顺序操作的架构。
• 开发可跨多种硬件平台泛化的跨平台预测模型，涵盖GPU、FPGA及可重构加速器。
• 将硬件感知模型集成到协同设计流程中，联合优化DNN架构与硬件参数（如处理单元数量）。
• 构建多目标优化问题，以在不同平台上平衡精度、延迟、能耗与硬件利用率。

实验结果

研究问题

• RQ1 能否基于架构超参数，在训练前构建出准确估算DNN运行时与能耗的预测模型？
• RQ2 如何在不牺牲精度的前提下，有效将硬件效率整合到神经架构搜索（NAS）与超参数优化中？
• RQ3 将硬件感知模型扩展至非线性与复杂DNN架构（如NAS中使用的架构）时面临哪些关键挑战？
• RQ4 如何使预测模型在包括移动设备、边缘设备与服务器系统在内的多样化硬件平台上具备可移植性？
• RQ5 联合优化DNN架构与硬件参数（如处理单元数量）在提升性能与能效方面有何优势？

主要发现

• 在实现相同精度的DNN之间，能耗差异可达40倍之多，凸显了硬件感知设计的必要性。
• 硬件感知的NAS与超参数优化可识别出更接近精度、延迟与能效帕累托前沿的模型。
• 现有预测模型大多局限于线性或流水线式DNN，需进一步扩展以支持非线性、基于单元的NAS架构。
• 跨平台的硬件感知模型对于实现平台间知识迁移及支持可移植、高效的DNN部署至关重要。
• 在预测模型引导下，对DNN与硬件加速器（如可重构FPGA）进行联合优化，可在能耗与性能方面实现显著提升。
• 将多目标优化扩展至精度与延迟之外的维度（如硬件利用率）可带来更高效、更具可扩展性的部署方案。