[论文解读] VEER: Disagreement-Free Multi-objective Configuration.

Kewen Peng, Christian Kaltenecker+3 more|arXiv (Cornell University)|Jun 4, 2021

Advanced Multi-Objective Optimization Algorithms参考文献 30被引用 6

一句话总结

VEER 通过在 FLASH 优化器之上应用维度约简启发式方法，对 N 个目标空间进行一维近似，解决了多目标软件配置中的模型不一致问题，实现了对大规模配置空间的高效、无冲突探索，性能最高提升 1,000 倍，且解决方案质量与原始优化器相当。

ABSTRACT

Software comes with many configuration options, satisfying varying needs from users. Exploring those options for non-functional requirements can be tedious, time-consuming, and even error-prone (if done manually). Worse, many software systems can be tuned to multiple objectives (e.g., faster response time, fewer memory requirements, decreased network traffic, decreased energy consumption, etc.). Learning how to adjust the system among these multiple objectives is complicated due to the trade-off among objectives; i.e., things that seem useful to achieve one objective could be detrimental to another objective. Consequentially, the optimizer built for one objective may have different (or even opposite) insights on how to locate good solutions from the optimizer built from another objective. In this paper, we define this scenario as the model disagreement problem. One possible solution to this problem is to find a one-dimensional approximation to the N-objective space. In this way, the case is converted to a single-objective optimization, which is naturally confusion-free. This paper demonstrates VEER, a tool that builds such an approximation by combining our dimensionality-reduction heuristic on top of one of the state-of-the-art optimizers, FLASH. VEER can explore very large configuration spaces by evaluating just a small fraction of the total number of configurations (e.g., a space of 81,000 configurations can be explored by 70 samples). The experimental result in this paper demonstrates the feasibility of our approach in terms of the on-par quality of the solution set generated by the optimizer and the resolved model disagreement within the optimizer. Moreover, we demonstrate that VEER has an improved computational complexity compared to the original optimizer (up to 1,000 times faster while maintaining on-par performance).

研究动机与目标

• 解决多目标软件配置中的模型不一致问题，即在不同目标上训练的优化器提供相互冲突的见解。
• 在不全面评估所有可能配置的情况下，实现对大规模配置空间的高效探索。
• 在保持与最先进优化器相当的解决方案质量的同时，降低计算复杂度。
• 为响应时间、内存使用量和能耗等非功能性需求提供统一、无冲突的系统调优方法。

提出的方法

• 应用一种新颖的维度约简启发式方法，将 N 维目标空间映射为一维近似，将多目标优化简化为单目标问题。
• 将维度约简技术与 FLASH 优化器集成，以在减少搜索空间的同时保持高质量的解集。
• 使用少量具有代表性的配置样本（例如 70 个，共 81,000 个）来训练和评估一维近似。
• 利用一维近似引导优化器寻找帕累托最优解，避免多个目标带来的冲突指导。
• 设计启发式方法以在维度约简过程中保留目标之间的权衡关系。
• 通过最小化优化过程中所需的配置评估次数，确保方法具有高效的可扩展性。

实验结果

研究问题

• RQ1 对 N 个目标配置空间的一维近似是否能有效解决在不同目标上训练的优化器之间的模型不一致问题？
• RQ2 VEER 在保持解决方案质量的前提下，能在多大程度上减少探索大规模配置空间所需的配置评估次数？
• RQ3 VEER 的计算复杂度与原始 FLASH 优化器相比，在速度和可扩展性方面如何？
• RQ4 维度约简启发式方法是否保留了识别高质量、帕累托最优配置所必需的权衡结构？

主要发现

• VEER 的解决方案集质量与原始 FLASH 优化器相当，表明一维近似成功保留了关键的权衡信息。
• VEER 仅需 70 个样本（在 81,000 个配置中）即可完成配置评估，实现了高效的探索。
• VEER 的计算复杂度相比原始 FLASH 优化器最高提升 1,000 倍，同时保持了相当的解决方案质量。
• 维度约简启发式方法通过统一多个目标上的冲突优化器见解，成功解决了模型不一致问题。
• 该方法可在不牺牲解决方案保真度或引入新错误源的前提下，实现大规模配置空间的可扩展优化。