[论文解读] LORA: Learning to optimize for resource allocation in wireless networks with few training samples

Yifei Shen, Yuanming Shi+2 more|arXiv (Cornell University)|Dec 18, 2018

Advanced Wireless Network Optimization被引用 3

一句话总结

本文提出LORM，一种用于无线网络资源分配的样本高效学习优化框架，利用模仿学习对分支定界树进行剪枝，显著减少了训练数据需求。此外，本文还引入LORM-TL，一种基于自模仿的迁移学习方法，可在极少标注数据下实现对新网络条件的快速适应，达到近似最优性能，并在速度上显著优于传统方法。

ABSTRACT

Effective resource management plays a pivotal role in wireless networks, which, unfortunately, results in challenging mixed-integer nonlinear programming (MINLP) problems in most cases. Machine learning-based methods have recently emerged as a disruptive way to obtain near-optimal performance for MINLPs with affordable computational complexity. There have been some attempts in applying such methods to resource management in wireless networks, but these attempts require huge amounts of training samples and lack the capability to handle constrained problems. Furthermore, they suffer from severe performance deterioration when the network parameters change, which commonly happens and is referred to as the task mismatch problem. In this paper, to reduce the sample complexity and address the feasibility issue, we propose a framework of Learning to Optimize for Resource Management (LORM). Instead of the end-to-end learning approach adopted in previous studies, LORM learns the optimal pruning policy in the branch-and-bound algorithm for MINLPs via a sample-efficient method, namely, imitation learning. To further address the task mismatch problem, we develop a transfer learning method via self-imitation in LORM, named LORM-TL, which can quickly adapt a pre-trained machine learning model to the new task with only a few additional unlabeled training samples. Numerical simulations will demonstrate that LORM outperforms specialized state-of-the-art algorithms and achieves near-optimal performance, while achieving significant speedup compared with the branch-and-bound algorithm. Moreover, LORM-TL, by relying on a few unlabeled samples, achieves comparable performance with the model trained from scratch with sufficient labeled samples.

研究动机与目标

• 解决现有基于机器学习的无线网络资源分配方法中存在的高样本复杂度和可行性问题。
• 通过实现对新网络条件的快速适应，克服模型在参数变化时性能下降的任务不匹配问题。
• 在保持近似最优性能的同时，降低计算复杂度和训练数据需求，适用于混合整数非线性规划（MINLP）问题。
• 开发一种迁移学习机制，利用未标注样本实现动态无线环境中模型的快速适应。

提出的方法

• LORM采用模仿学习训练策略网络，学习在MINLP问题的分支定界算法中做出最优剪枝决策。
• 该框架在少量标注的最优解上进行训练，实现高效策略学习，无需端到端训练。
• LORM-TL引入自模仿机制，利用来自新任务分布的少量未标注样本对预训练模型进行微调。
• LORM-TL中的自模仿使模型能够从自身预测生成伪标签，从而提升适应效率。
• 该方法与分支定界框架集成，加速收敛过程，同时保持解的可行性。
• 该方法专为无线资源分配中常见的约束型MINLP问题设计，确保实际可行性。

实验结果

研究问题

• RQ1 模仿学习能否降低无线资源分配机器学习模型训练的样本复杂度？
• RQ2 基于自模仿的迁移学习方法能否在极少标注数据下实现对新网络条件的快速适应？
• RQ3 所提出的框架是否在显著优于传统分支定界算法速度的同时，实现近似最优性能？
• RQ4 当仅有少量标注样本时，LORM-TL的性能与从零开始训练的模型相比如何？

主要发现

• LORM在无线资源分配中实现近似最优性能，显著优于专用的最先进算法。
• 与标准分支定界算法相比，LORM降低了计算复杂度并加快了收敛速度。
• LORM-TL在仅使用少量额外未标注样本的情况下，性能可与从零开始训练的模型相媲美。
• 该框架通过实现对新网络参数配置的快速适应，有效解决了任务不匹配问题。
• 模仿学习的使用实现了样本高效的训练，减少了对大规模标注数据集的依赖。