[Paper Review] Transfer Learning for Mixed-Integer Resource Allocation Problems in Wireless Networks

Yifei Shen, Yuanming Shi+2 more|arXiv (Cornell University)|Nov 17, 2018

Sparse and Compressive Sensing Techniques15 references4 citations

TL;DR

This paper proposes a transfer learning method via self-imitation to address task mismatch in machine learning-based resource allocation for wireless networks, where pre-trained models degrade under changing network conditions. By leveraging unlabeled samples from the new environment and using the best solution found during exploration as pseudo-labels, the method fine-tunes the policy with minimal labeled data, achieving near-optimal performance and up to 15x faster training compared to training from scratch.

ABSTRACT

Effective resource allocation plays a pivotal role for performance optimization in wireless networks. Unfortunately, typical resource allocation problems are mixed-integer nonlinear programming (MINLP) problems, which are NP-hard. Machine learning based methods recently emerge as a disruptive way to obtain near-optimal performance for MINLP problems with affordable computational complexity. However, they suffer from severe performance deterioration when the network parameters change, which commonly happens in practice and can be characterized as the task mismatch issue. In this paper, we propose a transfer learning method via self-imitation, to address this issue for effective resource allocation in wireless networks. It is based on a general "learning to optimize" framework for solving MINLP problems. A unique advantage of the proposed method is that it can tackle the task mismatch issue with a few additional unlabeled training samples, which is especially important when transferring to large-size problems. Numerical experiments demonstrate that with much less training time, the proposed method achieves comparable performance with the model trained from scratch with sufficient amount of labeled samples. To the best of our knowledge, this is the first work that applies transfer learning for resource allocation in wireless networks.

Motivation & Objective

• To address the task mismatch problem in machine learning-based resource allocation for dynamic wireless networks, where model performance degrades when network parameters change.
• To reduce training time and labeling cost for retraining models in new network settings, especially for large-scale mixed-integer nonlinear programming (MINLP) problems.
• To enable efficient adaptation of pre-trained policies to new scenarios using only a few unlabeled samples and self-generated labels.
• To demonstrate that transfer learning via self-imitation can achieve performance comparable to models trained from scratch with full labeled datasets.

Proposed method

• A general 'learning to optimize' framework is used, based on the branch-and-bound algorithm for solving MINLP problems in wireless networks.
• The method first trains a policy on a source dataset with abundant labeled samples from a specific network configuration.
• For transfer, the pre-trained policy is blended with an exploration policy to traverse the branch-and-bound tree in the target network setting, generating unlabeled samples.
• The best solution found during exploration is used as a pseudo-label for fine-tuning the policy, avoiding the need for external labeling.
• The self-imitation process uses a dynamic blending ratio α = min(1, 0.2k) to balance exploitation and exploration during tree traversal.
• The approach is applied to Cloud-RAN fronthaul power minimization, where integer variables represent RRH selection and user association.

Experimental results

Research questions

• RQ1 Can transfer learning via self-imitation effectively mitigate performance degradation in machine learning-based resource allocation when network conditions change?
• RQ2 To what extent can a few unlabeled samples from a new environment enable effective policy fine-tuning without external labeling?
• RQ3 How does the proposed method compare to training from scratch in terms of performance gap and training speed?
• RQ4 Does the method maintain high performance across different network configurations, such as varying user counts, RRHs, and user distributions?

Key findings

• The proposed self-imitation transfer learning method achieves a performance gap of 0.7% to 2.4% from the optimal solution across different target SINR levels, comparable to the 'Train From Scratch' baseline.
• The method achieves a 10.1x to 15.6x speedup in training time compared to training from scratch, with the highest speedup observed in scenarios with similar source and target distributions.
• For dynamic user mobility, the method achieves a 22.9x to 23.4x speedup over the branch-and-bound algorithm, matching the performance of the full-label training baseline.
• The training speedup is attributed to knowledge transfer from the source task and elimination of label generation time, as no external labels are needed for fine-tuning.
• In the case of different network configurations (e.g., different RRH and user counts), the method still achieves a performance gap of 0.5% to 1.9% and a 13.4x speedup over branch-and-bound.