[论文解读] Wireless Data Acquisition for Edge Learning: Data-Importance Aware Retransmission

Dongzhu Liu, Guangxu Zhu+2 more|arXiv (Cornell University)|Dec 5, 2018

Wireless Signal Modulation Classification参考文献 31被引用 7

一句话总结

本文提出重要性ARQ（importance ARQ），一种面向无线边缘学习的数据重要性感知重传协议，通过基于支持向量机（SVM）中到决策边界的距离或卷积神经网络（CNN）中预测类别概率的熵来衡量样本不确定性，优先重传不确定性更高的样本，并利用信噪比（SNR）与不确定性之间的关系实现高效重传。该方法在信道衰落和数据不平衡条件下均显著提升了模型收敛速度，尤其在数据不平衡场景下性能提升最高达10%。

ABSTRACT

By deploying machine-learning algorithms at the network edge, edge learning can leverage the enormous real-time data generated by billions of mobile devices to train AI models, which enable intelligent mobile applications. In this emerging research area, one key direction is to efficiently utilize radio resources for wireless data acquisition to minimize the latency of executing a learning task at an edge server. Along this direction, we consider the specific problem of retransmission decision in each communication round to ensure both reliability and quantity of those training data for accelerating model convergence. To solve the problem, a new retransmission protocol called data-importance aware automatic-repeat-request (importance ARQ) is proposed. Unlike the classic ARQ focusing merely on reliability, importance ARQ selectively retransmits a data sample based on its uncertainty which helps learning and can be measured using the model under training. Underpinning the proposed protocol is a derived elegant communication-learning relation between two corresponding metrics, i.e., signal-to-noise ratio (SNR) and data uncertainty. This relation facilitates the design of a simple threshold based policy for importance ARQ. The policy is first derived based on the classic classifier model of support vector machine (SVM), where the uncertainty of a data sample is measured by its distance to the decision boundary. The policy is then extended to the more complex model of convolutional neural networks (CNN) where data uncertainty is measured by entropy. Extensive experiments have been conducted for both the SVM and CNN using real datasets with balanced and imbalanced distributions. Experimental results demonstrate that importance ARQ effectively copes with channel fading and noise in wireless data acquisition to achieve faster model convergence than the conventional channel-aware ARQ.

研究动机与目标

• 为解决由分布式设备不可靠的无线数据采集所导致的边缘学习延迟瓶颈问题。
• 克服经典ARQ的局限性，后者对所有数据一视同仁，不考虑其学习价值。
• 设计一种基于数据样本对模型收敛重要性的重传协议。
• 利用训练数据中重要性分布不均的特性——尤其是靠近决策边界的样本——实现更高效的无线资源分配。
• 在包含信道衰落和数据不平衡等现实条件下的性能进行评估。

提出的方法

• 提出一种新型重传协议——重要性ARQ，利用数据不确定性作为重传优先级的度量标准。
• 建立信噪比（SNR）与数据不确定性的通信-学习关联关系，实现基于阈值的重传决策。
• 对于SVM，不确定性通过样本到决策超平面的距离衡量；对于CNN，不确定性通过预测类别概率的熵衡量。
• 基于SNR-不确定性权衡关系推导出阈值策略，以判断是否应对某一样本进行重传。
• 在真实世界数据集上，分别在数据平衡与不平衡场景下应用该协议，评估学习收敛性能。
• 采用t-SNE可视化与重传预算分析，验证协议对高不确定性样本的自适应行为。

实验结果

研究问题

• RQ1 基于数据不确定性的重传优先级策略在无线边缘学习中如何提升模型收敛性能？
• RQ2 SNR与数据不确定性之间的基本关系是什么，使其能够实现高效的重传控制？
• RQ3 在信道衰落与噪声条件下，重要性ARQ相较于传统信道感知ARQ的性能表现如何？
• RQ4 在数据不平衡分类任务中，该协议是否在少数类样本更为关键的场景下带来更大收益？
• RQ5 该重传策略能否有效从SVM扩展至复杂模型（如CNN）？

主要发现

• 重要性ARQ在高噪声与信道衰落条件下，相比传统ARQ实现了更快的模型收敛速度。
• 在数据不平衡分类任务中，重要性ARQ在F-measure与G-mean指标上相较信道感知ARQ最高提升10%。
• 该协议将更多重传资源分配给高不确定性样本，特别是对学习至关重要的少数类样本。
• t-SNE可视化结果表明，少数类样本具有更高不确定性，且被重要性ARQ优先处理，而信道感知ARQ则对所有类别一视同仁。
• SNR-不确定性关系使基于阈值的重传策略简单而有效，能够根据信道状态与数据价值自适应调整。
• 该方法可从SVM成功迁移至CNN，基于熵的不确定性度量在深度学习模型中实现了有效的优先级分配。