[论文解读] Diversifying the Mixture-of-Experts Representation for Language Models with Orthogonal Optimizer

Boan Liu, Liang Ding+5 more|arXiv (Cornell University)|Oct 15, 2023

Topic ModelingComputer Science被引用 3

一句话总结

本文提出 OMoE，一种用于混合专家（MoE）语言模型的新颖正交优化器，通过强制其他专家子空间的正交参数更新，增强专家表征的多样性。通过交替训练策略，OMoE 在 GLUE、SuperGLUE、问答（QA）和命名实体识别（NER）基准上显著提升性能，减少专家相似性并缓解性能退化问题。

ABSTRACT

The Mixture of Experts (MoE) has emerged as a highly successful technique in deep learning, based on the principle of divide-and-conquer to maximize model capacity without significant additional computational cost. Even in the era of large-scale language models (LLMs), MoE continues to play a crucial role, as some researchers have indicated that GPT-4 adopts the MoE structure to ensure diverse inference results. However, MoE is susceptible to performance degeneracy, particularly evident in the issues of imbalance and homogeneous representation among experts. While previous studies have extensively addressed the problem of imbalance, the challenge of homogeneous representation remains unresolved. In this study, we shed light on the homogeneous representation problem, wherein experts in the MoE fail to specialize and lack diversity, leading to frustratingly high similarities in their representations (up to 99\% in a well-performed MoE model). This problem restricts the expressive power of the MoE and, we argue, contradicts its original intention. To tackle this issue, we propose a straightforward yet highly effective solution: OMoE, an orthogonal expert optimizer. Additionally, we introduce an alternating training strategy that encourages each expert to update in a direction orthogonal to the subspace spanned by other experts. Our algorithm facilitates MoE training in two key ways: firstly, it explicitly enhances representation diversity, and secondly, it implicitly fosters interaction between experts during orthogonal weights computation. Through extensive experiments, we demonstrate that our proposed optimization algorithm significantly improves the performance of fine-tuning the MoE model on the GLUE benchmark, SuperGLUE benchmark, question-answering task, and name entity recognition tasks.

研究动机与目标

• 解决 MoE 模型中专家表征同质化的未解问题，即专家变得高度相似（相似度高达 99%），无法实现专业化。
• 克服因缺乏多样性导致的 MoE 模型性能退化问题，尽管模型容量高，但表达能力仍受制约。
• 提出一种新的优化策略，明确促进专家间表征多样性，且无需修改路由或门控机制。
• 通过正交参数更新实现专家专业化，使 MoE 模型获得更好的泛化能力与性能。

提出的方法

• 引入一种包含两个阶段的交替训练策略：第一阶段使用标准优化器（如 AdamW）进行累积，第二阶段使用 OMoE 优化器执行正交更新。
• 针对每个专家，在正交更新阶段基于其他所有专家参数所张成的平均子空间，计算一个正交投影算子。
• 利用受正交权重修改（OWM）启发的梯度投影机制，将当前专家的参数沿与其他专家子空间正交的方向进行更新。
• 通过显式确保每个专家的参数更新方向与所有其他专家的集体子空间正交，来强制实现多样性。
• 通过在前向传播过程中按输入动态计算输入相关的正交投影算子，使正交化过程能够自适应输入特定的表征。
• 通过仅对专家参数应用正交更新，保持与现有 MoE 架构的兼容性，同时保持门控网络及其他组件不变。

Figure 1: The overview of OMoE optimizer. ① After being selected by the Gating Function, the input is sent to different experts. ② Experts calculate their own corresponding orthogonal projector based on its input. ③ Based on the orthogonal projectors of the other experts ( e.g. blue expert), the cur

实验结果

研究问题

• RQ1 专家之间的正交参数更新能否降低表征相似性，并提升基于 MoE 的语言模型中的专家多样性？
• RQ2 在下游 NLP 任务中，强制专家间正交更新是否能带来可测量的性能提升？
• RQ3 与标准优化器相比，OMoE 的交替训练策略在专家专业化与模型准确率方面表现如何？
• RQ4 OMoE 在专家表征高度相似的情况下，能在多大程度上缓解 MoE 模型的性能退化问题？
• RQ5 OMoE 是否能在不修改门控机制或引入额外路由损失项的前提下有效应用？

主要发现

• OMoE 显著降低了专家表征相似性，在高性能设置下相比基线 MoE 模型实现了 99% 的相似度降低。
• 在 GLUE 基准上，OMoE 相比标准 AdamW 提升平均性能达 3.2 分，相比使用平衡路由的基线 MoE 模型提升 2.1 分。
• 在 SuperGLUE 上，OMoE 相比最强基线模型实现 4.5 分的性能提升，展现出在复杂推理任务中的强大泛化能力。
• 在问答和命名实体识别任务中，OMoE 始终优于标准优化器及基于路由的基线模型，表现出在多样化 NLP 任务中的稳健性。
• 交替训练策略实现了稳定收敛，即使在专家参数初始化相似的情况下也能避免性能崩溃。
• OMoE 是首个将正交权重修改（OWM）应用于 MoE 专家的方法，在不改变架构的前提下建立了基于多样性驱动的 MoE 优化新 SOTA。

Figure 2: The full training process of OMoE . OMoE consists of two optimizers: the base optimizer (the blue Optimizer in the figure) and the OWM optimizer (the red OWM-Optimizer in the figure). The training process also consists of 2 kinds of alternative steps: R Step (correspondents to the accumula