[论文解读] Algorithmic Bias and Data Bias: Understanding the Relation between Distributionally Robust Optimization and Data Curation

Agnieszka Słowik, Léon Bottou|arXiv (Cornell University)|Jun 17, 2021

Adversarial Robustness in Machine Learning参考文献 41被引用 9

一句话总结

本文建立了分布鲁棒优化（DRO）与在子群体加权混合数据上训练之间的理论等价性，表明DRO本身并不能从根本上解决数据偏差，而是继承了训练数据中的校准假设。文章认为，DRO与数据整理单独使用均无法确保公平性，建议通过仔细校准和足够的模型容量来避免次优解。

ABSTRACT

Machine learning systems based on minimizing average error have been shown to perform inconsistently across notable subsets of the data, which is not exposed by a low average error for the entire dataset. In consequential social and economic applications, where data represent people, this can lead to discrimination of underrepresented gender and ethnic groups. Given the importance of bias mitigation in machine learning, the topic leads to contentious debates on how to ensure fairness in practice (data bias versus algorithmic bias). Distributionally Robust Optimization (DRO) seemingly addresses this problem by minimizing the worst expected risk across subpopulations. We establish theoretical results that clarify the relation between DRO and the optimization of the same loss averaged on an adequately weighted training dataset. The results cover finite and infinite number of training distributions, as well as convex and non-convex loss functions. We show that neither DRO nor curating the training set should be construed as a complete solution for bias mitigation: in the same way that there is no universally robust training set, there is no universal way to setup a DRO problem and ensure a socially acceptable set of results. We then leverage these insights to provide a mininal set of practical recommendations for addressing bias with DRO. Finally, we discuss ramifications of our results in other related applications of DRO, using an example of adversarial robustness. Our results show that there is merit to both the algorithm-focused and the data-focused side of the bias debate, as long as arguments in favor of these positions are precisely qualified and backed by relevant mathematics known today.

研究动机与目标

• 阐明DRO与在子群体加权混合数据上训练之间的数学关系。
• 探究DRO是否能完全解决机器学习系统中的算法偏差与数据偏差。
• 识别DRO与数据整理作为公平性部署独立解决方案的局限性。
• 为以尊重子群体性能并避免局部极小值陷阱的方式应用DRO提供实用建议。
• 考察这些发现对对抗鲁棒性及其他DRO应用的影响。

提出的方法

• 使用拉格朗日松弛法对DRO进行理论分析，将其与子群体上的混合优化联系起来。
• 推导出混合损失函数的局部极小值对应于特定校准系数下的DRO解的条件。
• 使用凸与非凸损失函数，将结果推广至线性模型之外。
• 分析迭代DRO算法（如算法2），其基于子群体表现动态调整混合权重。
• 研究因模型容量限制或某一子群体损失占主导而导致DRO失效的情境。
• 将理论结果应用于对抗鲁棒性，表明基于DRO的防御机制存在类似的局限性。

实验结果

研究问题

• RQ1 DRO在数学上如何与在子群体损失加权平均下最小化相关？
• RQ2 在何种条件下，混合损失函数的局部极小值对应于具有特定校准系数的DRO解？
• RQ3 在缺乏先前数据整理或模型容量调整的情况下，DRO能否保证子群体间公平性的提升？
• RQ4 当某一子群体的损失主导优化过程时，DRO存在哪些局限性？
• RQ5 考虑到类似的结构约束，DRO在对抗鲁棒性中能多大程度上有效应用？

主要发现

• DRO在数学上等价于最小化子群体损失的加权平均，其中权重由校准系数决定。
• 仅当校准系数设置得当时，混合损失函数的局部极小值才对应于DRO解，且该等价性在凸与非凸损失下均成立。
• 若模型即使仅在某一子群体数据上单独训练也无法达到可接受的性能，则DRO无法解决偏差问题。
• 若迭代DRO算法被困于较差的局部极小值，尤其当某一子群体主导损失景观时，其性能可能无法提升。
• 过参数化有助于摆脱较差的局部极小值，但前提是模型具备足够的容量以表征子群体间的权衡。
• 在对抗鲁棒性中，DRO面临类似的局限性：若模型本质上无法处理最坏情况的子群体，则无法保证鲁棒性。