[论文解读] Shielding Google's language toxicity model against adversarial attacks

Néstor Rodríguez, Sergio Rojas Galeano|arXiv (Cornell University)|Jan 5, 2018

Hate Speech and Cyberbullying Detection参考文献 7被引用 14

一句话总结

本文提出一种两阶段防御机制，以抵御针对 Google 的 Perspective 毒性模型的对抗性攻击，此类攻击利用拼写混淆和语法否定进行规避。通过应用文本去混淆方法，随后进行毒性评分，该方法将处理时间增加约两倍，但能有效应对字符级和极性相关的规避技术，恢复准确的毒性预测。

ABSTRACT

Lack of moderation in online communities enables participants to incur in personal aggression, harassment or cyberbullying, issues that have been accentuated by extremist radicalisation in the contemporary post-truth politics scenario. This kind of hostility is usually expressed by means of toxic language, profanity or abusive statements. Recently Google has developed a machine-learning-based toxicity model in an attempt to assess the hostility of a comment; unfortunately, it has been suggested that said model can be deceived by adversarial attacks that manipulate the text sequence of the comment. In this paper we firstly characterise such adversarial attacks as using obfuscation and polarity transformations. The former deceives by corrupting toxic trigger content with typographic edits, whereas the latter deceives by grammatical negation of the toxic content. Then, we propose a two--stage approach to counter--attack these anomalies, bulding upon a recently proposed text deobfuscation method and the toxicity scoring model. Lastly, we conducted an experiment with approximately 24000 distorted comments, showing how in this way it is feasible to restore toxicity of the adversarial variants, while incurring roughly on a twofold increase in processing time. Even though novel adversary challenges would keep coming up derived from the versatile nature of written language, we anticipate that techniques combining machine learning and text pattern recognition methods, each one targeting different layers of linguistic features, would be needed to achieve robust detection of toxic language, thus fostering aggression--free digital interaction.

研究动机与目标

• 解决 Google 的 Perspective 毒性模型在面对通过拼写混淆和语法否定操纵文本的对抗性攻击时的脆弱性。
• 开发一种预处理流程，将被篡改或被否定的有毒内容恢复为原始形式，再进行毒性评分。
• 在大规模对抗性修改的评论数据集上评估该防御策略的有效性。
• 证明鲁棒的毒性检测需要结合机器学习与模式识别的多层语言分析。

提出的方法

• 该方法采用一种近期提出的文本去混淆技术，以逆转诸如字母替换、符号插入和空格异常等拼写混淆操作。
• 针对极性相关的攻击，该方法移除或替换被否定的有毒谓词（例如，'not stupid'），以恢复原始的有毒意图。
• 将预处理后的文本输入原始 Google Perspective 毒性模型进行评分。
• 该流程在约 24,000 条对抗性扭曲评论上进行评估，以衡量准确性恢复和计算开销。
• 系统尽可能保留原始内容，避免内容修改或添加，以维持真实性。
• 该方法可通过适配 n-gram 模型和自定义混淆过滤器，扩展至其他语言。

实验结果

研究问题

• RQ1 利用拼写混淆的对抗性攻击是否能成功规避 Google 的 Perspective 毒性模型？
• RQ2 去混淆预处理器在多大程度上能恢复被混淆评论的准确毒性评分？
• RQ3 移除或替换被否定的有毒谓词在应对极性相关对抗性攻击方面有多有效？
• RQ4 在实际应用中，该防御流程的计算成本是多少？
• RQ5 该两阶段方法是否可推广至其他语言，以应对类似的对抗性模式？

主要发现

• 所提出的防御方法对对抗性混淆评论恢复了高准确度的毒性评分，有效应对了字符级混淆攻击。
• 对于涉及否定的极性相关攻击，通过移除或替换被否定的有毒谓词，成功恢复了原始毒性水平，减少了假阴性。
• 与直接评分相比，该方法导致处理时间约增加两倍，表明鲁棒性与效率之间存在权衡。
• 即使高达一半的有毒序列被混淆，该防御仍保持有效，但随着污染率升高，效果有所下降。
• 该方法在多种对抗性模式下均展现出恢复预期毒性评分的可行性，支持构建分层检测系统的需求。