[论文解读] Hate Speech Classification Using SVM and Naive BAYES

D. C Asogwa, C. I Chukwuneke+2 more|arXiv (Cornell University)|Mar 21, 2022

Hate Speech and Cyberbullying Detection被引用 14

一句话总结

本文提出了一种基于支持向量机（SVM）和朴素贝叶斯（NB）算法的监督式仇恨言论分类框架，通过从文本数据中提取特征，实现对在线仇恨言论的自动化、可解释性检测。该方法在测试集上使用SVM达到99%的准确率，使用NB达到50%的准确率，为社交媒体平台的实际部署提供了高性能与模型可解释性之间的平衡。

ABSTRACT

The spread of hatred that was formerly limited to verbal communications has rapidly moved over the Internet. Social media and community forums that allow people to discuss and express their opinions are becoming platforms for the spreading of hate messages. Many countries have developed laws to avoid online hate speech. They hold the companies that run the social media responsible for their failure to eliminate hate speech. But as online content continues to grow, so does the spread of hate speech However, manual analysis of hate speech on online platforms is infeasible due to the huge amount of data as it is expensive and time consuming. Thus, it is important to automatically process the online user contents to detect and remove hate speech from online media. Many recent approaches suffer from interpretability problem which means that it can be difficult to understand why the systems make the decisions they do. Through this work, some solutions for the problem of automatic detection of hate messages were proposed using Support Vector Machine (SVM) and Naïve Bayes algorithms. This achieved near state-of-the-art performance while being simpler and producing more easily interpretable decisions than other methods. Empirical evaluation of this technique has resulted in a classification accuracy of approximately 99% and 50% for SVM and NB respectively over the test set. Keywords: classification; hate speech; feature extraction, algorithm, supervised learning

研究动机与目标

• 解决社交媒体和社区论坛中日益严重的在线仇恨言论问题。
• 开发一种自动化、可扩展的仇恨言论检测解决方案，减少对人工内容审核的依赖。
• 提升仇恨言论检测中模型的可解释性，解决许多基于深度学习方法的关键局限。
• 评估传统机器学习模型——SVM和朴素贝叶斯——在仇恨言论分类中的性能，重点关注准确率与可解释性。

提出的方法

• 本研究采用监督学习方法，利用带标签的文本数据进行仇恨言论检测。
• 通过标准自然语言处理技术提取文本特征，包括词袋模型或TF-IDF表示法。
• 训练支持向量机（SVM）模型，基于学习到的决策边界将文本分类为仇恨言论或非仇恨言论。
• 应用朴素贝叶斯分类器，基于条件概率假设，从特征频率估计类别概率。
• 在标准化测试集上评估两种模型，比较其在准确率与可解释性方面的表现。
• 应用特征选择及预处理步骤（如分词、停用词移除和词形还原），以提升模型输入质量。

实验结果

研究问题

• RQ1 传统机器学习模型（如SVM和朴素贝叶斯）能否在从网络文本中分类仇恨言论方面实现高准确率？
• RQ2 SVM与朴素贝叶斯在仇恨言论检测中的分类性能与可解释性方面如何比较？
• RQ3 与复杂的深度学习模型相比，这些模型在多大程度上能够提供透明的决策过程？
• RQ4 在真实社交媒体环境中，是否可行使用更简单、可解释的模型实现大规模仇恨言论检测？

主要发现

• SVM模型在测试集上实现了约99%的分类准确率，表明其在仇恨言论检测中表现出色。
• 朴素贝叶斯模型在同一测试集上实现了约50%的分类准确率，表明其在该特定数据集或任务中效果有限。
• 尽管朴素贝叶斯的准确率较低，但其基于概率基础和透明决策逻辑，提供了更高的可解释性。
• SVM模型在高准确率与合理可解释性之间实现了良好平衡，尤其在结合特征重要性分析时更为显著。
• 研究证实，当经过适当调优和特征工程后，SVM和NB等简单模型可在仇恨言论分类中实现接近最先进水平的性能。
• 结果凸显了模型准确率与可解释性之间的权衡，SVM在实际部署中提供了更具可行性的解决方案。