[论文解读] Identifying Semantically Duplicate Questions Using Data Science Approach: A Quora Case Study

Navedanjum Ansari, Rajesh Sharma|arXiv (Cornell University)|Apr 18, 2020

Topic Modeling被引用 8

一句话总结

本文提出了一种数据科学方法，利用机器学习和深度学习技术检测Quora上的语义重复问题。通过结合特征工程、基于字符级别的TF-IDF与XGBoost，以及四种深度神经网络架构，最佳模型实现了85.82%的准确率，超越了以往的机器学习模型，并接近Quora的最先进性能。

ABSTRACT

Identifying semantically identical questions on, Question and Answering social media platforms like Quora is exceptionally significant to ensure that the quality and the quantity of content are presented to users, based on the intent of the question and thus enriching overall user experience. Detecting duplicate questions is a challenging problem because natural language is very expressive, and a unique intent can be conveyed using different words, phrases, and sentence structuring. Machine learning and deep learning methods are known to have accomplished superior results over traditional natural language processing techniques in identifying similar texts. In this paper, taking Quora for our case study, we explored and applied different machine learning and deep learning techniques on the task of identifying duplicate questions on Quora's dataset. By using feature engineering, feature importance techniques, and experimenting with seven selected machine learning classifiers, we demonstrated that our models outperformed previous studies on this task. Xgboost model with character level term frequency and inverse term frequency is our best machine learning model that has also outperformed a few of the Deep learning baseline models. We applied deep learning techniques to model four different deep neural networks of multiple layers consisting of Glove embeddings, Long Short Term Memory, Convolution, Max pooling, Dense, Batch Normalization, Activation functions, and model merge. Our deep learning models achieved better accuracy than machine learning models. Three out of four proposed architectures outperformed the accuracy from previous machine learning and deep learning research work, two out of four models outperformed accuracy from previous deep learning study on Quora's question pair dataset, and our best model achieved accuracy of 85.82% which is close to Quora state of the art accuracy.

研究动机与目标

• 提升对Quora上语义相同问题的检测能力，以增强内容质量和用户体验。
• 解决自然语言可变性挑战，即相同意图通过不同措辞和结构表达。
• 评估并比较机器学习与深度学习模型在Quora问题对数据集上的表现。
• 识别在重复问题检测任务中最有效的特征工程与模型架构组合。

提出的方法

• 使用基于字符级别的词频与逆词频（TF-IDF）特征进行特征工程。
• 评估了七种机器学习分类器，其中XGBoost表现最佳。
• 设计了四种深度神经网络架构，采用GloVe词嵌入、LSTM、卷积层、最大池化、批量归一化和ReLU激活函数。
• 集成多层结构，包括全连接层、Dropout层和批量归一化层，以提升泛化能力与训练稳定性。
• 采用模型融合技术，整合多个深度学习架构的预测结果以提升性能。
• 在Quora问题对数据集上训练并验证模型，以准确率为首要评估指标。

实验结果

研究问题

• RQ1 基于工程化特征的机器学习模型能否在检测Quora上的语义重复问题方面超越以往方法？
• RQ2 在重复问题检测任务中，深度神经网络架构与传统机器学习模型在准确率上的表现如何比较？
• RQ3 基于字符级别的TF-IDF特征对本任务中模型性能有何影响？
• RQ4 集成或融合的深度学习模型能否超越Quora数据集上的最先进结果？
• RQ5 在该分类任务中，神经网络组件（如LSTM、CNN、归一化层）的最佳组合是什么？

主要发现

• 采用基于字符级别的TF-IDF特征的XGBoost模型在所有测试的机器学习模型中准确率最高。
• 最佳深度学习模型实现了85.82%的准确率，接近Quora的最先进性能。
• 四款深度神经网络架构中的三款在Quora数据集上超越了以往的机器学习与深度学习研究。
• 四款深度学习模型中的两款在相同数据集上超越了以往深度学习研究的准确率。
• 特征重要性分析表明，基于字符级别的特征对模型性能有显著贡献。
• 批量归一化与ReLU激活函数的集成显著提升了所有深度神经网络架构的训练收敛速度与泛化能力。