[論文レビュー] A Deep Ensemble Framework for Fake News Detection and Classification

Arjun Roy, Kingshuk Basak+2 more|arXiv (Cornell University)|Nov 12, 2018

Misinformation and Its Impacts参考文献 8被引用数 71

ひとこと要約

tldr: An ensemble deep learning framework combining CNN and Bi-LSTM with relation-based attribute processing to perform fine-grained fake news classification on LIAR dataset, achieving 44.87% accuracy.

ABSTRACT

Fake news, rumor, incorrect information, and misinformation detection are nowadays crucial issues as these might have serious consequences for our social fabrics. The rate of such information is increasing rapidly due to the availability of enormous web information sources including social media feeds, news blogs, online newspapers etc. In this paper, we develop various deep learning models for detecting fake news and classifying them into the pre-defined fine-grained categories. At first, we develop models based on Convolutional Neural Network (CNN) and Bi-directional Long Short Term Memory (Bi-LSTM) networks. The representations obtained from these two models are fed into a Multi-layer Perceptron Model (MLP) for the final classification. Our experiments on a benchmark dataset show promising results with an overall accuracy of 44.87\%, which outperforms the current state of the art.

研究の動機と目的

• Motivate and address the need for automated, fine-grained fake news detection in short statements.
• Develop an ensemble deep learning model that leverages sequential patterns and hidden features to classify statements into six fine-grained classes.
• Incorporate speaker-profile attributes and their relations to improve information synthesis for classification.
• Evaluate the approach against existing models and analyze errors to identify limitations and future directions.

提案手法

• Use Bi-LSTM networks to capture sequential information from statements and attribute-specific inputs.
• Use CNNs to extract hidden features from attribute embeddings and credit-history related inputs.
• Process multiple relation-based attribute pairs through separate network layers to learn inter-attribute relations.
• Merge representations from CNN and Bi-LSTM branches and pass through dense layers to perform multi-class softmax classification across six labels.
• Train with Adadelta optimization and categorical cross-entropy loss on the LIAR dataset.
• Embed inputs using 300-dimensional Google News vectors and pad inputs to fixed lengths as described.

実験結果

リサーチクエスチョン

• RQ1 Can a deep ensemble of CNN and Bi-LSTM models improve fine-grained fake news classification over existing single-model approaches?
• RQ2 How do speaker attributes and their inter-relations contribute to improved classification of short political statements?
• RQ3 What is the impact of combining relation-specific networks on overall accuracy and per-class performance?

主な発見

モデル	ネットワーク	取得した属性	精度
William Yang Wang [8]	ハイブリッド CNN	すべて	0.274
Y. Long et al. [9]	ハイブリッド LSTM	すべて	0.415
Bi-LSTM Model	Bi-LSTM	すべて	0.4265
CNN Model	CNN	すべて	0.4289
私たちの提案モデル	RNN-CNN 組み合わせ	すべて	0.4487

• The proposed RNN-CNN ensemble achieves 0.4487 accuracy, outperforming state-of-the-art models cited (0.415 and 0.4265/0.4289 for individual baselines).
• Bi-LSTM and CNN individually reach 0.4265 and 0.4289 accuracy, while the combined model yields the best performance among tested configurations.
• Per-class precision, recall, and F1 show variability with TRUE class having high precision but low recall, indicating conservative yet accurate TRUE predictions.
• Confusion analysis reveals most errors occur among closely related classes (e.g., Pants-Fire vs False, Half-True vs Mostly True).
• The model benefits from explicit encoding of speaker profiles and their history counts, and later analyses suggest more labeled data would further improve performance.