[Paper Review] A Deep Learning Approach for Automatic Detection of Fake News

Tanik Saikh, Arkadipta De+2 more|arXiv (Cornell University)|May 11, 2020

Misinformation and Its Impacts20 references21 citations

TL;DR

This paper proposes two deep learning models—BiGRU-based and ELMo-based—for multi-domain fake news detection, achieving state-of-the-art performance by outperforming handcrafted feature-based methods by 3.08% and 9.3% on FakeNews AMT and Celebrity datasets, respectively, without relying on manual feature engineering.

ABSTRACT

Fake news detection is a very prominent and essential task in the field of journalism. This challenging problem is seen so far in the field of politics, but it could be even more challenging when it is to be determined in the multi-domain platform. In this paper, we propose two effective models based on deep learning for solving fake news detection problem in online news contents of multiple domains. We evaluate our techniques on the two recently released datasets, namely FakeNews AMT and Celebrity for fake news detection. The proposed systems yield encouraging performance, outperforming the current handcrafted feature engineering based state-of-the-art system with a significant margin of 3.08% and 9.3% by the two models, respectively. In order to exploit the datasets, available for the related tasks, we perform cross-domain analysis (i.e. model trained on FakeNews AMT and tested on Celebrity and vice versa) to explore the applicability of our systems across the domains.

Motivation & Objective

• Address the limitations of existing fake news detection systems that focus narrowly on political or satirical content.
• Extend fake news detection to a multi-domain setting, including politics, entertainment, sports, education, and business.
• Develop end-to-end deep learning models that eliminate the need for handcrafted linguistic features or complex NLP pipelines.
• Evaluate model generalization across domains through cross-domain training and testing on FakeNews AMT and Celebrity datasets.
• Investigate the role of domain-specific patterns in fake news detection and explore future integration of external knowledge and transfer learning.

Proposed method

• Employ a Bi-directional Gated Recurrent Unit (BiGRU) network to capture sequential dependencies in news text for classification.
• Utilize pre-trained ELMo embeddings to capture contextual word representations without relying on manual feature extraction.
• Design end-to-end deep neural networks that directly map raw text input to fake/legitimate labels, avoiding feature engineering.
• Train models on two multi-domain datasets: FakeNews AMT and Celebrity news, which contain topic-document-label triples across diverse domains.
• Perform cross-domain evaluation by training on one dataset and testing on the other to assess generalization capability.
• Apply attention mechanisms implicitly through the architecture's ability to focus on salient phrases in news content.

Experimental results

Research questions

• RQ1 Can deep learning models outperform traditional handcrafted feature-based approaches in multi-domain fake news detection?
• RQ2 How effective are BiGRU and ELMo-based models in detecting fake news across diverse domains such as politics, entertainment, and education?
• RQ3 To what extent do models trained on one domain generalize to another in the context of fake news detection?
• RQ4 What is the impact of domain-specific patterns on the performance and reliability of fake news detection systems?
• RQ5 How can external knowledge sources like WordNet or transfer learning models (e.g., BERT, XLNet) enhance detection performance in future extensions?

Key findings

• The BiGRU-based model achieved a 3.08% higher accuracy than the state-of-the-art SVM-based system on the FakeNews AMT dataset.
• The ELMo-based model outperformed the SOTA system by 9.3% on the Celebrity news dataset, demonstrating superior generalization in multi-domain settings.
• Cross-domain evaluation revealed that models trained on one dataset showed notable performance when tested on the other, indicating some degree of domain transferability.
• Misclassifications were observed in both domains, with legitimate news being falsely labeled as fake—particularly in entertainment and business topics—highlighting challenges in nuanced content.
• The study confirms that domain context plays a vital role in fake news detection, suggesting that domain-aware modeling could further improve performance.
• The results support the effectiveness of end-to-end deep learning architectures in fake news detection, especially when avoiding reliance on manual linguistic feature engineering.