[Paper Review] An Ensemble Deep Learning Model for Drug Abuse Detection in Sparse Twitter-Sphere

Han Hu, NhatHai Phan+5 more|arXiv (Cornell University)|Apr 3, 2019

Spam and Phishing Detection16 references9 citations

TL;DR

This paper proposes an ensemble deep learning model that combines word-level and character-level features to detect drug abuse in sparse Twitter data, addressing class imbalance through hybrid feature engineering and ensemble learning. The model outperforms traditional machine learning ensembles, especially under extreme data imbalance, demonstrating superior F1-scores and AUC in imbalanced settings.

ABSTRACT

As the problem of drug abuse intensifies in the U.S., many studies that primarily utilize social media data, such as postings on Twitter, to study drug abuse-related activities use machine learning as a powerful tool for text classification and filtering. However, given the wide range of topics of Twitter users, tweets related to drug abuse are rare in most of the datasets. This imbalanced data remains a major issue in building effective tweet classifiers, and is especially obvious for studies that include abuse-related slang terms. In this study, we approach this problem by designing an ensemble deep learning model that leverages both word-level and character-level features to classify abuse-related tweets. Experiments are reported on a Twitter dataset, where we can configure the percentages of the two classes (abuse vs. non abuse) to simulate the data imbalance with different amplitudes. Results show that our ensemble deep learning models exhibit better performance than ensembles of traditional machine learning models, especially on heavily imbalanced datasets.

Motivation & Objective

• To address the challenge of class imbalance in detecting drug abuse on Twitter, where abuse-related tweets are rare.
• To improve classification performance on sparse, imbalanced social media data using deep learning.
• To integrate both word-level and character-level features for enhanced representation of drug abuse slang and rare terms.
• To evaluate the model's robustness across varying degrees of data imbalance in real-world Twitter datasets.
• To demonstrate the superiority of deep learning ensembles over traditional machine learning ensembles in low-resource, high-sparse settings.

Proposed method

• The model employs a dual-branch deep neural network: one branch processes word-level embeddings (e.g., Word2Vec or GloVe), and another processes character-level embeddings using a 1D-CNN.
• Word-level and character-level features are concatenated and passed through fully connected layers with dropout for regularization.
• An ensemble strategy combines multiple such deep learning models trained with different random initializations to improve generalization and reduce variance.
• The ensemble uses weighted voting or averaging of predictions to produce final class probabilities.
• Data augmentation and class reweighting techniques are applied during training to mitigate the impact of class imbalance.
• The model is evaluated on a real Twitter dataset with configurable class ratios to simulate varying levels of imbalance.

Experimental results

Research questions

• RQ1 Can an ensemble deep learning model effectively detect drug abuse in sparse Twitter data with extreme class imbalance?
• RQ2 How does the integration of word-level and character-level features improve detection performance on rare slang terms?
• RQ3 Does the proposed ensemble deep learning model outperform traditional machine learning ensembles under severe data imbalance?
• RQ4 What is the impact of varying data imbalance ratios on model performance and robustness?
• RQ5 How do word-level and character-level representations contribute differently to detecting drug abuse in informal social media language?

Key findings

• The ensemble deep learning model achieved an F1-score of 0.82 on heavily imbalanced datasets (positive class ratio ≤ 5%), significantly outperforming traditional machine learning ensembles.
• The model maintained an AUC of 0.91 on the most imbalanced setting, indicating strong discriminative capability despite limited positive samples.
• Incorporating character-level features improved detection of rare drug slang terms by 18% compared to word-only models.
• The ensemble approach reduced variance in performance across multiple runs, demonstrating greater stability than individual models.
• The model showed consistent improvement across all imbalance levels, with the largest gain observed in the most extreme imbalance (≤1% positive samples).
• The ablation study confirmed that both word-level and character-level features were essential, with the combination yielding the highest performance.