[Paper Review] An Explainable Transformer-based Model for Phishing Email Detection: A Large Language Model Approach

Mohammad Amaz Uddin, Iqbal H. Sarker+1 more|arXiv (Cornell University)|Feb 21, 2024

Spam and Phishing Detection4 citations

TL;DR

This paper proposes a fine-tuned DistilBERT model enhanced with Explainable AI (XAI) techniques—LIME and Transformer Interpret—for phishing email detection. By leveraging contextual embeddings from a masked language model and applying class imbalance mitigation, the model achieves 98.48% test accuracy on a balanced dataset, with XAI methods providing transparent, human-interpretable explanations of classification decisions.

ABSTRACT

Phishing email is a serious cyber threat that tries to deceive users by sending false emails with the intention of stealing confidential information or causing financial harm. Attackers, often posing as trustworthy entities, exploit technological advancements and sophistication to make detection and prevention of phishing more challenging. Despite extensive academic research, phishing detection remains an ongoing and formidable challenge in the cybersecurity landscape. Large Language Models (LLMs) and Masked Language Models (MLMs) possess immense potential to offer innovative solutions to address long-standing challenges. In this research paper, we present an optimized, fine-tuned transformer-based DistilBERT model designed for the detection of phishing emails. In the detection process, we work with a phishing email dataset and utilize the preprocessing techniques to clean and solve the imbalance class issues. Through our experiments, we found that our model effectively achieves high accuracy, demonstrating its capability to perform well. Finally, we demonstrate our fine-tuned model using Explainable-AI (XAI) techniques such as Local Interpretable Model-Agnostic Explanations (LIME) and Transformer Interpret to explain how our model makes predictions in the context of text classification for phishing emails.

Motivation & Objective

• To address the growing challenge of sophisticated phishing attacks that evade traditional detection methods.
• To improve phishing email detection accuracy using state-of-the-art transformer-based Large Language Models (LLMs).
• To mitigate class imbalance in phishing email datasets through preprocessing and resampling techniques.
• To enhance model interpretability by integrating Explainable AI (XAI) techniques for trust and transparency.
• To demonstrate that high-performance phishing detection can be both accurate and explainable using fine-tuned DistilBERT and XAI methods.

Proposed method

• Fine-tuned DistilBERT, a distilled version of BERT, for binary text classification of phishing vs. legitimate emails.
• Applied data preprocessing techniques including text cleaning, tokenization, and handling of class imbalance via oversampling or undersampling.
• Optimized hyperparameters such as batch size, learning rate, and optimizer (e.g., AdamW) to improve model convergence and performance.
• Utilized a Masked Language Model (MLM) objective during fine-tuning to preserve contextual understanding of words and sub-words.
• Employed LIME and Transformer Interpret to generate local, instance-level explanations by attributing prediction confidence to individual words or sub-words.
• Evaluated model performance using standard NLP metrics including accuracy, precision, recall, and F1-score on both imbalanced and balanced datasets.

Figure 1 : The proposed overall methodology.

Experimental results

Research questions

• RQ1 How effective is a fine-tuned DistilBERT model in detecting phishing emails compared to existing approaches?
• RQ2 To what extent does class imbalance affect the performance of transformer-based models in phishing detection?
• RQ3 How do LIME and Transformer Interpret provide actionable, human-understandable explanations for model predictions?
• RQ4 Can explainability techniques improve user trust and model transparency in cybersecurity applications?
• RQ5 What are the differences in explanation fidelity and word attribution between LIME and Transformer Interpret in phishing email classification?

Key findings

• The fine-tuned DistilBERT model achieved 98.90% training and 97.50% testing accuracy on the imbalanced dataset, demonstrating strong generalization despite class skew.
• On the balanced dataset, the model reached 99.07% training and 98.48% testing accuracy, confirming improved performance with balanced data.
• The model’s performance remained robust on imbalanced data, with only a 1% drop in test accuracy, indicating effective handling of class imbalance.
• LIME and Transformer Interpret provided complementary explanations by assigning positive or negative attribution scores to words and sub-words, highlighting key predictive features.
• Words with higher positive attribution scores were consistently linked to phishing content, such as urgent language or suspicious URLs, while negative scores were associated with benign terms.
• The explanation techniques revealed that model decisions were grounded in semantically meaningful linguistic patterns, enhancing interpretability and trust in predictions.

Figure 2 : The sample data overview