[Paper Review] Reliability and Robustness analysis of Machine Learning based Phishing URL Detectors

Bushra Sabir, M. Ali Babar+2 more|arXiv (Cornell University)|May 18, 2020

Adversarial Robustness in Machine Learning4 citations

TL;DR

This paper proposes a comprehensive methodology to evaluate the reliability and robustness of 50 state-of-the-art machine learning-based phishing URL (MLPU) detectors. It introduces URLBUG, a cost-effective adversarial URL generator that creates syntactically valid, registrable URLs with a median annual registration cost of $11.99, 63.94% of which are used maliciously. The study reveals that high-performing MLPU models (median MCC 0.92) degrade drastically under adversarial inputs (median MCC drops to 0.02), exposing critical security vulnerabilities in current systems.

ABSTRACT

ML-based Phishing URL (MLPU) detectors serve as the first level of defence to protect users and organisations from being victims of phishing attacks. Lately, few studies have launched successful adversarial attacks against specific MLPU detectors raising questions about their practical reliability and usage. Nevertheless, the robustness of these systems has not been extensively investigated. Therefore, the security vulnerabilities of these systems, in general, remain primarily unknown which calls for testing the robustness of these systems. In this article, we have proposed a methodology to investigate the reliability and robustness of 50 representative state-of-the-art MLPU models. Firstly, we have proposed a cost-effective Adversarial URL generator URLBUG that created an Adversarial URL dataset. Subsequently, we reproduced 50 MLPU (traditional ML and Deep learning) systems and recorded their baseline performance. Lastly, we tested the considered MLPU systems on Adversarial Dataset and analyzed their robustness and reliability using box plots and heat maps. Our results showed that the generated adversarial URLs have valid syntax and can be registered at a median annual price of \$11.99. Out of 13\% of the already registered adversarial URLs, 63.94\% were used for malicious purposes. Moreover, the considered MLPU models Matthew Correlation Coefficient (MCC) dropped from a median 0.92 to 0.02 when tested against $Adv_\mathrm{data}$, indicating that the baseline MLPU models are unreliable in their current form. Further, our findings identified several security vulnerabilities of these systems and provided future directions for researchers to design dependable and secure MLPU systems.

Motivation & Objective

• To investigate the robustness and reliability of state-of-the-art MLPU detectors against adversarial inputs.
• To identify security vulnerabilities in existing MLPU models that could be exploited in real-world phishing attacks.
• To develop a scalable and cost-effective method for generating realistic adversarial URLs to test model robustness.
• To provide actionable insights for designing more secure and dependable MLPU systems.

Proposed method

• Proposed URLBUG, a targeted adversarial URL generator that applies word-level and character-level perturbations to create syntactically valid, registrable URLs.
• Reproduced 50 representative MLPU models, including both traditional ML and deep learning models, using reported architectures and feature sets.
• Constructed an adversarial URL dataset ($Adv_{\mathrm{data}}$) by generating URLs with specific perturbations such as TLD swaps, homoglyphs, and character-level substitutions.
• Evaluated model performance on $Adv_{\mathrm{data}}$ using standard metrics including Matthew Correlation Coefficient (MCC), accuracy, and F1-score.
• Visualized results using box plots and heat maps to compare robustness across different models, features, and perturbation types.
• Conducted ablation studies to identify which feature types and classifiers are most resilient to adversarial attacks.

Experimental results

Research questions

• RQ1 How effective are existing MLPU models in detecting adversarial URLs generated via realistic, registrable perturbations?
• RQ2 What is the real-world feasibility and malicious utility of generated adversarial URLs in terms of registration cost and actual use in phishing campaigns?
• RQ3 Which model architectures and feature representations are most resilient to adversarial perturbations in the phishing URL detection task?
• RQ4 What are the key security vulnerabilities in current MLPU detectors that lead to high failure rates under adversarial conditions?
• RQ5 Can robustness metrics from computer vision and NLP be adapted to the phishing detection domain for black-box evaluation?

Key findings

• Adversarial URLs generated by URLBUG are syntactically valid and can be registered at a median annual cost of $11.99, indicating real-world feasibility.
• Of the 13% of adversarial URLs that were already registered, 63.94% were actively used for malicious purposes, confirming their practical threat potential.
• The median Matthew Correlation Coefficient (MCC) of 50 MLPU models dropped from 0.92 to 0.02 when tested on the adversarial dataset, indicating severe reliability degradation.
• Basic lexical and external features (e.g., URL length, TLD, presence of suspicious keywords) showed higher resilience, with less than 30% misclassification under adversarial testing.
• LSTM-based classifiers demonstrated near-perfect detection (99.99% accuracy) for path-level adversarial perturbations, suggesting a strong sensitivity to structural changes.
• Character-level perturbations caused the highest failure rate, with 90% of models misclassifying such adversarial URLs, highlighting a critical vulnerability in current models.