[Paper Review] Explaining Machine Learning DGA Detectors from DNS Traffic Data

Giorgio Piras, Maura Pintor+2 more|arXiv (Cornell University)|Aug 10, 2022

Network Security and Intrusion Detection4 citations

TL;DR

This paper proposes a comprehensive explainable AI framework for machine learning-based DGA (Domain Generation Algorithm) detection in DNS traffic, using global and local interpretability techniques to decode model decisions. It re-implements the EXPOSURE system on a new dataset, demonstrating that feature importance varies across models and revealing context-dependent biases critical for debugging and trust.

ABSTRACT

One of the most common causes of lack of continuity of online systems stems from a widely popular Cyber Attack known as Distributed Denial of Service (DDoS), in which a network of infected devices (botnet) gets exploited to flood the computational capacity of services through the commands of an attacker. This attack is made by leveraging the Domain Name System (DNS) technology through Domain Generation Algorithms (DGAs), a stealthy connection strategy that yet leaves suspicious data patterns. To detect such threats, advances in their analysis have been made. For the majority, they found Machine Learning (ML) as a solution, which can be highly effective in analyzing and classifying massive amounts of data. Although strongly performing, ML models have a certain degree of obscurity in their decision-making process. To cope with this problem, a branch of ML known as Explainable ML tries to break down the black-box nature of classifiers and make them interpretable and human-readable. This work addresses the problem of Explainable ML in the context of botnet and DGA detection, which at the best of our knowledge, is the first to concretely break down the decisions of ML classifiers when devised for botnet/DGA detection, therefore providing global and local explanations.

Motivation & Objective

• Address the lack of interpretability in ML-based DGA and botnet detection systems, which hinders debugging, deployment, and trust.
• Provide both global and local explanations for ML classifiers used in DGA detection to improve model transparency and human readability.
• Investigate how feature selection and prioritization vary across different classifiers, revealing model-specific decision patterns.
• Analyze the impact of deployment context on feature relevance and model behavior, highlighting potential biases in real-world settings.
• Enable security analysts to interpret model decisions, detect misclassifications, and improve system robustness through explainability

Proposed method

• Re-implemented the EXPOSURE system on a newly collected DNS traffic dataset to ensure reproducibility and contextual relevance.
• Extracted a comprehensive feature set including Time-Based, DNS-Answer-Based, TTL-Value-Based, and Name-Based features from DNS packets.
• Trained multiple machine learning classifiers (e.g., XGBoost, Random Forest, SVM) and compared their performance on DGA detection.
• Applied SHAP (SHapley Additive exPlanations) for global and local interpretability to analyze feature contributions to predictions.
• Conducted statistical analysis on feature usage across models to identify prioritization patterns and potential biases.
• Evaluated model behavior under different deployment contexts to assess feature sensitivity and decision robustness

Experimental results

Research questions

• RQ1 How do different machine learning classifiers prioritize features in DGA detection, and what are the implications for model interpretability?
• RQ2 To what extent do feature importance patterns vary across models when applied to the same DNS traffic data?
• RQ3 How can local explanations from SHAP help identify and debug misclassified DGA samples in real-world deployment?
• RQ4 What contextual biases emerge in feature relevance when models are trained and deployed in different network environments?
• RQ5 Can global and local explainability techniques improve trust, fairness, and maintainability of ML-based DGA detectors?

Key findings

• Different classifiers prioritize distinct sets of features for DGA detection, indicating that model choice significantly influences decision-making patterns.
• The EXPOSURE feature set demonstrated strong performance across multiple classifiers, with Name-Based and TTL-Value-Based features showing high predictive importance.
• SHAP-based local explanations successfully highlighted specific DNS queries and features responsible for misclassifications, enabling targeted debugging.
• Feature relevance was context-dependent, with certain features (e.g., TTL statistics) becoming less reliable when deployed in environments with high traffic variability.
• Global explanation revealed that models often rely on a small subset of features—especially those related to domain name entropy and DNS response patterns—limiting robustness in adversarial settings.
• The study demonstrated that explainability enhances model trust and supports integration into security pipelines for continuous monitoring and refinement