[Paper Review] Review of Peer-to-Peer Botnets and Detection Mechanisms

Khoh Choon Hwa, Selvakumar Manickam+1 more|arXiv (Cornell University)|Jul 17, 2022

Network Security and Intrusion Detection4 citations

TL;DR

This paper provides a comprehensive review of peer-to-peer (P2P) botnet architectures, command-and-control (C&C) mechanisms, and detection techniques, analyzing their strengths, limitations, and real-time detection challenges. It evaluates graph-based, data mining, machine learning, and generic frameworks, concluding that no single method reliably detects evolving botnets in real time, highlighting the urgent need for adaptive, on-the-fly detection systems.

ABSTRACT

Cybercrimes are becoming a bigger menace to both people and corporations. It poses a serious challenge to the modern digital world. According to a press release from 2019 Cisco and Cybersecurity Ventures, Cisco stopped seven trillion threats in 2018, or 20 billion threats every day, on behalf of its clients. According to Cybersecurity Ventures, the global cost of cybercrime will reach \$6 trillion annually by 2021, which is significantly more than the annual damage caused by all natural disasters and more profitable than the global trade in all major illegal narcotics put together. Malware software, including viruses, worms, spyware, keyloggers, Trojan horses, and botnets, is therefore frequently used in cybercrime. The most common malware employed by attackers to carry out cybercrimes is the botnet, which is available in a variety of forms and for a variety of purposes when attacking computer assets. However, the issue continues to exist and worsen, seriously harming both enterprises and people who conduct their business online. The detection of P2P (Peer to Peer) botnet, which has emerged as one of the primary hazards in network cyberspace for acting as the infrastructure for several cyber-crimes, has proven more difficult than regular botnets using a few existing approaches. As a result, this study will explore various P2P botnet detection algorithms by outlining their essential characteristics, advantages and disadvantages, obstacles, and future research.

Motivation & Objective

• To analyze the lifecycle, taxonomy, and communication structures of P2P botnets.
• To evaluate existing detection mechanisms—graph-based, data mining, machine learning, and generic frameworks—across key performance dimensions.
• To identify critical limitations in current detection approaches, especially their reliance on offline analysis and poor scalability.
• To highlight the urgent need for real-time, adaptive detection systems capable of on-the-fly traffic analysis and classification.
• To provide a foundation for future research by summarizing open challenges and research gaps in P2P botnet detection.

Proposed method

• Conducts a systematic review of P2P botnet detection techniques using a taxonomy of detection mechanisms: graph-based (e.g., BotGrep), data mining, machine learning, and generic frameworks (e.g., BotMiner).
• Analyzes graph-based detection by modeling P2P botnet traffic as network graphs and identifying structural anomalies using Kademlia-based topology patterns.
• Applies data mining techniques involving feature extraction (e.g., API calls, binary strings, program actions) followed by classification using clustering or supervised learning models.
• Evaluates machine learning-based detection using flow-based features extracted via random forests and supervised learning for real-time classification of network traffic.
• Assesses generic frameworks like BotMiner that rely on packet and flow-level correlation to detect botnet behavior, though with limited scalability.
• Compares detection methods across criteria: detection of known/unknown bots, encrypted C&C channels, real-time capability, and accuracy, using tabulated performance summaries.

Experimental results

Research questions

• RQ1 How do P2P botnet architectures differ in design and communication structure, particularly in decentralized C&C mechanisms?
• RQ2 What are the key strengths and limitations of graph-based, data mining, and machine learning-based detection methods for identifying P2P botnets?
• RQ3 Why do existing detection techniques fail to achieve consistent real-time detection of evolving botnet traffic?
• RQ4 What are the primary challenges in detecting botnets with encrypted command-and-control channels or unknown threat families?
• RQ5 What are the critical research gaps that must be addressed to enable scalable, real-time, and adaptive P2P botnet detection?

Key findings

• Graph-based detection methods like BotGrep effectively identify P2P botnets by modeling network traffic as graphs and detecting structural anomalies, particularly in Kademlia-based topologies.
• Data mining-based approaches achieve high detection accuracy by extracting dynamic features such as API calls and binary patterns, but require high-quality training data and are sensitive to model quality.
• Machine learning-based detection using flow-based features and random forests enables real-time classification of network traffic, though performance depends heavily on feature engineering and model training.
• Generic frameworks like BotMiner are effective for small-scale detection but suffer from poor scalability due to reliance on fine-grained packet and flow analysis.
• No single detection method consistently identifies both known and unknown botnets, especially when C&C channels are encrypted or protocols are obfuscated.
• The majority of current detection techniques are offline, batch-oriented, and lack real-time adaptability, creating a critical gap in defending against rapidly evolving P2P botnet threats.