[Paper Review] Detection of Unauthorized IoT Devices Using Machine Learning Techniques

Yair Meidan, Michael Bohadana+5 more|arXiv (Cornell University)|Sep 14, 2017

Network Security and Intrusion Detection6 references198 citations

TL;DR

The paper uses Random Forests on TCP/IP traffic features to automatically detect unauthorized IoT devices and correctly classify whitelisted types, achieving about 96% unknown detection and 99% correct whitelisted classification with a 20-session moving window.

ABSTRACT

Security experts have demonstrated numerous risks imposed by Internet of Things (IoT) devices on organizations. Due to the widespread adoption of such devices, their diversity, standardization obstacles, and inherent mobility, organizations require an intelligent mechanism capable of automatically detecting suspicious IoT devices connected to their networks. In particular, devices not included in a white list of trustworthy IoT device types (allowed to be used within the organizational premises) should be detected. In this research, Random Forest, a supervised machine learning algorithm, was applied to features extracted from network traffic data with the aim of accurately identifying IoT device types from the white list. To train and evaluate multi-class classifiers, we collected and manually labeled network traffic data from 17 distinct IoT devices, representing nine types of IoT devices. Based on the classification of 20 consecutive sessions and the use of majority rule, IoT device types that are not on the white list were correctly detected as unknown in 96% of test cases (on average), and white listed device types were correctly classified by their actual types in 99% of cases. Some IoT device types were identified quicker than others (e.g., sockets and thermostats were successfully detected within five TCP sessions of connecting to the network). Perfect detection of unauthorized IoT device types was achieved upon analyzing 110 consecutive sessions; perfect classification of white listed types required 346 consecutive sessions, 110 of which resulted in 99.49% accuracy. Further experiments demonstrated the successful applicability of classifiers trained in one location and tested on another. In addition, a discussion is provided regarding the resilience of our machine learning-based IoT white listing method to adversarial attacks.

Motivation & Objective

• Motivate enterprise IoT security through automated detection of unauthorized devices on networks.
• Develop a scalable, traffic-based white listing system that requires no specialized hardware.
• Demonstrate classification of IoT device types using machine learning on TCP/IP traffic data.
• Show transferability of trained models across labs and discuss resilience to adversarial attempts.

Proposed method

• Treat device type identification as a multi-class classification problem using labeled TCP/IP session features.
• Train a Random Forest classifier on sessions from whitelisted device types to produce posterior probabilities for each type.
• Use a classification threshold tr to label a session as a device type or as unknown based on posterior probabilities.
• Apply majority voting over a moving window of 20 consecutive sessions to decide the stream’s device type and improve robustness.
• Partition data temporally into training, validation, and test sets to simulate real-world deployment and mitigate overfitting.
• Undersample to address class imbalance and optimize the threshold using F1 on the validation set.

Experimental results

Research questions

• RQ1 Can TCP/IP traffic features distinguish among authorized IoT device types in a real enterprise setting?
• RQ2 Can unknown (unauthorized) IoT device types be reliably detected as 'unknown' based on single sessions and sequences of sessions?
• RQ3 Does majority voting over a sequence of sessions improve detection and classification accuracy of both authorized and unauthorized devices?
• RQ4 How well do classifiers trained in one lab transfer to data from another lab (transportability)?

Key findings

• Unauthorized IoT device types were detected as unknown in 96% of test cases when using a moving window of 110 sessions (single-stage analysis reported in abstract/sections).
• White listed device types were correctly classified by their actual type in 99% of cases on the test set with a 20-session majority vote.
• Across nine experiments (leaving one device type out of the white list each time), the average accuracy for detecting unknown devices was 96% and the average accuracy for classifying white listed devices was 99% on the test set.
• Transportability experiments showed that classifiers trained in Lab A achieved 85% average accuracy for unauthorized TVs when tested on Lab B, and complete transportability (100%) when detecting unauthorized security cameras across labs.
• The majority-vote stage substantially improved overall performance, raising unauthorized detection to 96% and white-listed classification to 99% on the test set for most device types.