[Paper Review] pForest: In-Network Inference with Random Forests

Coralie Busse-Grawitz, Roland Meier+3 more|arXiv (Cornell University)|Sep 12, 2019

Internet Traffic Analysis and Secure E-voting31 references20 citations

TL;DR

pForest enables ASAP (As Soon As Possible) in-network traffic classification using programmable data planes by training a sequence of context-aware random forest models that dynamically switch during flow lifetime. It achieves accuracy on par with software-based ML systems while operating under hardware constraints of switches, enabling early, accurate classification for hundreds of thousands of flows.

ABSTRACT

When classifying network traffic, a key challenge is deciding when to perform the classification, i.e., after how many packets. Too early, and the decision basis is too thin to classify a flow confidently; too late, and the tardy labeling delays crucial actions (e.g., shutting down an attack) and invests computational resources for too long (e.g., tracking and storing features). Moreover, the optimal decision timing varies across flows. We present pForest, a system for "As Soon As Possible" (ASAP) in-network classification according to supervised machine learning models on top of programmable data planes. pForest automatically classifies each flow as soon as its label is sufficiently established, not sooner, not later. A key challenge behind pForest is finding a strategy for dynamically adapting the features and the classification logic during the lifetime of a flow. pForest solves this problem by: (i) training random forest models tailored to different phases of a flow; and (ii) dynamically switching between these models in real time, on a per-packet basis. pForest models are tuned to fit the constraints of programmable switches (e.g., no floating points, no loops, and limited memory) while providing a high accuracy. We implemented a prototype of pForest in Python (training) and P4 (inference). Our evaluation shows that pForest can classify traffic ASAP for hundreds of thousands of flows, with a classification score that is on-par with software-based solutions.

Motivation & Objective

• To address the challenge of deciding the optimal timing for in-network traffic classification, avoiding both premature and delayed decisions.
• To enable accurate, real-time classification in the data plane using machine learning models that adapt to changing flow characteristics over time.
• To design a system that operates within the strict constraints of programmable switches—no floating points, no loops, limited memory—while maintaining high classification accuracy.
• To automatically generate and deploy a sequence of random forest models tailored to different phases of a flow’s lifetime, enabling dynamic model switching on a per-packet basis.
• To demonstrate that in-network inference can be both early and accurate, outperforming existing approaches that classify at fixed packet counts or only at flow end.

Proposed method

• pForest trains a sequence of context-dependent random forests, each optimized for a specific phase of a flow’s lifetime (e.g., early, mid, late).
• It dynamically switches between these models in real time, on a per-packet basis, based on flow progress and confidence in label prediction.
• The system automatically approximates statistical features (e.g., mean, variance) using integer arithmetic to avoid floating-point operations, fitting switch hardware constraints.
• It compiles trained random forest models into P4 programs using a custom compilation strategy that optimizes for memory and pipeline efficiency.
• The framework tunes model hyperparameters (e.g., depth, number of trees, thresholds) via a non-sampled grid search to maximize accuracy under hardware limits.
• It leverages the interpretability and robustness of random forests to maintain high accuracy even under feature drift and limited computational resources.

Experimental results

Research questions

• RQ1 Can in-network classification be performed as early as possible while maintaining high accuracy, without relying on fixed packet thresholds?
• RQ2 How can machine learning models be dynamically adapted during a flow’s lifetime to reflect evolving feature relevance and prediction confidence?
• RQ3 To what extent can random forests be compiled and executed efficiently in programmable data planes with no floating-point support and limited memory?
• RQ4 Can a sequence of context-specific random forests outperform static models in early and accurate traffic classification?
• RQ5 How does the performance of pForest compare to software-based ML systems in terms of classification accuracy and timing?

Key findings

• pForest achieves classification accuracy comparable to software-based scikit-learn implementations, demonstrating that hardware-constrained in-network inference can match offline performance.
• The system classifies flows as soon as the label is sufficiently established—neither too early nor too late—enabling timely responses to network events like DDoS attacks.
• pForest successfully handles hundreds of thousands of concurrent flows with low-latency, per-packet model switching based on flow progression.
• The use of context-dependent random forests allows the system to adapt feature importance and decision logic across flow phases, improving prediction confidence over time.
• The compilation pipeline effectively maps complex random forest models into P4 code under strict switch constraints, including integer-only arithmetic and no dynamic memory allocation.
• pForest outperforms existing in-network classifiers that classify at fixed packet counts or only at flow end, proving the value of dynamic, phase-aware model selection.