[Paper Review] Cache Telepathy: Leveraging Shared Resource Attacks to Learn DNN Architectures

Mengjia Yan, Christopher W. Fletcher+1 more|arXiv (Cornell University)|Aug 14, 2018

Adversarial Robustness in Machine Learning85 citations

TL;DR

The paper presents Cache Telepathy, a cache-side-channel attack that reverse-engineers DNN architectures by analyzing GEMM parameter usage during inference, drastically narrowing the architecture search space.

ABSTRACT

Deep Neural Networks (DNNs) are fast becoming ubiquitous for their ability to attain good accuracy in various machine learning tasks. A DNN's architecture (i.e., its hyper-parameters) broadly determines the DNN's accuracy and performance, and is often confidential. Attacking a DNN in the cloud to obtain its architecture can potentially provide major commercial value. Further, attaining a DNN's architecture facilitates other, existing DNN attacks. This paper presents Cache Telepathy: a fast and accurate mechanism to steal a DNN's architecture using the cache side channel. Our attack is based on the insight that DNN inference relies heavily on tiled GEMM (Generalized Matrix Multiply), and that DNN architecture parameters determine the number of GEMM calls and the dimensions of the matrices used in the GEMM functions. Such information can be leaked through the cache side channel. This paper uses Prime+Probe and Flush+Reload to attack VGG and ResNet DNNs running OpenBLAS and Intel MKL libraries. Our attack is effective in helping obtain the architectures by very substantially reducing the search space of target DNN architectures. For example, for VGG using OpenBLAS, it reduces the search space from more than $10^{35}$ architectures to just 16.

Motivation & Objective

• Demonstrate that DNN inference relies on tiled GEMM and that architecture hyper-parameters influence GEMM calls and matrix shapes.
• Show that cache side channels can leak GEMM parameters to infer DNN architectures.
• Develop a practical attack pipeline combining cache attacks with analysis to prune the DNN search space.
• Evaluate the attack on common DNNs (VGG, ResNet) using OpenBLAS and Intel MKL to demonstrate effectiveness.

Proposed method

• Analyze how DNN hyper-parameters map to GEMM execution across fully-connected and convolutional layers.
• Translate DNN layer configurations into GEMM matrix sizes and batch dimensions to identify observable patterns.
• Implement a cache-based side-channel attack (Prime+Probe and Flush+Reload) to extract GEMM dimensions (m, n, k) from OpenBLAS (and MKL) implementations.
• Use observed GEMM parameters to reconstruct and prune the DNN architecture search space.
• Apply a post-processing step to correlate layer connections and activations, refining architecture candidates.

Experimental results

Research questions

• RQ1 Can cache timing side channels reveal the GEMM parameter patterns corresponding to DNN hyper-parameters?
• RQ2 To what extent can observed GEMM dimensions constrain the DNN architecture search space in MLaaS environments?
• RQ3 How effectively can the attack identify layer types, counts, and connectivity in VGG/ResNet-like networks running common BLAS libraries?
• RQ4 What is the impact of different libraries (OpenBLAS, MKL) on the observability of GEMM parameters?
• RQ5 Can the method distinguish activation functions and other non-GEMM components through cache observations?

Key findings

• Cache Telepathy can deduce DNN architectures by monitoring GEMM parameter usage during inference.
• For VGG with OpenBLAS, the attack reduces the architecture search space from over 1e35 to 16 possible architectures.
• The attack targets general-purpose CPUs used in MLaaS and relies on Prime+Probe and Flush+Reload techniques.
• The mapping from DNN hyper-parameters to GEMM arguments is analyzable for both fully-connected and convolutional layers, including pooling and padding effects.
• The approach applies to modern frameworks (TensorFlow, Caffe, Theano, MXNet) and libraries (OpenBLAS, MKL).
• The method identifies non-sequential connections (shortcuts/branches) by exploiting inter-GEMM latency patterns and dimension constraints.