[Paper Review] Dropping Activation Outputs with Localized First-layer Deep Network for Enhancing User Privacy and Data Security

Hao Dong, Chao Wu+2 more|arXiv (Cornell University)|Nov 20, 2017

Privacy-Preserving Technologies in Data5 citations

TL;DR

This paper proposes a localized first-layer deep network architecture that enhances user privacy by performing initial deep network computations on local devices, applying activation functions locally, and using 'dropping activation outputs' to render transmitted data non-invertible. The method achieves strong privacy protection without additional encryption overhead, enabling secure model inference and training while preserving data confidentiality for sensitive health data.

ABSTRACT

Deep learning methods can play a crucial role in anomaly detection, prediction, and supporting decision making for applications like personal health-care, pervasive body sensing, etc. However, current architecture of deep networks suffers the privacy issue that users need to give out their data to the model (typically hosted in a server or a cluster on Cloud) for training or prediction. This problem is getting more severe for those sensitive health-care or medical data (e.g fMRI or body sensors measures like EEG signals). In addition to this, there is also a security risk of leaking these data during the data transmission from user to the model (especially when it's through Internet). Targeting at these issues, in this paper we proposed a new architecture for deep network in which users don't reveal their original data to the model. In our method, feed-forward propagation and data encryption are combined into one process: we migrate the first layer of deep network to users' local devices, and apply the activation functions locally, and then use "dropping activation output" method to make the output non-invertible. The resulting approach is able to make model prediction without accessing users' sensitive raw data. Experiment conducted in this paper showed that our approach achieves the desirable privacy protection requirement, and demonstrated several advantages over the traditional approach with encryption / decryption

Motivation & Objective

• To address the critical privacy and security risks associated with transmitting raw sensitive data—such as fMRI or EEG signals—to remote servers for deep learning inference and training.
• To eliminate the need for traditional encryption/decryption by integrating data obfuscation directly into the feed-forward process of deep networks.
• To enable secure model prediction and training where the server never accesses the user's original data, even under honest-but-curious threat models.
• To demonstrate that dropping activation outputs can effectively prevent reconstruction of original inputs, especially when combined with non-invertible or carefully randomized activation functions.

Proposed method

• Migrate the first layer of the deep neural network to the user’s local device, performing all initial computations—including weight multiplication and bias addition—on-device.
• Apply activation functions (e.g., sigmoid, ReLU) locally on the raw input data, generating activation outputs before transmission.
• Implement 'dropping activation outputs' by randomly removing a subset of activation values before sending them to the server, making the data non-invertible.
• Use the maximum activation value in the input as a seed for random dropout selection, ensuring consistent and deterministic dropping for the same input across multiple transmissions.
• Combine feed-forward propagation and data obfuscation into a single process, eliminating the need for separate encryption and decryption steps.
• Propose a privacy-preserving error-backpropagation mechanism that allows the server to train the model without accessing original user data, by splitting the network across local and remote components.

Experimental results

Research questions

• RQ1 Can dropping activation outputs in a deep neural network’s first layer effectively prevent reconstruction of the original input data, even for invertible activation functions like sigmoid?
• RQ2 How does the proposed method compare to traditional encryption in terms of computational cost, security, and feasibility for real-time applications?
• RQ3 Can the method preserve model performance while ensuring that the transmitted data cannot be inverted to recover sensitive inputs?
• RQ4 What is the impact of different activation functions (e.g., sigmoid vs. ReLU) on the security and privacy guarantees of the proposed approach?
• RQ5 Is it possible to perform error-backpropagation securely across the local-server boundary without exposing raw data?

Key findings

• The proposed method successfully prevents reconstruction of original input data from transmitted activation outputs, even under an honest-but-curious threat model, by making the data non-invertible through random output dropping.
• A brute-force attack on a sigmoid-activated MNIST classifier with 800 outputs and 0.5% dropout would require approximately 5,083 hours on a Titan X Pascal GPU, demonstrating high computational infeasibility.
• The KL divergence between reconstructed and original inputs remained low (e.g., 1.35–2.04) even with 0.5% to 2% dropout, indicating that while reconstruction is difficult, model utility is preserved.
• Using the maximum activation value as a random seed for dropout selection ensures consistent dropping for the same input, preventing reconstruction through repeated queries.
• The method achieves data privacy without additional encryption overhead, as obfuscation is inherently integrated into the feed-forward process.
• The approach enables secure error-backpropagation during training, allowing the server to update model weights without accessing raw user data, thus supporting privacy-preserving model learning.