[Paper Review] Degree-Preserving Randomized Response for Graph Neural Networks under Local Differential Privacy

Seira Hidano, Takao Murakami|arXiv (Cornell University)|Feb 21, 2022

Privacy-Preserving Technologies in Data4 citations

TL;DR

This paper proposes Degree-Preserving Randomized Response (DPRR), a novel local differential privacy (LDP) mechanism for graph neural networks that preserves each node's degree while providing strong privacy for edges in unattributed graphs. By combining Warner's randomized response with Laplacian mechanism-based degree estimation and adaptive edge sampling, DPRR achieves graph classification accuracy close to non-private models at ε=1, significantly outperforming baselines in both accuracy and efficiency.

ABSTRACT

Differentially private GNNs (Graph Neural Networks) have been recently studied to provide high accuracy in various tasks on graph data while strongly protecting user privacy. In particular, a recent study proposes an algorithm to protect each user's feature vector in an attributed graph, which includes feature vectors along with node IDs and edges, with LDP (Local Differential Privacy), a strong privacy notion without a trusted third party. However, this algorithm does not protect edges (friendships) in a social graph, hence cannot protect user privacy in unattributed graphs, which include only node IDs and edges. How to provide strong privacy with high accuracy in unattributed graphs remains open. In this paper, we propose a novel LDP algorithm called the DPRR (Degree-Preserving Randomized Response) to provide LDP for edges in GNNs. Our DPRR preserves each user's degree hence a graph structure while providing edge LDP. Technically, our DPRR uses Warner's RR (Randomized Response) and strategic edge sampling, where each user's sampling probability is automatically tuned using the Laplacian mechanism to preserve the degree information under edge LDP. We also propose a privacy budget allocation method to make the noise in both Warner's RR and the Laplacian mechanism small. We focus on graph classification as a task of GNNs and evaluate the DPRR using three social graph datasets. Our experimental results show that the DPRR significantly outperforms three baselines and provides accuracy close to a non-private algorithm in all datasets with a reasonable privacy budget, e.g., epsilon=1. Finally, we introduce data poisoning attacks to our DPRR and a defense against the attacks. We evaluate them using the three social graph datasets and discuss the experimental results.

Motivation & Objective

• To address the lack of strong privacy protection for edges in unattributed graphs under local differential privacy (LDP).
• To preserve graph structural properties—specifically node degrees—while ensuring LDP for edges in GNNs.
• To develop an efficient and accurate LDP mechanism that outperforms existing baselines in graph classification tasks.
• To enable practical deployment of LDP in decentralized social networks where no trusted third party exists.

Proposed method

• Proposes DPRR, a two-stage LDP mechanism: first, use the Laplacian mechanism to add noise to each node’s true degree, then tune the sampling probability of Warner’s randomized response based on the noisy degree.
• Uses post-processing invariance to ensure that the final edge sampling step preserves the overall LDP guarantee.
• Applies sequential composition of LDP: the Laplacian mechanism provides ε₁-edge LDP, and Warner’s RR provides ε₂-edge LDP, resulting in total (ε₁+ε₂)-edge LDP.
• Introduces a privacy budget allocation strategy to minimize noise in both the degree estimation and edge flipping components.
• Employs adaptive edge sampling where each user samples neighbors with a probability proportional to their noisy degree, preserving structural information.
• Combines DPRR with existing feature-vector LDP methods to support joint protection of edges and attributes in attributed graphs.

Experimental results

Research questions

• RQ1 Can we design an LDP mechanism that preserves node degrees while protecting edges in unattributed graphs under local differential privacy?
• RQ2 How can we achieve high classification accuracy in GNNs with a small privacy budget (e.g., ε=1) while maintaining strong privacy guarantees?
• RQ3 Is DPRR more efficient and accurate than existing LDP baselines such as Randomized Response and LocalLap in graph classification tasks?
• RQ4 Does preserving node degrees lead to better model performance compared to methods that distort graph structure?

Key findings

• DPRR achieves graph classification accuracy within 2.5% of a non-private baseline at ε=1 across all three social graph datasets.
• DPRR significantly outperforms three baselines—RR, LocalLap, and NonPriv-Part—in classification accuracy, especially in the customized setting.
• The training time for DPRR is approximately 4 hours for a graph with 40,000 nodes and 5 edges per node, compared to ~21 days for the RR baseline.
• DPRR preserves node degrees with high fidelity, while RR and LocalLap distort degree distributions, which explains the performance gap.
• The degree distribution is critical for GNN performance in unattributed graphs, and DPRR maintains this distribution effectively.
• DPRR is highly efficient, requiring significantly less memory and time than RR, making it practical for large-scale decentralized SNSs.