[Paper Review] Data-Dependent Differentially Private Parameter Learning for Directed Graphical Models

Amrita Roy Chowdhury, Theodoros Rekatsinas+1 more|arXiv (Cornell University)|May 30, 2019

Privacy-Preserving Technologies in Data59 references4 citations

TL;DR

This paper proposes a data-dependent differentially private algorithm for learning parameters in directed graphical models (DGMs), optimizing privacy budget allocation across conditional probability distributions based on data and graph structure. By customizing noise addition to node-specific sensitivity and influence, the method reduces required privacy budget by ~3×—achieving the same inference utility at ε=1.0 as a data-independent baseline at ε=3.0.

ABSTRACT

Directed graphical models (DGMs) are a class of probabilistic models that are widely used for predictive analysis in sensitive domains, such as medical diagnostics. In this paper we present an algorithm for differentially private learning of the parameters of a DGM with a publicly known graph structure over fully observed data. Our solution optimizes for the utility of inference queries over the DGM and extit{adds noise that is customized to the properties of the private input dataset and the graph structure of the DGM}. To the best of our knowledge, this is the first explicit data-dependent privacy budget allocation algorithm for DGMs. We compare our algorithm with a standard data-independent approach over a diverse suite of DGM benchmarks and demonstrate that our solution requires a privacy budget that is $3 imes$ smaller to obtain the same or higher utility.

Motivation & Objective

• Address the suboptimal utility of data-independent differential privacy (DP) in DGM parameter learning, where noise is not tailored to data or graph structure.
• Minimize error in inference queries over DGMs by explicitly allocating privacy budgets based on data-dependent properties and graph topology.
• Develop a novel, data-dependent optimization framework for privacy budget allocation across CPDs in DGMs, improving utility-efficiency.
• Provide theoretical bounds on inference query error, linking error to treewidth and maximum node degree.
• Enable practical deployment of DP in sensitive domains like healthcare by reducing the privacy cost for equivalent utility.

Proposed method

• Decompose DGM parameter learning into separate DP sub-problems for each conditional probability distribution (CPD), enabling per-node budget allocation.
• Use a two-stage approach: first, sample a subset of data at rate β to estimate preliminary parameters and error sensitivities with a small privacy budget ε^I.
• Formulate a data-dependent optimization objective that weighs node influence by height (h_i), out-degree (o_i), and estimated parameter error (δ̃_i) and sensitivity (Δ̃_i^N).
• Compute sensitivity of each CPD using a novel formulation that targets only the child node’s distribution, improving accuracy and privacy accounting.
• Optimize privacy budget allocation across nodes using a weighted objective: ∑(W[i]·δ̃_i/ε_i) + W[n]·δ̃_n/(ε^B - ε^I - ∑ε_i), where W[i] = (h_i+1)(o_i+1)(Δ̃_i^N + 1).
• Apply the Laplace mechanism with data-specific noise calibrated to the optimized ε_i values to ensure ε-BDP for the final model.

Experimental results

Research questions

• RQ1 Can data-dependent privacy budget allocation across CPDs in DGMs reduce the overall privacy budget required for a given inference utility?
• RQ2 How does the influence of a node—measured by height, out-degree, and parameter sensitivity—affect its optimal privacy budget allocation?
• RQ3 What are the theoretical bounds on inference query error in DP DGMs, and how do they relate to graph structure (e.g., treewidth, maximum degree)?
• RQ4 Can a two-stage sampling and optimization pipeline reduce the cost of initial parameter estimation while preserving utility?
• RQ5 How does the proposed method compare to data-independent DP baselines in terms of utility and privacy budget efficiency?

Key findings

• The proposed algorithm achieves the same or higher inference utility as a data-independent baseline at a privacy budget of ε=1.0, compared to ε=3.0 for the baseline.
• The method reduces the required privacy budget by approximately 3× while maintaining or improving utility across all four DGM benchmarks tested.
• Theoretical analysis shows that inference error upper bound depends exponentially on the treewidth of the DGM, while the lower bound depends on the maximum node degree.
• The sensitivity of a CPD is computed based on the child node’s distribution only, enabling more accurate and efficient noise calibration.
• The optimization objective successfully identifies high-impact nodes (e.g., root nodes) for higher budget allocation, improving overall inference accuracy.
• The two-stage approach with sampling (β) reduces the privacy cost of initial estimation (ε^I), freeing more budget for final parameter computation.