[Paper Review] Optimal Query Complexity of Secure Stochastic Convex Optimization

Wei Tang, Chien-Ju Ho+1 more|arXiv (Cornell University)|Apr 5, 2021

Stochastic Gradient Optimization Techniques4 citations

TL;DR

This paper proposes a secure learning protocol for stochastic convex optimization that protects the learner’s privacy against an adversary observing only query points. By combining uniform random queries over partitioned domains with efficient optimization subroutines, the method achieves optimal query complexity up to logarithmic factors, balancing accuracy and privacy under formal PAC-style guarantees for both error types.

ABSTRACT

We study the secure stochastic convex optimization problem. A learner aims to learn the optimal point of a convex function through sequentially querying a (stochastic) gradient oracle. In the meantime, there exists an adversary who aims to free-ride and infer the learning outcome of the learner from observing the learner's queries. The adversary observes only the points of the queries but not the feedback from the oracle. The goal of the learner is to optimize the accuracy, i.e., obtaining an accurate estimate of the optimal point, while securing her privacy, i.e., making it difficult for the adversary to infer the optimal point. We formally quantify this tradeoff between learner's accuracy and privacy and characterize the lower and upper bounds on the learner's query complexity as a function of desired levels of accuracy and privacy. For the analysis of lower bounds, we provide a general template based on information theoretical analysis and then tailor the template to several families of problems, including stochastic convex optimization and (noisy) binary search. We also present a generic secure learning protocol that achieves the matching upper bound up to logarithmic factors.

Motivation & Objective

• To formalize the trade-off between learner accuracy and privacy in stochastic convex optimization under adversarial observation of query points.
• To characterize the query complexity required to achieve desired levels of accuracy and privacy using PAC-style definitions.
• To develop a secure learning protocol that achieves matching upper bounds on query complexity up to logarithmic factors.
• To generalize information-theoretical lower bounds for secure optimization to broader problem families, including noisy binary search.
• To analyze the dependence of query complexity on privacy parameters, especially $\delta^{\rm adv}$, and show it represents a necessary complexity premium for privacy.

Proposed method

• A general information-theoretical template is developed to derive lower bounds on query complexity for convex optimization problems.
• The secure learning protocol partitions the domain $[0,1]$ into $\lfloor 1/\delta^{\rm adv} \rfloor$ subintervals of length $\delta^{\rm adv}$, ensuring randomization across intervals.
• Within each phase, the learner submits one query per subinterval, with one query updated via a non-secure but efficient optimization subroutine.
• The protocol mixes uniform random querying (for privacy) with adaptive optimization (for accuracy), ensuring resilience against any adversary strategy.
• The convergence rates are derived using a replicated bisection strategy inspired by Tsitsiklis et al., adapted for privacy.
• The protocol is proven to achieve $\tilde{\mathcal{O}}\big{(}(T\delta^{\rm adv})^{-\frac{\kappa}{2\kappa-2}}\big{)}$ function error and $\tilde{\mathcal{O}}\big{(}(T\delta^{\rm adv})^{-\frac{1}{2\kappa-2}}\big{)}$ point error with high probability.

Experimental results

Research questions

• RQ1 What is the minimum number of queries required to achieve a given level of accuracy and privacy in secure stochastic convex optimization?
• RQ2 How does the query complexity scale with privacy parameters $\delta^{\rm adv}$ and accuracy $\epsilon$?
• RQ3 Can a single information-theoretical template be used to derive lower bounds for both convex optimization and private binary search?
• RQ4 Is there a secure learning protocol that achieves the optimal query complexity up to logarithmic factors?
• RQ5 How does the secure protocol maintain privacy while preserving high optimization accuracy?

Key findings

• For $\kappa$-uniformly convex functions, the query complexity is $\Theta\left(1/(\delta^{\rm adv} \epsilon^{(2\kappa-2)/\kappa})\right)$ for function error, matching the lower and upper bounds up to logarithmic factors.
• For point error, the query complexity is $\Theta\left(1/(\delta^{\rm adv} \epsilon^{2\kappa-2})\right)$, again matching the bounds up to logarithmic factors.
• The lower bound analysis is generalized via an information-theoretical template applicable to both stochastic convex optimization and private binary search problems.
• The secure learning protocol achieves the same asymptotic query complexity as the lower bound, proving tightness of the derived bounds.
• The protocol remains private against any adversary strategy, as the randomization over $\lfloor 1/\delta^{\rm adv} \rfloor$ intervals prevents inference beyond $\epsilon^{\rm adv}$.
• The results recover classical convex optimization complexity bounds when $\delta^{\rm adv} \to 1$, showing the privacy cost is precisely $\Theta(1/\delta^{\rm adv})$.