[论文解读] Statistical MIA: Rethinking Membership Inference Attack for Reliable Unlearning Auditing

Jialong Sun, Zeming Wei+6 more|arXiv (Cornell University)|Feb 1, 2026

Adversarial Robustness in Machine Learning被引用 0

一句话总结

SMIA 提供一种训练无关、基于统计的机器学习忘记侦测框架，它直接比较成员数据和非成员数据分布以在置信区间内估计遗忘率，解决基于 MIA 的审计导致的错觉性遗忘问题。

ABSTRACT

Machine unlearning (MU) is essential for enforcing the right to be forgotten in machine learning systems. A key challenge of MU is how to reliably audit whether a model has truly forgotten specified training data. Membership Inference Attacks (MIAs) are widely used for unlearning auditing, where samples that evade membership detection are often regarded as successfully forgotten. After carefully revisiting the reliability of MIA, we show that this assumption is flawed: failed membership inference does not imply true forgetting. We theoretically demonstrate that MIA-based auditing, when formulated as a binary classification problem, inevitably incurs statistical errors whose magnitude cannot be observed during the auditing process. This leads to overly optimistic evaluations of unlearning performance, while incurring substantial computational overhead due to shadow model training. To address these limitations, we propose Statistical Membership Inference Attack (SMIA), a novel training-free and highly effective auditing framework. SMIA directly compares the distributions of member and non-member data using statistical tests, eliminating the need for learned attack models. Moreover, SMIA outputs both a forgetting rate and a corresponding confidence interval, enabling quantified reliability of the auditing results. Extensive experiments show that SMIA provides more reliable auditing with significantly lower computational cost than existing MIA-based approaches. Notably, the theoretical guarantees and empirical effectiveness of SMIA suggest it as a new paradigm for reliable machine unlearning auditing.

研究动机与目标

• 质疑基于 MIA 的无忘迟审计的可靠性。
• 提出一种训练无关的方法，通过分布比较对遗忘进行审计。
• 提供带置信区间的遗忘率估计，以量化审计可靠性。
• 显示基于 RKHS/MMD 的方法在较低计算成本下实现稳健审计。

提出的方法

• 将 SMIA 作为一个训练无关、模型不可知的审计框架引入。
• 定义使用低阶矩、核均值嵌入（RKHS）和 Wasserstein 距离的变体 SMIA-0、SMIA-M 和 SMIA-W。
• 将审计数据建模为混合分布 D_f = alpha D_t^v + (1-alpha) D_t^t，并通过对分布统计量的优化来估计 alpha。
• 使用自举法推导遗忘率 alpha 的置信区间。
• 在 SMIA-M 中，将分布嵌入 RKHS，将二阶矩转化为一阶信息，以实现高效优化。
• 在 SMIA-W 中，采用带熵正则化的 Wasserstein 距离进行鲁棒的分布距离估计。

Figure 1 : The relationship between the proportion of non-member data and successful TNR detection, under the configuration of MIA accuracy=0.99 and member detection success rate 0.9999.

实验结果

研究问题

• RQ1 基于 MIA 的审计是否能对遗忘给出可靠结论，还是分布偏移会产生错觉性遗忘？
• RQ2 是否存在训练无关、模型无关的审计方法能够准确估计遗忘率并给出置信区间？
• RQ3 核/ RKHS 基方法（SMIA-M）是否在审计鲁棒性和效率方面优于传统 MIA 方法？
• RQ4 在实际的无忘审计场景中，SMIA 的变体（SMIA-0、SMIA-M、SMIA-W）的相对性能和成本如何？

主要发现

• SMIA 通过避免学习的攻击模型，相较传统的基于 MIA 的方法提供更可靠的审计。
• SMIA-0 和 SMIA-M 在不同数据集的无忘审计中表现出强辨别能力，优于现有的 MIA 基线方法。
• SMIA-W 在报道的实验中表现出较弱的审计鲁棒性，未被进一步评估所青睐。
• 基于核均值嵌入的 SMIA-M 提供鲁棒性、较小样本需求以及有利的计算特征。
• 自举法的置信区间使遗忘率估计的可靠性可被定量化。
• 与影子模型基的 MIA 方法相比，SMIA 具有更低的计算成本，因为无需训练影子模型。

Figure 2 : (a) The dilemma faced by the attacker; (b) The dilemma faced by the auditer.