[论文解读] Characterizing the Decision Boundary of Deep Neural Networks

Hamid Reza Karimi, Tyler Derr+1 more|arXiv (Cornell University)|Dec 24, 2019

Adversarial Robustness in Machine Learning参考文献 38被引用 48

一句话总结

该论文提出 DeepDIG 框架，使用基于自编码器的定向对抗样本和二进制细化，在 DNN 决策边界附近生成边界样本，以刻画输入空间和嵌入空间的边界复杂性。

ABSTRACT

Deep neural networks and in particular, deep neural classifiers have become an integral part of many modern applications. Despite their practical success, we still have limited knowledge of how they work and the demand for such an understanding is evergrowing. In this regard, one crucial aspect of deep neural network classifiers that can help us deepen our knowledge about their decision-making behavior is to investigate their decision boundaries. Nevertheless, this is contingent upon having access to samples populating the areas near the decision boundary. To achieve this, we propose a novel approach we call Deep Decision boundary Instance Generation (DeepDIG). DeepDIG utilizes a method based on adversarial example generation as an effective way of generating samples near the decision boundary of any deep neural network model. Then, we introduce a set of important principled characteristics that take advantage of the generated instances near the decision boundary to provide multifaceted understandings of deep neural networks. We have performed extensive experiments on multiple representative datasets across various deep neural network models and characterized their decision boundaries. The code is publicly available at https://github.com/hamidkarimi/DeepDIG/.

研究动机与目标

• Motivate the study of deep neural network decision boundaries for safety and security applications.
• Develop a method to generate samples near decision boundaries that resemble real data.
• Characterize the geometry and complexity of decision boundaries in both input and embedding spaces.
• Provide metrics and experimental evidence across multiple datasets and models.

提出的方法

• Propose DeepDIG to generate borderline instances between two classes using a three-component pipeline.
• Component I: generate targeted adversarial examples from class s to class t via an autoencoder-based loss combining reconstruction and cross-entropy.
• Component II: generate reverse targeted adversarial examples from the first component’s outputs to push samples back toward class s.
• Component III: refine borderline samples with a binary search between the two adversarial samples to locate points where class probabilities are nearly equal (be near the decision boundary).
• Define and compute boundary-related metrics: (a) input-space boundary complexity via oscillation along trajectories between borderline samples, and (b) embedding-space boundary complexity via linear separability of embeddings using a trained linear SVM.]
• Component II: generate reverse targeted adversarial examples from the first component’s outputs to push samples back toward class s.
• Component III: refine borderline samples with a binary search between the two adversarial samples to locate points where class probabilities are nearly equal (be near the decision boundary).
• Define and compute boundary-related metrics: (a) input-space boundary complexity via oscillation along trajectories between borderline samples, and (b) embedding-space boundary complexity via linear separability of embeddings using a trained linear SVM.

实验结果

研究问题

• RQ1 How can we generate samples that are near the decision boundary between two classes in a pre-trained DNN?
• RQ2 What are effective metrics to quantify the complexity of decision boundaries in input and embedding spaces?
• RQ3 Do borderline samples in input space correspond to similarly informative structure in the embedding space?
• RQ4 Can the proposed border-focused samples reveal geometrical properties of DNN decision regions across datasets and architectures?

主要发现

• DeepDIG can generate borderline instances close to the decision boundary for pairs of classes across MNIST, FashionMNIST, and CIFAR-10.
• Borderline samples are produced on both sides of a boundary, enabling analysis of the boundary’s geometry.
• Embedding-space analysis shows that linear separability of embeddings persists for borderline instances, enabling embedding-space complexity metrics.
• Two embedding-space metrics (EDC1 and EDC2) provide insight into how close borderline samples lie to a separating hyperplane and how well a linear classifier can separate them.
• Experiments demonstrate consistency between input-space and embedding-space boundary complexity measures across several networks.