[论文解读] NPC: Neuron Path Coverage via Characterizing Decision Logic of Deep Neural Networks

Xiaofei Xie, Tianlin Li+5 more|arXiv (Cornell University)|Mar 24, 2022

Adversarial Robustness in Machine Learning被引用 5

一句话总结

本文提出NPC（Neuron Path Coverage），一种用于深度神经网络（DNNs）的可解释性测试准则，通过从关键神经元激活中导出的决策图来建模决策逻辑。通过分析该图中的控制流和数据流路径，NPC利用两种路径覆盖变体——SNPC（基于控制流）和ANPC（基于数据流）——衡量测试充分性，展示了与缺陷检测及模型公平性之间的强相关性。

ABSTRACT

Deep learning has recently been widely applied to many applications across different domains, e.g., image classification and audio recognition. However, the quality of Deep Neural Networks (DNNs) still raises concerns in the practical operational environment, which calls for systematic testing, especially in safety-critical scenarios. Inspired by software testing, a number of structural coverage criteria are designed and proposed to measure the test adequacy of DNNs. However, due to the blackbox nature of DNN, the existing structural coverage criteria are difficult to interpret, making it hard to understand the underlying principles of these criteria. The relationship between the structural coverage and the decision logic of DNNs is unknown. Moreover, recent studies have further revealed the non-existence of correlation between the structural coverage and DNN defect detection, which further posts concerns on what a suitable DNN testing criterion should be. In this paper, we propose the interpretable coverage criteria through constructing the decision structure of a DNN. Mirroring the control flow graph of the traditional program, we first extract a decision graph from a DNN based on its interpretation, where a path of the decision graph represents a decision logic of the DNN. Based on the control flow and data flow of the decision graph, we propose two variants of path coverage to measure the adequacy of the test cases in exercising the decision logic. The higher the path coverage, the more diverse decision logic the DNN is expected to be explored. Our large-scale evaluation results demonstrate that: the path in the decision graph is effective in characterizing the decision of the DNN, and the proposed coverage criteria are also sensitive with errors including natural errors and adversarial examples, and strongly correlated with the output impartiality.

研究动机与目标

• 解决现有DNN结构化覆盖准则缺乏可解释性的问题，这些准则通常不透明且与实际决策逻辑的相关性较差。
• 通过结构化表示建模DNN的决策过程，克服基于神经元激活的覆盖（如神经元覆盖）的局限性。
• 建立基于DNN控制流和数据流的测试充分性准则，以更好地捕捉多样的决策行为。
• 通过使覆盖与实际决策逻辑对齐，提高测试对自然错误和对抗性样本的敏感性。
• 建立测试覆盖与模型行为之间的清晰、可解释的联系，从而实现更系统化和有意义的DNN测试。

提出的方法

• 通过分析训练数据中的激活模式，识别关键神经元，从DNN构建决策图。
• 将DNN的决策逻辑建模为决策图中的路径，其中每条路径代表跨层的神经元激活序列。
• 基于控制流路径定义SNPC（结构化神经元路径覆盖），衡量测试输入是否触发了不同的决策路径。
• 基于数据流路径定义ANPC（基于激活的神经元路径覆盖），衡量关键决策路径上激活值的多样性。
• 通过重要神经元激活值的聚类，对路径进行分组并识别代表性决策路径。
• 将决策图与路径覆盖准则集成到自动化测试流程中，以指导测试用例生成，从而最大化覆盖度。

实验结果

研究问题

• RQ1 能否有效提取并以图结构表示DNN中的决策路径，以建模网络的决策逻辑？
• RQ2 决策图中的控制流和数据流路径如何与DNN的实际行为和鲁棒性相关联？
• RQ3 所提出的路径覆盖（SNPC与ANPC）在多大程度上与缺陷检测相关，包括自然错误和对抗性样本？
• RQ4 与传统的基于神经元激活的覆盖相比，NPC的可解释性在多大程度上提升了对测试充分性的理解？
• RQ5 路径覆盖准则能否指导生成更多样化且更有效的DNN测试用例？

主要发现

• 决策图通过建模跨层关键神经元激活路径，成功捕捉了DNN的决策逻辑。
• SNPC与ANPC对自然错误和对抗性样本均表现出强敏感性，表明其在检测模型缺陷方面的有效性。
• NPC与模型输出的公正性高度相关，表明其在评估公平性与鲁棒性方面的实用性。
• 基于路径的覆盖准则在反映实际决策行为方面优于传统神经元覆盖，因其建模了跨层的逻辑依赖关系。
• 评估结果表明，更高的路径覆盖度可促进对多样化决策逻辑的更充分探索，从而提升测试充分性。
• NPC具有可解释性，能提供关于哪些决策路径未被充分测试的可操作洞察，从而支持有针对性的测试生成。