[论文解读] PoTrojan: powerful neural-level trojan designs in deep learning models

Minhui Zou, Yang Shi+4 more|arXiv (Cornell University)|Feb 8, 2018

Adversarial Robustness in Machine Learning参考文献 11被引用 49

一句话总结

这篇论文介绍 PoTrojan，即可在不重新训练的情况下插入到预训练神经网络中的神经元级木马，能够在罕见输入下触发并导致系统故障或错分，已在 AlexNet 和 VGG16 上进行了演示。

ABSTRACT

With the popularity of deep learning (DL), artificial intelligence (AI) has been applied in many areas of human life. Neural network or artificial neural network (NN), the main technique behind DL, has been extensively studied to facilitate computer vision and natural language recognition. However, the more we rely on information technology, the more vulnerable we are. That is, malicious NNs could bring huge threat in the so-called coming AI era. In this paper, for the first time in the literature, we propose a novel approach to design and insert powerful neural-level trojans or PoTrojan in pre-trained NN models. Most of the time, PoTrojans remain inactive, not affecting the normal functions of their host NN models. PoTrojans could only be triggered in very rare conditions. Once activated, however, the PoTrojans could cause the host NN models to malfunction, either falsely predicting or classifying, which is a significant threat to human society of the AI era. We would explain the principles of PoTrojans and the easiness of designing and inserting them in pre-trained deep learning models. PoTrojans doesn't modify the existing architecture or parameters of the pre-trained models, without re-training. Hence, the proposed method is very efficient.

研究动机与目标

• Introduce the concept of neuron-level trojans (PoTrojans) in pre-trained neural networks.
• Show toy examples and illustrate how PoTrojans can be designed and inserted without retraining.
• Present a general algorithm for crafting triggers and payloads for PoTrojans.
• Validate tacitness and harmfulness of PoTrojans on real-world models (AlexNet and VGG16).

提出的方法

• Define PoTrojan components: trigger synapses and payload synapses.
• Demonstrate single-neuron and multi-neuron PoTrojan designs to create rare triggering conditions.
• Provide trigger design equations to ensure low probability of activation.
• Describe payload design to steer outputs after activation, with and without access to training instances.
• Outline an algorithm (including gradient-based steps) to compute triggering and payload parameters.
• Experimentally validate on AlexNet and VGG16 trained on ImageNet to assess triggering and impact.

实验结果

研究问题

• RQ1 Can neuron-level trojans be inserted into pre-trained models without modifying architecture or retraining?
• RQ2 How can triggers be designed to activate PoTrojans under extremely rare conditions?
• RQ3 How should the payloads be crafted to meaningfully alter model outputs when triggered?
• RQ4 What is the tacitness (accidental triggering rate) and harmful impact of PoTrojans in real networks like AlexNet and VGG16?

主要发现

• PoTrojans can be inserted in pre-trained models without changing existing parameters or architecture, enabling efficient exploitation.
• Trigger inputs can activate PoTrojans with high reliability when present (examples show 100% triggering in tested insertions).
• Accidental triggering on non-trigger inputs is negligible in experiments (accident triggering rate reported as 0 for tested PoTrojans).
• Triggered PoTrojans can cause significant misclassification or targeted outputs, including forcing specific labels with high confidence in tested cases.
• Payload design allows steering outputs toward a chosen target (e.g., mislabeling input {1,1,1,1} as a specific label with high probability).
• Experiments on AlexNet and VGG16 demonstrate both triggering behavior and impactful outputs under trigger conditions.