[논문 리뷰] Witches' Brew: Industrial Scale Data Poisoning via Gradient Matching

Jonas Geiping, Liam Fowl외 5명|arXiv (Cornell University)|2020. 09. 04.

Adversarial Robustness in Machine Learning참고 문헌 53인용 수 36

한 줄 요약

이 논문은 gradient alignment (gradient matching)을 사용하여 처음부터 학습된 깊은 네트워크에 대해 확장 가능한 clean-label 대상 데이터 중독 공격을 소개합니다. 이 공격은 선택된 대상 이미지를 오분류하도록 훈련을 유도하기 위해 poisoned 데이터를 제작합니다.

ABSTRACT

Data Poisoning attacks modify training data to maliciously control a model trained on such data. In this work, we focus on targeted poisoning attacks which cause a reclassification of an unmodified test image and as such breach model integrity. We consider a particularly malicious poisoning attack that is both "from scratch" and "clean label", meaning we analyze an attack that successfully works against new, randomly initialized models, and is nearly imperceptible to humans, all while perturbing only a small fraction of the training data. Previous poisoning attacks against deep neural networks in this setting have been limited in scope and success, working only in simplified settings or being prohibitively expensive for large datasets. The central mechanism of the new attack is matching the gradient direction of malicious examples. We analyze why this works, supplement with practical considerations. and show its threat to real-world practitioners, finding that it is the first poisoning method to cause targeted misclassification in modern deep networks trained from scratch on a full-sized, poisoned ImageNet dataset. Finally we demonstrate the limitations of existing defensive strategies against such an attack, concluding that data poisoning is a credible threat, even for large-scale deep learning systems.

연구 동기 및 목표

• 작은 집합의 트레이닝 이미지가 bound 내에서 perturbation되어 특정 대상 이미지가 오분류되도록 만들기 위한 targeted data poisoning를 동기화하고 형식화한다.
• 대규모 데이터셋(ImageNet 등)에서 처음부터 학습된 심층 네트워크를 대상으로 작동하는 확장 가능한 공격을 개발한다.
• poisoned 데이터의 gradient를 adversarial target gradient와 정렬시키는 효율적인 최적화 objective를 제안한다.
• 아키텍처와 학습 설정에 걸친 공격의 실용성과 전이 가능성을 평가한다.
• 현 defenses를 평가하고 현재 완화 전략의 한계를 논의한다.]
• method':['대상 데이터의 gradient alignment를 통해 poisoned 데이터를 구성: adversarial loss gradient와 poisoned 데이터 gradient의 합 사이의 음의 코사인 유사도 최소화','perturbation을 l_infty bound 하에 최적화하여 clean-label 의미를 보존하고 사람이 인지하지 못하게 한다',' differentiable 데이터 증강과 무작위 재시작을 사용하여 초기화와 아키텍처 간 전이 가능성을 높인다','전체 bilevel 역전파를 피하고 하나의 사전 학습된 모델과 한 에폭에 해당하는 최적화만으로 효율성을 입증한다','포이즈닝 과정에서 theta를 업데이트하지 않고 포이즈닝에 영향을 주도록 단일 매개변수 벡터 theta를 활용한다]
• research_questions:[

제안 방법

• 대상 데이터의 gradient alignment를 통해 poisoned 데이터를 구성: adversarial loss gradient와 poisoned 데이터 gradient의 합 사이의 음의 코사인 유사도 최소화
• perturbation을 l_infty bound 하에 최적화하여 clean-label 의미를 보존하고 사람이 인지하지 못하게 한다
• differentiable 데이터 증강과 무작위 재시작을 사용하여 초기화와 아키텍처 간 전이 가능성을 높인다
• 전체 bilevel 역전파를 피하고 하나의 사전 학습된 모델과 한 에폭에 해당하는 최적화만으로 효율성을 입증한다
• 포이즈닝 과정에서 theta를 업데이트하지 않고 포이즈닝에 영향을 주도록 단일 매개변수 벡터 theta를 활용한다]
• research_questions: ["Can gradient alignment enable effective clean-label targeted data poisoning on modern deep nets trained from scratch?","How does the proposed gradient-matching poisoning scale to large datasets like ImageNet and to different architectures?","What role do data augmentation, restarts, and model ensembles play in transferability and robustness of the attack?","Are existing defenses (sanitization, differential privacy) effective against gradient-matching poisoning, and what are their trade-offs?"]
• key_findings: ["The attack achieves targeted misclassification with as little as 0.1% poisoned data on ImageNet when perturbations are bounded (ε=8).","Gradient alignment-based poisoning substantially outperforms prior methods (e.g., MetaPoison) in both efficiency and success on CIFAR-10 and large-scale ImageNet experiments.","Differentiable data augmentation can substitute for large model ensembles, achieving comparable poisoning effectiveness with lower computational costs.","Poisoning transfers to other architectures (e.g., MobileNet-V2, ResNet-50) and can be effective in black-box settings (Cloud AutoML) under realistic threat models.","Defenses like sanitization are largely ineffective against this attack, and differential privacy trades off validation accuracy to reduce poisoning success.","Theoretical analysis via an adversarial descent framework explains why gradient alignment can steer training toward minimizing the adversarial loss."]
• table_headers: []
• table_rows: []} {

실험 결과

연구 질문

• RQ1 Can gradient alignment enable effective clean-label targeted data poisoning on modern deep nets trained from scratch?
• RQ2 How does the proposed gradient-matching poisoning scale to large datasets like ImageNet and to different architectures?
• RQ3 What role do data augmentation, restarts, and model ensembles play in transferability and robustness of the attack?
• RQ4 Are existing defenses (sanitization, differential privacy) effective against gradient-matching poisoning, and what are their trade-offs?

주요 결과

• The attack achieves targeted misclassification with as little as 0.1% poisoned data on ImageNet when perturbations are bounded (ε=8).
• Gradient alignment-based poisoning substantially outperforms prior methods (e.g., MetaPoison) in both efficiency and success on CIFAR-10 and large-scale ImageNet experiments.
• Differentiable data augmentation can substitute for large model ensembles, achieving comparable poisoning effectiveness with lower computational costs.
• Poisoning transfers to other architectures (e.g., MobileNet-V2, ResNet-50) and can be effective in black-box settings (Cloud AutoML) under realistic threat models.
• Defenses like sanitization are largely ineffective against this attack, and differential privacy trades off validation accuracy to reduce poisoning success.
• Theoretical analysis via an adversarial descent framework explains why gradient alignment can steer training toward minimizing the adversarial loss.