[论文解读] Attacking Automatic Video Analysis Algorithms: A Case Study of Google Cloud Video Intelligence API

Hossein Hosseini, Baicen Xiao+2 more|arXiv (Cornell University)|Aug 14, 2017

Anomaly Detection Techniques and Applications参考文献 34被引用 9

一句话总结

本文提出针对 Google Cloud Video Intelligence API 的定向对抗性攻击，通过微妙地操纵视频帧，欺骗视频分类与镜头切换检测算法。该攻击利用确定性帧采样机制——每秒插入一张图像，即可导致 API 根据该图像对整个视频进行标注；同时，通过极小的帧扰动即可操纵基于直方图的镜头切换检测，揭示了真实世界视频分析系统中严重的鲁棒性缺陷。

ABSTRACT

Due to the growth of video data on Internet, automatic video analysis has gained a lot of attention from academia as well as companies such as Facebook, Twitter and Google. In this paper, we examine the robustness of video analysis algorithms in adversarial settings. Specifically, we propose targeted attacks on two fundamental classes of video analysis algorithms, namely video classification and shot detection. We show that an adversary can subtly manipulate a video in such a way that a human observer would perceive the content of the original video, but the video analysis algorithm will return the adversary's desired outputs. We then apply the attacks on the recently released Google Cloud Video Intelligence API. The API takes a video file and returns the video labels (objects within the video), shot changes (scene changes within the video) and shot labels (description of video events over time). Through experiments, we show that the API generates video and shot labels by processing only the first frame of every second of the video. Hence, an adversary can deceive the API to output only her desired video and shot labels by periodically inserting an image into the video at the rate of one frame per second. We also show that the pattern of shot changes returned by the API can be mostly recovered by an algorithm that compares the histograms of consecutive frames. Based on our equivalent model, we develop a method for slightly modifying the video frames, in order to deceive the API into generating our desired pattern of shot changes. We perform extensive experiments with different videos and show that our attacks are consistently successful across videos with different characteristics. At the end, we propose introducing randomness to video analysis algorithms as a countermeasure to our attacks.

研究动机与目标

• 研究在对抗性条件下自动视频分析算法的鲁棒性。
• 仅通过黑盒访问，开发针对视频分类与镜头切换检测系统的定向攻击。
• 证明细微的帧操纵可欺骗 Google Cloud Video Intelligence API，生成期望的标签与镜头切换模式。
• 揭示该 API 对确定性帧采样机制的依赖性，以及基于直方图的镜头切换检测机制。
• 提出采样随机化作为提升系统鲁棒性的对策。

提出的方法

• 通过在固定时间间隔（每秒一次）插入或修改帧，对视频分类与镜头切换检测实施定向攻击。
• 利用帧直方图分析逆向推导 API 的镜头切换检测逻辑。
• 采用黑盒攻击策略，无需了解模型架构或输出标签。
• 开发一种方法，通过扰动帧直方图，生成虚假镜头切换或消除真实镜头切换。
• 将 API 的行为建模为从每一秒中均匀采样帧，从而实现精确操控。
• 建议采用随机子采样作为对策，以降低帧选择的可预测性。

实验结果

研究问题

• RQ1 Google Cloud Video Intelligence API 在视频分类方面的定向对抗性攻击下有多脆弱？
• RQ2 攻击者能否仅通过修改少数帧，便在不被察觉的情况下操纵镜头切换输出？
• RQ3 该 API 所采用的帧采样机制的底层原理是什么？是否可被逆向推导？
• RQ4 通过极小的帧修改，基于直方图的镜头切换检测在多大程度上可被操纵？
• RQ5 随机子采样作为防御手段，对这类对抗性攻击的有效性如何？

主要发现

• 该 API 基于每一秒的第一个帧生成视频与镜头切换标签，因此易受图像插入攻击影响。
• 每秒仅插入一张图像，即可导致 API 将整个视频标记为该图像的标签。
• 通过分析连续帧之间的直方图差异，可重建镜头切换的模式。
• 该攻击在多种视频中成功操纵了镜头检测，包括真实世界视频、合成视频及网络下载内容。
• 系统忽略音频，允许非法内容嵌入而不影响视觉标签。
• 随机子采样显著降低了基于帧的攻击成功率，表明其作为有效防御机制的潜力。