[Paper Review] Robust Watermarking for Video Forgery Detection with Improved Imperceptibility and Robustness

Yangming Zhou, Qichao Ying+4 more|arXiv (Cornell University)|Jul 7, 2022

Digital Media Forensic Detection4 citations

TL;DR

This paper proposes RWVFD, an end-to-end 3D-UNet-based video watermarking framework that jointly embeds imperceptible watermarks and enables robust tamper localization. By simulating video compression and post-processing attacks through ensembled distortions (e.g., JPEG, blurring), the method achieves high imperceptibility (PSNR: 37.78 dB, SSIM: 0.987) and strong robustness, outperforming passive forensics in detecting tampered regions under H.264 compression and frame manipulation.

ABSTRACT

Videos are prone to tampering attacks that alter the meaning and deceive the audience. Previous video forgery detection schemes find tiny clues to locate the tampered areas. However, attackers can successfully evade supervision by destroying such clues using video compression or blurring. This paper proposes a video watermarking network for tampering localization. We jointly train a 3D-UNet-based watermark embedding network and a decoder that predicts the tampering mask. The perturbation made by watermark embedding is close to imperceptible. Considering that there is no off-the-shelf differentiable video codec simulator, we propose to mimic video compression by ensembling simulation results of other typical attacks, e.g., JPEG compression and blurring, as an approximation. Experimental results demonstrate that our method generates watermarked videos with good imperceptibility and robustly and accurately locates tampered areas within the attacked version.

Motivation & Objective

• Address the challenge of detecting video tampering in real-world scenarios where post-processing attacks like H.264 compression destroy traditional forensic clues.
• Develop a video watermarking system that enables accurate localization of tampered regions even after aggressive video processing.
• Improve robustness against common video attacks by simulating codec distortions without a differentiable video codec, using ensembled attacks as a proxy.
• Achieve high imperceptibility in watermarked videos to ensure visual quality is preserved for public sharing on social media.
• Provide an active forensics solution that embeds detectable signals into videos to enable reliable tamper detection and localization.

Proposed method

• Employ a 3D-UNet-based encoder to embed a watermark into video frames in a spatiotemporal manner, ensuring consistency across time and space.
• Train a decoder network end-to-end to predict a tampering mask from the watermarked video, enabling localization of modified regions.
• Design a custom attack simulation layer that ensembles typical video distortions—such as JPEG compression and blurring—to approximate real-world video codec effects.
• Use the simulated attack layer during adversarial training to improve the watermark’s resilience to common post-processing attacks.
• Train the entire system jointly using a loss function that balances watermark imperceptibility (via PSNR/SSIM) and tamper localization accuracy.
• Leverage the absence of a differentiable video codec by approximating its behavior through a combination of known, non-differentiable distortions.

Figure 1: Sketch of the pipeline of RWVMD. A 3D-UNet based encoder is used to embed the watermark information into the video. Video processing attacks in social media are simulated by the attack layer. Finally, the decoder predicts the tampering mask from the watermarked video.

Experimental results

Research questions

• RQ1 Can a deep learning-based watermarking system achieve high imperceptibility while maintaining robustness against real-world video compression and post-processing?
• RQ2 How effective is an ensembled attack simulation (e.g., JPEG, blurring) in approximating the behavior of a real video codec for adversarial training?
• RQ3 Can the proposed watermarking framework outperform passive forensics methods in detecting tampered regions under video compression and frame manipulation?
• RQ4 To what extent does the 3D-UNet architecture improve spatiotemporal consistency and robustness compared to 2D-based watermarking?
• RQ5 Does the inclusion of a dedicated codec simulator significantly enhance the localization accuracy of tampered regions under H.264 compression?

Key findings

• The proposed RWVFD framework achieves high imperceptibility with an average PSNR of 37.78 dB and SSIM of 0.987 on the YouTube-VOS dataset.
• Under H.264 compression (CRF=29), the method maintains an F1-score of 0.87 for tamper localization, demonstrating strong robustness even under high compression.
• The ablation study confirms that removing the 3D-UNet leads to a significant drop in performance under codec attacks, with F1-score dropping to 0.56 at CRF=29.
• The codec simulator is critical: removing it reduces the F1-score to 0.80 at CRF=23, showing its necessity for robustness against compression.
• RWVFD outperforms passive forensics methods like MVSS and Xception, which fail to detect tampered regions under codec attacks due to distribution shift.
• The method achieves high precision and recall (F1-score ≥ 0.95) in frame deletion and frame rate change attacks, indicating resilience to temporal manipulations.

Figure 2: Architectures of the encoder and decoder, which are based on the 3D-Unet [ 15 ] .