[Paper Review] CopyRNeRF: Protecting the CopyRight of Neural Radiance Fields

Ziyuan Luo, Qing Guo+3 more|arXiv (Cornell University)|Jul 21, 2023

Generative Adversarial Networks and Image SynthesisComputer Science3 citations

TL;DR

This paper proposes CopyRNeRF, a method to protect Neural Radiance Fields (NeRF) copyright by embedding watermarks directly into the model's color representation rather than rendered images. By using a distortion-resistant rendering scheme during training, the approach ensures high visual quality and robust message extraction across various distortions and rendering strategies, achieving 100% bit accuracy and PSNR of 32.68 with minimal visual distortion.

ABSTRACT

Neural Radiance Fields (NeRF) have the potential to be a major representation of media. Since training a NeRF has never been an easy task, the protection of its model copyright should be a priority. In this paper, by analyzing the pros and cons of possible copyright protection solutions, we propose to protect the copyright of NeRF models by replacing the original color representation in NeRF with a watermarked color representation. Then, a distortion-resistant rendering scheme is designed to guarantee robust message extraction in 2D renderings of NeRF. Our proposed method can directly protect the copyright of NeRF models while maintaining high rendering quality and bit accuracy when compared among optional solutions.

Motivation & Objective

• To address the lack of direct NeRF model copyright protection despite the high cost of training.
• To overcome limitations of watermarking rendered images, which fail to protect the core model and may introduce visible artifacts.
• To ensure that embedded copyright messages remain robust across different rendering strategies and 2D distortions.
• To maintain high visual fidelity (PSNR, SSIM) while embedding messages invisibly in the model's color representation.
• To develop a method that enables reliable ownership verification through message extraction from any 2D rendering of the protected NeRF model.

Proposed method

• Embedding copyright messages into a watermarked color representation derived from a subset of NeRF model weights, preserving the original geometry.
• Designing a distortion-resistant rendering process that applies synthetic distortions (e.g., blur, noise, rotation) during model optimization to enhance robustness.
• Using a CNN-based message extractor to decode embedded messages from 2D renderings of the NeRF model.
• Applying a random sampling strategy during training to improve robustness against varying inference sampling methods.
• Maintaining the original NeRF geometry and only modifying the color representation to ensure visual invisibility.
• Integrating spatial information into the watermarked color representation to ensure consistent message embedding across different viewpoints.

Experimental results

Research questions

• RQ1 Can a watermark be embedded directly into the NeRF model’s color representation to protect its intellectual property without degrading rendering quality?
• RQ2 How can watermark robustness be maintained across diverse 2D distortions such as blurring, noise, and rotation during rendering?
• RQ3 Can the watermark survive different rendering strategies, including varying sampling methods, during inference?
• RQ4 Does embedding messages in the color representation outperform watermarking rendered images in terms of invisibility and robustness?
• RQ5 Can the method ensure consistent message extraction regardless of viewpoint or rendering configuration?

Key findings

• The proposed method achieves 100% bit accuracy in message extraction, even when tested under distortion-resistant rendering strategies.
• The method maintains high visual quality with a PSNR of 32.68 and SSIM of 0.948, indicating minimal perceptual distortion.
• Robustness is validated across different sampling strategies during inference, with bit accuracy remaining at 100%.
• Removing the message feature field or color feature field module leads to significant performance drops, confirming their necessity.
• The method outperforms direct 2D watermarking (e.g., NeRF+HiDDeN) by avoiding visible artifacts, especially in low-geometry regions.
• Even under model weight corruption (e.g., Gaussian noise with std=0.01), bit accuracy remains high at 93.97%, indicating resilience to some adversarial attacks.