[Paper Review] StegaNeRF: Embedding Invisible Information within Neural Radiance Fields

Chenxin Li, Brandon Y. Feng+3 more|arXiv (Cornell University)|Dec 3, 2022

Generative Adversarial Networks and Image Synthesis4 citations

TL;DR

StegaNeRF proposes a novel steganographic framework for embedding customizable, imperceptible, and recoverable information directly into Neural Radiance Fields (NeRF) during training. By using adaptive gradient masking to steer hidden data toward low-impact weights and a joint optimization with a detector network, StegaNeRF achieves high-fidelity hidden information recovery without degrading rendering quality, as validated across diverse 3D scenes and perturbations.

ABSTRACT

Recent advances in neural rendering imply a future of widespread visual data distributions through sharing NeRF model weights. However, while common visual data (images and videos) have standard approaches to embed ownership or copyright information explicitly or subtly, the problem remains unexplored for the emerging NeRF format. We present StegaNeRF, a method for steganographic information embedding in NeRF renderings. We design an optimization framework allowing accurate hidden information extractions from images rendered by NeRF, while preserving its original visual quality. We perform experimental evaluations of our method under several potential deployment scenarios, and we further discuss the insights discovered through our analysis. StegaNeRF signifies an initial exploration into the novel problem of instilling customizable, imperceptible, and recoverable information to NeRF renderings, with minimal impact to rendered images. Project page: https://xggnet.github.io/StegaNeRF/.

Motivation & Objective

• Address the lack of steganographic techniques for NeRF, a growing 3D content format, despite their widespread use in sharing and distribution.
• Enable the embedding of customizable, imperceptible, and recoverable information in NeRF renderings across arbitrary viewpoints.
• Preserve high-fidelity NeRF rendering quality while ensuring robust extraction of hidden signals through optimized weight allocation.
• Explore practical applications such as ownership identification, multi-source identity tracking, and multi-modal steganography in 3D scenes.
• Overcome the limitations of 2D steganography applied to NeRF training images, which fails due to information loss during NeRF optimization.

Proposed method

• Train a standard NeRF model first to initialize weights (θ₀), then fine-tune with a steganographic objective.
• Introduce an adaptive gradient masking strategy that identifies low-impact weights and redirects hidden information gradients toward them to minimize visual distortion.
• Jointly optimize a detector network Fψ to extract hidden information from NeRF renderings, while ensuring no false positives on original NeRF renderings.
• Employ Classifier Guided Recovery to improve decoding accuracy by conditioning the detector on a classifier that distinguishes between stego and clean renderings.
• Use a soft masking mechanism that dynamically adjusts gradient updates based on the importance of each weight, balancing steganography and rendering fidelity.
• Apply the framework end-to-end on diverse 3D scenes with varying camera layouts and scene complexity to evaluate robustness and generalization.

Figure 1 : We introduce the new problem of NeRF steganography: hiding information in NeRF renderings. Our proposed framework, StegaNeRF, can embed and recover customized hidden information while preserving the original NeRF rendering quality.

Experimental results

Research questions

• RQ1 Can we embed and recover hidden information in NeRF renderings while preserving visual quality, even when rendered from novel viewpoints?
• RQ2 Can NeRF steganography preserve the identities of multiple data contributors in collaborative 3D scene reconstruction?
• RQ3 Can we support multiple modalities of hidden signals (e.g., text, images) within NeRF renderings, beyond simple message strings?
• RQ4 Why do traditional 2D steganography methods fail when applied to NeRF training images, and how can we overcome this?
• RQ5 How robust is the hidden information recovery under common image distortions such as JPEG compression and Gaussian blur?

Key findings

• StegaNeRF achieves a PSNR of 28.23 dB and SSIM of 0.8593 on standard NeRF renderings, with a hidden image recovery accuracy of 94.44% under the No Soft Masking setting.
• The proposed adaptive gradient masking strategy significantly improves the balance between steganographic performance and rendering quality, outperforming baselines without masking.
• Classifier Guided Recovery increases hidden signal recovery accuracy by conditioning the detector on a classifier, reducing false positives and improving robustness.
• StegaNeRF maintains high recovery accuracy (SSIM > 0.85) even under JPEG compression (up to 90%) and Gaussian blur, demonstrating strong robustness to common image distortions.
• The framework successfully recovers hidden information across diverse 3D scenes with varying camera layouts and scene characteristics, confirming generalization capability.
• Ablation studies confirm that removing key components—such as gradient masking or classifier guidance—leads to significant drops in recovery accuracy and rendering quality, validating their necessity.

Figure 2 : StegaNeRF training overview. At the first stage, we optimize $\bm{\theta_{0}}$ with standard NeRF training. At the second stage, we initialize $\bm{\theta}$ with $\bm{\theta_{0}}$ and optimize for the steganography objectives. We train the detector $\bm{F_{\psi}}$ to recover hidden inform