[Paper Review] PREF: Phasorial Embedding Fields for Compact Neural Representations

Binbin Huang, Xinhao Yan+3 more|arXiv (Cornell University)|May 26, 2022

Model Reduction and Neural Networks4 citations

TL;DR

PREF introduces a compact, efficient frequency-based neural representation using a dilated 3D phasor volume that uniformly distributes frequencies across a 2D plane and dilates along a 1D axis, accelerating Fourier mapping via a hybrid FFT and local interpolation method. It achieves state-of-the-art performance in 2D image generalization, 3D SDF regression, and 5D neural radiance field reconstruction with significantly reduced memory and improved robustness compared to prior frequency-based and hybrid representations.

ABSTRACT

We present an efficient frequency-based neural representation termed PREF: a shallow MLP augmented with a phasor volume that covers significant border spectra than previous Fourier feature mapping or Positional Encoding. At the core is our compact 3D phasor volume where frequencies distribute uniformly along a 2D plane and dilate along a 1D axis. To this end, we develop a tailored and efficient Fourier transform that combines both Fast Fourier transform and local interpolation to accelerate naïve Fourier mapping. We also introduce a Parsvel regularizer that stables frequency-based learning. In these ways, Our PREF reduces the costly MLP in the frequency-based representation, thereby significantly closing the efficiency gap between it and other hybrid representations, and improving its interpretability. Comprehensive experiments demonstrate that our PREF is able to capture high-frequency details while remaining compact and robust, including 2D image generalization, 3D signed distance function regression and 5D neural radiance field reconstruction.

Motivation & Objective

• To address the inefficiency and high memory cost of frequency-based neural representations in capturing high-frequency details.
• To reduce reliance on large, costly MLPs in frequency-encoded models by shifting capacity to a compact phasor volume.
• To improve interpretability and controllability of neural fields by enabling direct manipulation of frequency components.
• To close the performance and efficiency gap between frequency-based and hybrid neural representations (e.g., dense grids, octrees).
• To stabilize training of frequency-based models through a novel Parsvel regularizer analogous to total variation regularization.

Proposed method

• Proposes a 3D phasor volume with uniform 2D frequency distribution and 1D dilation to cover full spectral bandwidth efficiently.
• Introduces a hybrid Fourier transform combining Fast Fourier Transform (FFT) and local interpolation to accelerate naïve Fourier mapping.
• Designs a Parsvel regularizer tailored for phasor layers to stabilize frequency-based learning, analogous to total variation regularization.
• Replaces the black-box MLP in traditional frequency encodings with a shallow network, shifting most capacity to the phasor volume for improved interpretability.
• Employs a compact, structured frequency layout that enables high-frequency detail recovery with minimal parameter count.
• Pre-computes and stores transformed features to ensure fast inference, despite additional training cost from FFT.

Experimental results

Research questions

• RQ1 Can a compact phasor volume with non-uniform frequency distribution outperform standard uniform Fourier encoding in capturing high-frequency details?
• RQ2 How can the computational cost of Fourier mapping in frequency-based neural representations be reduced without sacrificing spectral coverage?
• RQ3 To what extent can the phasor volume replace the MLP in neural representations while maintaining or improving performance?
• RQ4 Does a tailored regularizer for phasor layers improve training stability and generalization in frequency-based models?
• RQ5 How does the interpretability of frequency-based representations compare to hybrid representations when enabling direct editing of frequency components?

Key findings

• PREF achieves state-of-the-art performance in 2D image generalization, with a PSNR of 24.113 ± 3.464 on natural images and 28.329 ± 2.373 on text images, outperforming PE, SIREN, and NGP.
• In 3D signed distance function (SDF) regression, PREF achieves a PSNR of 31.86, slightly surpassing dense grid baselines (31.02) despite using only 545 KB memory.
• For 5D neural radiance field reconstruction, PREF produces high-quality novel view synthesis with reduced memory and improved visual fidelity compared to NeRF and NGP.
• The Parsvel regularizer stabilizes training of frequency-based models, enabling consistent convergence and robustness across diverse tasks.
• The phasor volume enables direct, interpretable editing of neural fields—e.g., level-of-detail filtering—demonstrating enhanced controllability over black-box MLPs.
• PREF reduces memory footprint to 545 KB while maintaining competitive performance, significantly outperforming PE (1318 KB) and SIREN (777 KB), and approaching NGP (696 KB) with better generalization.