[Paper Review] Image-based remapping of spatially-varying material appearance

Alejandro Sztrajman, Jaroslav Křivánek+2 more|arXiv (Cornell University)|Aug 20, 2018

Image Processing and 3D Reconstruction8 references4 citations

TL;DR

This paper presents an image-based method for automatic remapping of spatially-varying BRDFs between different rendering systems without access to shader implementations. By using rendered images under controlled illumination to compare appearance via SSIM, it enables robust parameter translation, and further introduces a parametric regression model that enables fast, real-time remapping of texture maps with superior extrapolation over existing methods.

ABSTRACT

BRDF models are ubiquitous tools for the representation of material appearance. However, there is now an astonishingly large number of different models in practical use. Both a lack of BRDF model standardisation across implementations found in different renderers, as well as the often semantically different capabilities of various models, have grown to be a major hindrance to the interchange of production assets between different rendering systems. Current attempts to solve this problem rely on manually finding visual similarities between models, or mathematical ones between their functional shapes, which requires access to the shader implementation, usually unavailable in commercial renderers. We present a method for automatic translation of material appearance between different BRDF models, which uses an image-based metric for appearance comparison, and that delegates the interaction with the model to the renderer. We analyse the performance of the method, both with respect to robustness and visual differences of the fits for multiple combinations of BRDF models. While it is effective for individual BRDFs, the computational cost does not scale well for spatially-varying BRDFs. Therefore, we further present a parametric regression scheme that approximates the shape of the transformation function and generates a reduced representation which evaluates instantly and without further interaction with the renderer. We present respective visual comparisons of the remapped SVBRDF models for commonly used renderers and shading models, and show that our approach is able to extrapolate transformed BRDF parameters better than other complex regression schemes.

Motivation & Objective

• Address the lack of standardization in BRDF models across commercial renderers, which hinders asset interchange.
• Overcome the limitations of manual or heuristic remapping methods that require shader access or oversimplify parameter mappings.
• Develop an automatic, renderer-agnostic method to translate material appearance between different BRDF models using only input parameters and output renderings.
• Enable efficient remapping of spatially-varying BRDFs (SVBRDFs) by reducing computational cost through parametric regression.
• Improve visual fidelity and robustness of remapped materials across diverse BRDFs and lighting conditions.

Proposed method

• Use image-based appearance comparison via structural similarity (SSIM) between renderings of the same material under identical lighting to quantify BRDF differences.
• Employ a headlight or non-frontal light setup to sample the BRDF response across multiple directions, ensuring sufficient illumination for optimization.
• Perform parameter optimization via numerical minimization of SSIM-based dissimilarity between reference and target BRDF renderings.
• Construct a parametric regression model using polynomial fitting to approximate the transformation function between BRDF parameters.
• Train the regression model on a set of sampled parameter pairs to generate a compact, fast-evaluating representation for real-time SVBRDF remapping.
• Enable chaining of transformations by composing regression models across multiple BRDF pairs for multi-step remapping workflows.

Experimental results

Research questions

• RQ1 Can an image-based metric like SSIM effectively capture perceptual differences between BRDFs without access to shader code?
• RQ2 How robust is the remapping process across diverse BRDF models, especially under varying lighting and material types?
• RQ3 Can a parametric regression model accurately approximate the BRDF parameter transformation with minimal interaction with the renderer?
• RQ4 Does the proposed regression method generalize better to unseen parameter ranges than existing approaches like Support Vector Regression?
• RQ5 How does lighting configuration during remapping affect the visual quality and stability of the resulting material appearance?

Key findings

• The image-based remapping method achieves high visual fidelity across multiple BRDF combinations, with SSIM-based optimization effectively minimizing perceptual differences.
• Non-frontal lighting (e.g., θₗ = 45°) reduces mean dissimilarity error by up to 10% compared to headlight illumination, improving robustness.
• The parametric regression model enables instant evaluation of remapped SVBRDF texture maps without further renderer interaction.
• The regression model demonstrates superior extrapolation performance compared to Support Vector Regression, especially in untrained parameter regions.
• Visual comparisons confirm that the method produces consistent and visually plausible results across complex materials like scratched gold, blue plastic, and bronze statues.
• The method supports chaining of transformations, enabling seamless transitions between multiple BRDF models in multi-renderer workflows.