[Paper Review] Photo-Guided Exploration of Volume Data Features

Mohammad Raji, Alok Hota+3 more|arXiv (Cornell University)|Oct 18, 2017

Computer Graphics and Visualization Techniques21 references3 citations

TL;DR

This paper proposes a photo-guided, deep learning–enhanced evolutionary optimization method that automatically discovers volume rendering parameters to match a target image, such as a satellite photograph of a hurricane. By using a Siamese neural network to measure image similarity and a genetic algorithm to iteratively refine transfer functions, the system produces visually accurate renderings of scientific simulation data in under 7 minutes on a supercomputer, enabling automated visual feature discovery without manual tuning.

ABSTRACT

In this work, we pose the question of whether, by considering qualitative information such as a sample target image as input, one can produce a rendered image of scientific data that is similar to the target. The algorithm resulting from our research allows one to ask the question of whether features like those in the target image exists in a given dataset. In that way, our method is one of imagery query or reverse engineering, as opposed to manual parameter tweaking of the full visualization pipeline. For target images, we can use real-world photographs of physical phenomena. Our method leverages deep neural networks and evolutionary optimization. Using a trained similarity function that measures the difference between renderings of a phenomenon and real-world photographs, our method optimizes rendering parameters. We demonstrate the efficacy of our method using a superstorm simulation dataset and images found online. We also discuss a parallel implementation of our method, which was run on NCSA's Blue Waters.

Motivation & Objective

• To automate the search for visual features in scientific volume datasets using a target photograph as input.
• To reduce reliance on expert knowledge and manual parameter tuning in scientific visualization pipelines.
• To enable users to query whether features in real-world images (e.g., hurricanes) exist within simulation data.
• To develop a scalable, reproducible method for rendering parameter optimization using image similarity metrics.
• To demonstrate the feasibility of reverse engineering visualization parameters via qualitative input.

Proposed method

• A Siamese convolutional neural network is trained to compute perceptual similarity between rendered volume images and target photographs.
• A genetic algorithm evolves a population of transfer function parameters to minimize the deep metric loss between rendered images and the target.
• The optimization process uses crossover and mutation operations on a population of rendering configurations to explore the parameter space efficiently.
• The system leverages parallel computing on the Blue Waters supercomputer, using 60 compute nodes for accelerated execution.
• The visualization pipeline is controlled entirely through VisIt’s Python API, ensuring reproducibility and integration with existing scientific workflows.
• The method preserves the original data and only modifies transfer functions, avoiding data alteration.

Experimental results

Research questions

• RQ1 Can a deep learning-based image similarity metric effectively guide the optimization of volume rendering parameters toward a target photograph?
• RQ2 To what extent can a genetic algorithm efficiently explore the high-dimensional rendering parameter space to find visually similar results?
• RQ3 Can this approach automate the process of identifying visual features in scientific simulations that match real-world observations?
• RQ4 How does population seeding affect convergence speed and solution quality in the optimization process?
• RQ5 What are the impacts of rendering method and camera viewpoint on the final output quality and consistency?

Key findings

• The system successfully generated volume-rendered images that visually match target hurricane photographs from Google Images using only the simulation data and the target image as input.
• With population seeding, the optimization converged to a minimum cost of 0.012 within 20 generations, significantly outperforming non-seeded runs.
• Without population seeding, the best cost achieved was 0.02, indicating that seeding improves convergence and solution quality.
• The method completed within 6–7 minutes using 60 compute nodes on the Blue Waters supercomputer, demonstrating scalability.
• Ray casting and 3D texturing produced different results, showing that the rendering method must be consistent between optimization and final rendering.
• The approach enables automated, reproducible rendering parameter selection without modifying the original data, supporting visual search in scientific datasets.