[Paper Review] Prediction Model for Semitransparent Watercolor Pigment Mixtures Using Deep Learning with a Dataset of Transmittance and Reflectance

Mei-Yun Chen, Yabo Huang+2 more|arXiv (Cornell University)|Mar 30, 2019

Color Science and Applications17 references4 citations

TL;DR

This paper proposes a deep learning-based prediction model for semitransparent watercolor pigment mixtures using a custom dataset of transmittance and reflectance measurements. By training a deep neural network on 13 primary pigments and 780 mixtures, the model enables a Smart Palette system that provides accurate mixing recipes, achieving 83% of test cases with ΔE*ab < 5, significantly outperforming traditional methods like intuition or the Itten color wheel.

ABSTRACT

Learning color mixing is difficult for novice painters. In order to support novice painters in learning color mixing, we propose a prediction model for semitransparent pigment mixtures and use its prediction results to create a Smart Palette system. Such a system is constructed by first building a watercolor dataset with two types of color mixing data, indicated by transmittance and reflectance: incrementation of the same primary pigment and a mixture of two different pigments. Next, we apply the collected data to a deep neural network to train a model for predicting the results of semitransparent pigment mixtures. Finally, we constructed a Smart Palette that provides easily-followable instructions on mixing a target color with two primary pigments in real life: when users pick a pixel, an RGB color, from an image, the system returns its mixing recipe which indicates the two primary pigments being used and their quantities.

Motivation & Objective

• To address the difficulty novices face in learning watercolor color mixing, especially with limited guidance from traditional tools like the Itten color wheel.
• To develop a data-driven prediction model that accounts for both pigment ratio and transmittance, crucial for highly transparent media like watercolor.
• To construct a practical Smart Palette system that provides real-time, accurate mixing recipes based on target RGB colors.
• To evaluate the model’s performance against ground truth and compare its effectiveness with traditional mixing methods using user studies.

Proposed method

• A custom dataset, NTU WPSM, was collected using a spectrometer (OTO SD1220) measuring transmittance and reflectance of 13 primary pigments across 12 quantities and 780 two-pigment mixtures.
• The dataset includes two types: Data Type I (incrementation of the same pigment) and Data Type M (mixing two different pigments at 1:1, 1:2, and 2:1 ratios).
• A deep neural network (DNN) implemented with TensorFlow was trained to predict the reflectance of pigment mixtures based on input pigment types and quantities.
• The trained model generates a lookup table for real-time color matching, enabling the Smart Palette system to output mixing recipes for any target RGB color.
• The system was evaluated via user studies comparing mixing accuracy using intuition, the Itten wheel, and the Smart Palette, with color differences measured using ΔE*ab.

Experimental results

Research questions

• RQ1 Can a deep learning model accurately predict the reflectance of semitransparent watercolor pigment mixtures using transmittance and reflectance data?
• RQ2 How does the performance of the Smart Palette system compare to traditional methods like intuition and the Itten color wheel in terms of color matching accuracy?
• RQ3 To what extent does incorporating transmittance improve prediction accuracy over models that only consider pigment ratio?
• RQ4 Can the proposed dataset and model be generalized to other translucent media such as acrylics or inks?

Key findings

• The model achieved 83% of test cases with a color difference (ΔE*ab) below 5, indicating that the predicted colors were indistinguishable from ground truth to the average observer.
• User evaluation showed a statistically significant improvement in color matching accuracy when using the Smart Palette compared to intuition or the Itten color wheel, as confirmed by t-tests on ΔE*ab values.
• The Smart Palette system produced mixing recipes that were significantly closer to target colors than those derived from traditional methods, demonstrating its practical utility for novices.
• The NTU WPSM dataset, containing transmittance and reflectance measurements of 13 pigments and 780 mixtures, is publicly available for future research and model development.
• The interdisciplinary approach—integrating physics of light, observer perception (CMF), and device-specific display factors—enables transferable color matching across devices and media.