[Paper Review] Fast and High Quality Highlight Removal from A Single Image

Dongsheng An, Jinli Suo+3 more|arXiv (Cornell University)|Dec 1, 2015

Color Science and Applications22 references3 citations

TL;DR

This paper proposes a fast, analytic highlight removal method for single natural images using an L₂-normalized dichromatic model and a pure diffuse pixel distribution rule (PDDR). By projecting pixels into illumination-orthogonal and parallel subspaces, it enables robust, adaptive clustering and specular-free diffuse component recovery without approximations, achieving high-quality results in under 0.02 seconds per 500×600 image.

ABSTRACT

Specular reflection exists widely in photography and causes the recorded color deviating from its true value, so fast and high quality highlight removal from a single nature image is of great importance. In spite of the progress in the past decades in highlight removal, achieving wide applicability to the large diversity of nature scenes is quite challenging. To handle this problem, we propose an analytic solution to highlight removal based on an L2 chromaticity definition and corresponding dichromatic model. Specifically, this paper derives a normalized dichromatic model for the pixels with identical diffuse color: a unit circle equation of projection coefficients in two subspaces that are orthogonal to and parallel with the illumination, respectively. In the former illumination orthogonal subspace, which is specular-free, we can conduct robust clustering with an explicit criterion to determine the cluster number adaptively. In the latter illumination parallel subspace, a property called pure diffuse pixels distribution rule (PDDR) helps map each specular-influenced pixel to its diffuse component. In terms of efficiency, the proposed approach involves few complex calculation, and thus can remove highlight from high resolution images fast. Experiments show that this method is of superior performance in various challenging cases.

Motivation & Objective

• Address the challenge of removing specular highlights from single natural images with wide applicability across diverse scenes.
• Overcome limitations of prior methods that rely on multiple images, polarization hardware, or complex optimization.
• Develop a method that is both efficient and accurate, avoiding strong assumptions or approximations.
• Enable robust separation of diffuse and specular components even in scenes with complex materials and varying illumination.
• Achieve real-time performance suitable for high-resolution images without sacrificing quality.

Proposed method

• Proposes an L₂-normalized dichromatic model to decompose pixel colors into diffuse and specular components using orthogonal and parallel subspaces relative to illumination.
• Uses the illumination-orthogonal subspace for robust, adaptive clustering of diffuse pixels via an explicit cluster number criterion.
• Applies the pure diffuse pixel distribution rule (PDDR) in the illumination-parallel subspace to map specular-affected pixels to their true diffuse components.
• Employs a down-sampling strategy to accelerate processing on high-resolution images while preserving material clustering accuracy.
• Avoids iterative or optimization-based computation, relying instead on analytical solutions and efficient K-means clustering.
• Uses chromaticity-based projection to ensure stability and avoid energy loss in gray or white regions.

Experimental results

Research questions

• RQ1 Can a single-image highlight removal method achieve both high quality and fast processing without relying on multi-image or hardware-based inputs?
• RQ2 How can diffuse and specular components be separated analytically without strong assumptions or approximations?
• RQ3 Can a robust, adaptive clustering method be designed to identify pure diffuse pixels across diverse materials in natural scenes?
• RQ4 To what extent can the PDDR principle improve the accuracy of diffuse component recovery in specular-affected regions?
• RQ5 How does the method perform on high-resolution images and in extreme cases such as all-specular or gray-colored scenes?

Key findings

• The method achieves highlight removal in under 0.02 seconds for 500×600 pixel images, outperforming C++-based methods from prior work.
• The approach produces superior results on challenging cases, including scenes with inter-reflection, strong specularity, and mixed materials.
• It successfully separates diffuse and specular components on outdoor scenes such as cars and plants, demonstrating wide applicability.
• The method preserves color fidelity and avoids artifacts in most natural scenes, though it may lose energy in pure specular or gray regions.
• In extreme cases like fully specular scenes, results are still affected by illumination uncertainty, confirming the intrinsic ill-posedness of such cases.
• The down-sampling strategy enables efficient processing at high resolutions without significant loss in accuracy.