[论文解读] Salient Object Detection by LTP Texture Characterization on Opposing Color Pairs under SLICO Superpixel Constraint

Didier Ndayikengurukiye|arXiv (Cornell University)|Apr 13, 2022

Visual Attention and Saliency Detection参考文献 70被引用 9

一句话总结

该论文提出了一种新颖的、近乎无参数的显著性检测模型，通过在SLICO超像素内对相反颜色对应用局部三值模式（LTP），实现了颜色与纹理的融合。通过使用基于FastMap的差异性度量方法，将来自多种颜色空间（RGB、HSL、LUV、CMY）的显著性图进行融合，该方法在ECSSD数据集上实现了最先进性能，平均Fβ得分为0.729，MAE为0.257。

ABSTRACT

The effortless detection of salient objects by humans has been the subject of research in several fields, including computer vision, as it has many applications. However, salient object detection remains a challenge for many computer models dealing with color and textured images. Most of them process color and texture separately and therefore implicitly consider them as independent features which is not the case in reality. Herein, we propose a novel and efficient strategy, through a simple model, almost without internal parameters, which generates a robust saliency map for a natural image. This strategy consists of integrating color information into local textural patterns to characterize a color micro-texture. It is the simple, yet powerful LTP (Local Ternary Patterns) texture descriptor applied to opposing color pairs of a color space that allows us to achieve this end. Each color micro-texture is represented by a vector whose components are from a superpixel obtained by the SLICO (Simple Linear Iterative Clustering with zero parameter) algorithm, which is simple, fast and exhibits state-of-the-art boundary adherence. The degree of dissimilarity between each pair of color micro-textures is computed by the FastMap method, a fast version of MDS (Multi-dimensional Scaling) that considers the color micro-textures’ non-linearity while preserving their distances. These degrees of dissimilarity give us an intermediate saliency map for each RGB (Red–Green–Blue), HSL (Hue–Saturation–Luminance), LUV (L for luminance, U and V represent chromaticity values) and CMY (Cyan–Magenta–Yellow) color space. The final saliency map is their combination to take advantage of the strength of each of them. The MAE (Mean Absolute Error), MSE (Mean Squared Error) and Fβ measures of our saliency maps, on the five most used datasets show that our model outperformed several state-of-the-art models. Being simple and efficient, our model could be combined with classic models using color contrast for a better performance.

研究动机与目标

• 开发一种简单、近乎无参数的显著性检测模型，以有效结合颜色与纹理特征。
• 解决在具有丰富纹理和颜色的复杂自然图像中显著目标检测的挑战。
• 通过将颜色信息直接整合到局部纹理描述符中，而非分别处理，从而提升显著性图的质量。
• 通过最小配置、高效的流水线，在多种图像数据集上实现稳健性能。

提出的方法

• 在RGB、HSL、LUV和CMY颜色空间中，对相反颜色对应用局部三值模式（LTP）纹理描述符，以生成颜色-微纹理特征。
• 使用SLICO超像素（无参数）定义空间区域，用于纹理与颜色特征的聚合，确保强边界保持能力。
• 使用FastMap（一种快速MDS变体）计算跨微纹理的差异性，以保留特征向量之间的非线性距离。
• 基于差异性评分，为每个颜色空间生成独立的显著性图，随后将其融合为最终的显著性图。
• 采用单一内部参数（超像素数量），其对性能影响极小，如在50、100和200个超像素设置下所示。
• 使用平均绝对误差（MAE）和Fβ度量在ECSSD与MSRA10K数据集上进行定量评估。

实验结果

研究问题

• RQ1 与分别处理颜色和纹理相比，将颜色直接整合到局部纹理描述符中是否能提升显著性检测性能？
• RQ2 在LTP中使用相反颜色对在多大程度上增强了显著性检测的纹理表征能力？
• RQ3 SLICO超像素方法在最小参数调优下，多大程度上支持鲁棒且高效的特征提取？
• RQ4 使用基于FastMap的差异性度量方法融合多个颜色空间的显著性图，是否优于单独处理每个颜色空间？
• RQ5 该模型在极少参数调整下，对不同复杂度的图像和数据集是否表现出高度稳定性？

主要发现

• 所提模型在ECSSD数据集上实现了0.729的平均Fβ得分和0.257的MAE，优于29种最先进模型中的18种。
• 该模型表现出更优的稳定性，其MAE的标准差为0.071，显著低于HS模型（0.108）和CHS模型（0.117）。
• 与HS和CHS模型相比，该模型在ECSSD数据集更复杂的后半部分（图像500–1000）的性能下降程度更小。
• 在50、100和200个超像素设置下，模型性能几乎保持不变，表明对唯一内部参数具有高度鲁棒性。
• 在LTP中使用相反颜色对可增强Fβ和精确率-召回率指标的个体贡献，尤其在RGB空间中表现更优。
• 将四个颜色空间（RGB、HSL、LUV、CMY）的结果进行融合，可生成比单独使用任一颜色空间更鲁棒的最终显著性图。