[Paper Review] Personalized Image Aesthetics Assessment with Rich Attributes

Yuzhe Yang, Liwu Xu+5 more|arXiv (Cornell University)|Mar 31, 2022

Olfactory and Sensory Function Studies4 citations

TL;DR

This paper introduces PARA, a large-scale personalized image aesthetics assessment database with 31,220 images annotated by 438 subjects, featuring 9 image-oriented objective attributes and 4 human-oriented subjective attributes. It proposes a conditional PIAA model that uses subject characteristics as prior knowledge, demonstrating improved performance over baseline models, thus enabling more accurate modeling of personalized aesthetic preferences through rich, diverse annotations.

ABSTRACT

Personalized image aesthetics assessment (PIAA) is challenging due to its highly subjective nature. People's aesthetic tastes depend on diversified factors, including image characteristics and subject characters. The existing PIAA databases are limited in terms of annotation diversity, especially the subject aspect, which can no longer meet the increasing demands of PIAA research. To solve the dilemma, we conduct so far, the most comprehensive subjective study of personalized image aesthetics and introduce a new Personalized image Aesthetics database with Rich Attributes (PARA), which consists of 31,220 images with annotations by 438 subjects. PARA features wealthy annotations, including 9 image-oriented objective attributes and 4 human-oriented subjective attributes. In addition, desensitized subject information, such as personality traits, is also provided to support study of PIAA and user portraits. A comprehensive analysis of the annotation data is provided and statistic study indicates that the aesthetic preferences can be mirrored by proposed subjective attributes. We also propose a conditional PIAA model by utilizing subject information as conditional prior. Experimental results indicate that the conditional PIAA model can outperform the control group, which is also the first attempt to demonstrate how image aesthetics and subject characters interact to produce the intricate personalized tastes on image aesthetics. We believe the database and the associated analysis would be useful for conducting next-generation PIAA study. The project page of PARA can be found at: https://cv-datasets.institutecv.com/#/data-sets.

Motivation & Objective

• To address the limited annotation diversity in existing personalized image aesthetics assessment (PIAA) databases, especially in capturing subject-level subjective attributes.
• To establish a comprehensive, large-scale PIAA database that captures both image-level objective attributes and human-level subjective attributes to reflect individualized aesthetic preferences.
• To enable deeper analysis of the relationship between user characteristics (e.g., personality, experience) and aesthetic judgments through desensitized subject information.
• To develop and validate a conditional PIAA model that leverages subject attributes as prior knowledge to improve personalized aesthetic prediction.
• To provide a benchmark and foundational dataset for next-generation PIAA research, supporting diverse modeling approaches.

Proposed method

• The authors collect 31,220 images annotated by 438 subjects, with each image receiving an average of 25.87 annotations across 13 dimensions: 9 objective image attributes (e.g., composition, color, light) and 4 subjective human attributes (e.g., content preference, emotion, willingness to share).
• Desensitized subject information, including age, gender, education, personality traits, and artistic/photographic experience, is collected to support user portrait and preference modeling.
• A conditional PIAA model is proposed that incorporates subject attributes as conditional priors by concatenating them with the final-layer features of a deep neural network backbone (e.g., ResNet-50).
• The model is fine-tuned on 10-shot and 100-shot support sets per subject, with performance evaluated on query sets using SROCC and PLCC metrics.
• A 10-fold cross-validation strategy is applied across 40 randomly selected test subjects, with results averaged over 10 random data splits to ensure robustness.
• The proposed method compares conditional PIAA (with subject priors) against unconditional PIAA (without priors), using the same backbone and training protocol.

Experimental results

Research questions

• RQ1 Can human-oriented subjective attributes such as content preference, emotion, and willingness to share effectively reflect personalized aesthetic preferences?
• RQ2 To what extent do subject characteristics like personality traits, artistic experience, and photographic experience influence image aesthetic judgments?
• RQ3 Does incorporating subject attributes as conditional priors in deep learning models improve personalized image aesthetics prediction over baseline models?
• RQ4 How does the amount of personalized data (10-shot vs. 100-shot) affect the performance of fine-tuned PIAA models?
• RQ5 Can the proposed PARA database serve as a reliable benchmark for evaluating next-generation PIAA models?

Key findings

• The conditional PIAA model using personality traits as prior achieves a PLCC of 0.7509 and SROCC of 0.7384, outperforming the unconditional baseline (PLCC: 0.7419, SROCC: 0.7329) on ResNet-50.
• The model with personality trait conditioning achieves a PLCC of 0.7509 and SROCC of 0.7384, showing a 0.009 increase in PLCC and 0.0055 in SROCC compared to the unconditional baseline.
• The conditional model using artistic experience as prior achieves a PLCC of 0.7447 and SROCC of 0.7326, demonstrating consistent improvement over the unconditional group.
• The backbone ablation shows that Swin-Tiny and Swin-Small achieve the best GIAA performance with PLCC of 0.9331 and 0.9355, respectively, on the PARA benchmark.
• Fine-tuning on 100-shot data consistently improves performance over 10-shot data, suggesting that more personalized data enhances model generalization.
• Statistical analysis confirms that subjective attributes such as emotion, content preference, and willingness to share are significantly correlated with aesthetic judgments, validating their use as preference mirrors.