[Paper Review] Playground v2.5: Three Insights towards Enhancing Aesthetic Quality in Text-to-Image Generation

Daiqing Li, Aleks Kamko+4 more|arXiv (Cornell University)|Feb 27, 2024

Virtual Reality Applications and Impacts4 citations

TL;DR

Playground v2.5 introduces three key insights to enhance aesthetic quality in text-to-image diffusion models: optimizing the noise schedule for improved realism and visual fidelity, using a balanced bucketed dataset to support diverse aspect ratios, and aligning outputs with human preferences for better human-centric details. The model achieves state-of-the-art performance, outperforming SDXL, Playground v2, DALL·E 3, and Midjourney v5.2 in user preference and FID metrics, with an overall FID of 4.48 on the MJHQ-30K benchmark.

ABSTRACT

In this work, we share three insights for achieving state-of-the-art aesthetic quality in text-to-image generative models. We focus on three critical aspects for model improvement: enhancing color and contrast, improving generation across multiple aspect ratios, and improving human-centric fine details. First, we delve into the significance of the noise schedule in training a diffusion model, demonstrating its profound impact on realism and visual fidelity. Second, we address the challenge of accommodating various aspect ratios in image generation, emphasizing the importance of preparing a balanced bucketed dataset. Lastly, we investigate the crucial role of aligning model outputs with human preferences, ensuring that generated images resonate with human perceptual expectations. Through extensive analysis and experiments, Playground v2.5 demonstrates state-of-the-art performance in terms of aesthetic quality under various conditions and aspect ratios, outperforming both widely-used open-source models like SDXL and Playground v2, and closed-source commercial systems such as DALLE 3 and Midjourney v5.2. Our model is open-source, and we hope the development of Playground v2.5 provides valuable guidelines for researchers aiming to elevate the aesthetic quality of diffusion-based image generation models.

Motivation & Objective

• To improve the aesthetic quality of text-to-image diffusion models beyond existing open-source and commercial systems.
• To address limitations in color and contrast fidelity, especially in generating pure backgrounds and vibrant visuals.
• To enhance model performance across diverse aspect ratios through structured dataset bucketing.
• To improve generation of human-centric details by aligning model outputs with human perceptual preferences.
• To establish a new benchmark (MJHQ-30K) for automatic evaluation of aesthetic quality in text-to-image generation.

Proposed method

• Optimizing the noise schedule during diffusion model training to enhance realism and visual fidelity.
• Creating a balanced, bucketed dataset that includes diverse aspect ratios to improve generalization across image dimensions.
• Applying human preference alignment techniques to fine-tune the model on prompts emphasizing realistic human features and fine details.
• Introducing the MJHQ-30K benchmark, a high-quality dataset of 30K images from Midjourney v5.2, curated with aesthetic and CLIP scores for evaluation.
• Using Fréchet Inception Distance (FID) to quantitatively evaluate image quality and alignment across 10 categories, including people and fashion.
• Refraining from architectural changes, focusing instead on training recipe improvements to elevate aesthetic quality from Playground v2.

(a) Generating solid backgrounds. The top row is sampled from SDXL [ 28 ] , bottom row is Playground v2.5. SDXL fails to generate pure black or white background while our model can follow the prompt faithfully.

Experimental results

Research questions

• RQ1 How does noise schedule optimization impact the realism and visual fidelity of text-to-image diffusion models?
• RQ2 To what extent can a balanced, bucketed dataset improve generation quality across multiple aspect ratios?
• RQ3 How does human preference alignment affect the generation of fine details in human-centric images?
• RQ4 Can an automatic benchmark like MJHQ-30K reliably correlate with human preference in aesthetic image generation?
• RQ5 To what extent does improving training recipe components surpass architectural changes in enhancing aesthetic quality?

Key findings

• Playground v2.5 achieves an overall FID of 4.48 on the MJHQ-30K benchmark, significantly outperforming SDXL (9.55) and Playground v2 (7.07).
• The model demonstrates superior performance in generating vibrant colors and contrast, especially in complex prompts and solid background generation, as shown in comparative samples.
• Playground v2.5 outperforms SDXL, RealStock v2, and Playground v2 in user preference studies on the People-200 prompt set, particularly in portrait and human-centric image generation.
• The model achieves state-of-the-art results across all aspect ratios, with consistent performance improvements over SDXL and Playground v2.
• The MJHQ-30K benchmark shows strong correlation between FID scores and human preference, validating its use as a reliable automatic evaluation metric.
• Playground v2.5 surpasses closed-source models like DALL·E 3 and Midjourney v5.2 in user preference studies, demonstrating the effectiveness of its training recipe improvements.

(b) Colors and contrast . The top row is SDXL, bottom row is Playground v2.5. Our model can generate samples with more vibrant colors and contrast.