[Paper Review] PSCR: Patches Sampling-based Contrastive Regression for AIGC Image Quality Assessment

Jiquan Yuan, Xinyan Cao+3 more|arXiv (Cornell University)|Dec 10, 2023

Advanced Image Fusion Techniques4 citations

TL;DR

This paper proposes PSCR, a patches sampling-based contrastive regression framework for AIGC image quality assessment that improves performance by learning a discriminative representation space through relative score regression between query and exemplar images, while avoiding geometric distortion via overlapping patch sampling. The method achieves state-of-the-art results on three major AIGCIQA benchmarks, with improvements up to +0.016 in SRCC and +0.012 in PLCC.

ABSTRACT

In recent years, Artificial Intelligence Generated Content (AIGC) has gained widespread attention beyond the computer science community. Due to various issues arising from continuous creation of AI-generated images (AIGI), AIGC image quality assessment (AIGCIQA), which aims to evaluate the quality of AIGIs from human perception perspectives, has emerged as a novel topic in the field of computer vision. However, most existing AIGCIQA methods directly regress predicted scores from a single generated image, overlooking the inherent differences among AIGIs and scores. Additionally, operations like resizing and cropping may cause global geometric distortions and information loss, thus limiting the performance of models. To address these issues, we propose a patches sampling-based contrastive regression (PSCR) framework. We suggest introducing a contrastive regression framework to leverage differences among various generated images for learning a better representation space. In this space, differences and score rankings among images can be measured by their relative scores. By selecting exemplar AIGIs as references, we also overcome the limitations of previous models that could not utilize reference images on the no-reference image databases. To avoid geometric distortions and information loss in image inputs, we further propose a patches sampling strategy. To demonstrate the effectiveness of our proposed PSCR framework, we conduct extensive experiments on three mainstream AIGCIQA databases including AGIQA-1K, AGIQA-3K and AIGCIQA2023. The results show significant improvements in model performance with the introduction of our proposed PSCR framework. Code will be available at \url{https://github.com/jiquan123/PSCR}.

Motivation & Objective

• Address the limitation of existing AIGCIQA methods that directly regress scores from single images without leveraging inter-image differences.
• Overcome the performance degradation caused by image preprocessing such as resizing and cropping, which induce geometric distortion and information loss.
• Enable no-reference AIGCIQA models to utilize exemplar reference images for improved score regression, overcoming a key limitation of prior methods.
• Develop a robust, generalizable framework that enhances representation learning by modeling relative quality rankings among AIGIs.
• Minimize information loss and structural disruption in AIGI inputs by introducing a sliding-window-based overlapping patches sampling strategy.

Proposed method

• Propose a contrastive regression framework that regresses relative scores between a query AIGI and a set of randomly selected exemplar AIGIs, enabling better representation learning through ranking-aware supervision.
• Introduce a patches sampling strategy using a sliding window over the original-resolution AIGI to extract overlapping, non-overlapping patches, preserving global structure and minimizing information loss.
• Use a deep neural network backbone (e.g., VGG, ResNet, InceptionV4) to extract features from the sampled patches, followed by global pooling and score prediction via a regression head.
• Train the model using a contrastive loss that encourages closer representations for images with similar relative scores and wider separation for dissimilar ones.
• Integrate the PSCR framework into existing IQA models as a plug-in module, enabling easy adaptation to different backbones and datasets.
• Apply data augmentation and normalization techniques to improve training stability and generalization across diverse AIGI distributions.

Experimental results

Research questions

• RQ1 Can a contrastive regression framework improve AIGCIQA performance by modeling relative quality differences between AIGIs?
• RQ2 Does using overlapping patches via a sliding window reduce geometric distortion and information loss compared to standard cropping or resizing?
• RQ3 Can exemplar-based reference images enhance no-reference AIGCIQA beyond single-image regression?
• RQ4 How does the PSCR framework compare to SOTA methods across multiple AIGCIQA benchmarks in terms of SRCC and PLCC?
• RQ5 What is the impact of patch sampling strategy hyperparameters (e.g., stride, patch size) on model performance and efficiency?

Key findings

• The PSCR framework achieves state-of-the-art performance on AGIQA-1K, AGIQA-3K, and AIGCIQA2023, with SRCC scores of 0.8498 and PLCC of 0.9059 using InceptionV4 as the backbone.
• The PSCR variant with InceptionV4 achieves a 0.016 increase in SRCC and 0.0058 increase in PLCC compared to the baseline model, demonstrating significant performance gains.
• The ablation study confirms that both the patches sampling (PS) and contrastive regression (CR) components contribute independently and synergistically to performance improvement.
• The method outperforms all baseline models (VGG16, VGG19, ResNet18, ResNet50, InceptionV4) across all three datasets, with the largest gains observed on AIGCIQA2023.
• The use of exemplar AIGIs as references enables the model to learn more robust and generalizable representations, especially in low-quality or highly distorted image regions.
• The sliding window patch sampling strategy reduces information loss and preserves local textures and artifacts critical for AIGI quality assessment.