[Paper Review] Enhancing Authorship Attribution with Synthetic Paintings

Clarissa Lima Loures, Caio Hosken+3 more|arXiv (Cornell University)|Mar 4, 2026

Aesthetic Perception and Analysis0 citations

TL;DR

The paper shows that adding synthetic paintings generated by DreamBooth fine-tuning of Stable Diffusion improves authorship attribution models on small, stylistically similar painter datasets, with hybrid real+synthetic training yielding robust gains.

ABSTRACT

Attributing authorship to paintings is a historically complex task, and one of its main challenges is the limited availability of real artworks for training computational models. This study investigates whether synthetic images, generated through DreamBooth fine-tuning of Stable Diffusion, can improve the performance of classification models in this context. We propose a hybrid approach that combines real and synthetic data to enhance model accuracy and generalization across similar artistic styles. Experimental results show that adding synthetic images leads to higher ROC-AUC and accuracy compared to using only real paintings. By integrating generative and discriminative methods, this work contributes to the development of computer vision techniques for artwork authentication in data-scarce scenarios.

Motivation & Objective

• Motivate robust authorship attribution on small, stylistically similar painting datasets.
• Investigate whether synthetic data can augment real paintings to improve generalization.
• Assess the impact of combining real and synthetic data under different patch sampling strategies.
• Examine domain shifts between synthetic and real paintings and their effect on model performance.

Proposed method

• Fine-tune Stable Diffusion models per artist using DreamBooth to capture individual painting styles.
• Generate 100 synthetic paintings per artist with a consistent prompt and negative prompts to avoid cropped figures.
• Extract patch-level visual embeddings from three transformers (MaxViT, BEiT v2, VOLO) and concatenate them.
• Train per-artist LightGBM classifiers on the combined embeddings with class balancing and early stopping.
• Evaluate using ROC-AUC, accuracy, precision, recall, and F1-score across four data configurations (Real-Only, Synthetic-Only, Synthetic→Real, Real+Synthetic) and two patch-sampling strategies (M1, M2).
• Compare domain generalization and analyze how denser sampling (M2) affects performance.

Experimental results

Research questions

• RQ1 Does synthetic data improve authorship attribution when real data is scarce?
• RQ2 How does training with a mix of real and synthetic paintings affect generalization across artists with varying data availability?
• RQ3 What is the impact of patch sampling density on model performance in data-scarce settings?
• RQ4 What are the limitations and domain gaps when transferring from synthetic to real paintings?
• RQ5 Can a unified feature-based classifier effectively leverage multiple embeddings for this task?

Key findings

• Synthetic data augmentation yields higher ROC-AUC and accuracy than real data alone across artists.
• Hybrid training with real and synthetic data (especially Hybrid-M2) provides the most stable and robust results.
• Synthetic-only setups achieve very high ROC-AUC (above 0.98) and accuracy (above 95%) across artists.
• Performance varies by artist, with some (e.g., TB) showing sensitivity to synthetic data and domain shifts.
• Denser patch sampling (M2) generally improves generalization, particularly for artists with fewer real paintings, though benefits are not uniform.
• A noticeable domain gap exists in Real→Synthetic transfer, indicating limitations in current generation techniques.