[Paper Review] What Happens During Finetuning of Vision Transformers: An Invariance Based Investigation

Gabriele Merlin, Vedant Nanda+2 more|arXiv (Cornell University)|Jul 12, 2023

Advanced Neural Network ApplicationsComputer Science3 citations

TL;DR

This paper investigates how invariances learned during pretraining are retained or altered during fine-tuning of Vision Transformers (ViTs). Using a novel invariance-based metric derived from STIR, it reveals that pretraining induces transferable invariances in shallow layers, while deeper-layer invariances are compressed toward shallower layers during fine-tuning—offering insight into why pretraining improves downstream performance and how model representations evolve during adaptation.

ABSTRACT

The pretrain-finetune paradigm usually improves downstream performance over training a model from scratch on the same task, becoming commonplace across many areas of machine learning. While pretraining is empirically observed to be beneficial for a range of tasks, there is not a clear understanding yet of the reasons for this effect. In this work, we examine the relationship between pretrained vision transformers and the corresponding finetuned versions on several benchmark datasets and tasks. We present new metrics that specifically investigate the degree to which invariances learned by a pretrained model are retained or forgotten during finetuning. Using these metrics, we present a suite of empirical findings, including that pretraining induces transferable invariances in shallow layers and that invariances from deeper pretrained layers are compressed towards shallower layers during finetuning. Together, these findings contribute to understanding some of the reasons for the successes of pretrained models and the changes that a pretrained model undergoes when finetuned on a downstream task.

Motivation & Objective

• Understand why pretraining improves downstream performance in Vision Transformers, especially in transfer learning settings.
• Investigate how invariances learned during pretraining are modified, retained, or forgotten during fine-tuning on downstream tasks.
• Characterize the dynamic trade-off between stability (retaining old invariances) and plasticity (learning new invariances) during fine-tuning.
• Develop new metrics to quantify invariance transfer and change across layers during fine-tuning, moving beyond traditional representation similarity or accuracy-based analysis.
• Provide a mechanistic explanation for the success of the pretrain-fine-tune paradigm by analyzing layer-wise invariance evolution.

Proposed method

• Adopt STIR (Similarity of Transferable Invariances) to measure shared invariance between a pretrained model and its fine-tuned version with respect to specific data perturbations.
• Define new metrics to quantify invariance forgetting (loss of pretrained invariances) and invariance learning (acquisition of new invariances) during fine-tuning.
• Apply these metrics across multiple benchmark datasets (e.g., CIFAR-10, ImageNet) and tasks to analyze layer-wise invariance dynamics.
• Use correlation analysis between early training dynamics (e.g., average accuracy and standard deviation of forgetting across layers) and final robustness on corrupted data to assess early indicators of model stability.
• Systematically vary hyperparameters (learning rate, optimizer, data augmentation) to validate robustness of observed invariance patterns.
• Perform ablation studies to isolate the contribution of early training epochs to invariance stability and robustness.

Experimental results

Research questions

• RQ1 To what extent are invariances learned during pretraining retained or forgotten during ViT fine-tuning?
• RQ2 How do invariances from deeper layers of a pretrained ViT change during fine-tuning, and is there evidence of compression toward shallower layers?
• RQ3 What is the relationship between invariance dynamics (forgetting and learning) and model robustness during fine-tuning?
• RQ4 Can early training dynamics—specifically the standard deviation of forgetting across layers—serve as a reliable indicator of final model robustness?
• RQ5 How do different optimizers (e.g., SGD vs. Adam) affect the relationship between invariance dynamics and model performance on corrupted data?

Key findings

• Pretraining induces transferable invariances, particularly in the shallow layers of Vision Transformers, which are crucial for downstream performance.
• Invariances from deeper layers of the pretrained model are compressed toward shallower layers during fine-tuning, suggesting a mechanism for reallocating representational capacity.
• The standard deviation of forgetting across layers during early training is strongly correlated with final model robustness on corrupted datasets, even when considering only the first 20 epochs.
• This correlation remains strong or even increases in early training phases, indicating that early invariance dynamics are predictive of robustness and not merely a result of later optimization behavior.
• The relationship between invariance dynamics and robustness holds across different hyperparameter settings with SGD, but not consistently with Adam, suggesting optimizer-dependent behavior in invariance stability.
• The findings suggest that invariance compression and layer-specific forgetting dynamics are key mechanisms underlying the success of the pretrain-fine-tune paradigm in ViTs.