[Paper Review] ALOFT: A Lightweight MLP-like Architecture with Dynamic Low-frequency Transform for Domain Generalization

Jintao Guo, Na Wang+2 more|arXiv (Cornell University)|Mar 21, 2023

Domain Adaptation and Few-Shot Learning4 citations

TL;DR

This paper proposes ALOFT, a lightweight MLP-like architecture with a dynamic low-frequency transform for domain generalization, which enhances model robustness by suppressing domain-specific texture features while preserving global structural representations. By leveraging frequency-space filtering and a learnable low-frequency spectrum perturbation mechanism, ALOFT achieves state-of-the-art performance with significantly fewer parameters than existing CNN-based methods.

ABSTRACT

Domain generalization (DG) aims to learn a model that generalizes well to unseen target domains utilizing multiple source domains without re-training. Most existing DG works are based on convolutional neural networks (CNNs). However, the local operation of the convolution kernel makes the model focus too much on local representations (e.g., texture), which inherently causes the model more prone to overfit to the source domains and hampers its generalization ability. Recently, several MLP-based methods have achieved promising results in supervised learning tasks by learning global interactions among different patches of the image. Inspired by this, in this paper, we first analyze the difference between CNN and MLP methods in DG and find that MLP methods exhibit a better generalization ability because they can better capture the global representations (e.g., structure) than CNN methods. Then, based on a recent lightweight MLP method, we obtain a strong baseline that outperforms most state-of-the-art CNN-based methods. The baseline can learn global structure representations with a filter to suppress structure irrelevant information in the frequency space. Moreover, we propose a dynAmic LOw-Frequency spectrum Transform (ALOFT) that can perturb local texture features while preserving global structure features, thus enabling the filter to remove structure-irrelevant information sufficiently. Extensive experiments on four benchmarks have demonstrated that our method can achieve great performance improvement with a small number of parameters compared to SOTA CNN-based DG methods. Our code is available at https://github.com/lingeringlight/ALOFT/.

Motivation & Objective

• To investigate the performance gap between MLP and CNN architectures in domain generalization tasks.
• To address the inherent texture bias in CNNs that limits generalization to unseen domains.
• To develop a lightweight, parameter-efficient architecture that enhances global structure representation while suppressing domain-specific texture features.
• To design a dynamic low-frequency spectrum transform that simulates diverse domain shifts during training.
• To achieve state-of-the-art performance in domain generalization with a small, efficient model.

Proposed method

• The method builds on a lightweight MLP-based backbone (GFNet) to learn global image representations through self-attention-like mechanisms across patches.
• It introduces a learnable filter in the frequency domain to suppress structure-irrelevant, domain-specific texture information.
• A novel dynamic low-frequency spectrum transform (ALOFT) perturbs low-frequency components by resampling from estimated distributions, simulating unseen domain shifts.
• The low-frequency components are modeled using a Gaussian distribution to generate diverse, realistic perturbations that preserve global structure.
• The framework applies frequency-domain filtering and dynamic spectrum augmentation during training to improve domain-invariant feature learning.
• The method is trained end-to-end with standard cross-entropy loss and uses a mask ratio of 0.5 for data augmentation.

Experimental results

Research questions

• RQ1 How do MLP-based architectures differ from CNNs in their ability to generalize across unseen domains?
• RQ2 Why do MLP-based models exhibit better generalization in domain generalization tasks compared to CNNs?
• RQ3 Can frequency-domain filtering effectively suppress domain-specific texture features while preserving global structure?
• RQ4 How does dynamic low-frequency spectrum perturbation improve robustness to domain shift?
• RQ5 Can a lightweight MLP-based model outperform state-of-the-art CNN-based domain generalization methods with fewer parameters?

Key findings

• ALOFT achieves an average accuracy of 91.58% on the PACS benchmark using the GFNet backbone, outperforming the SOTA CNN-based methods.
• The method improves generalization across all domains (Art, Cartoons, Sketch, Real-World) with consistent gains, especially on the Sketch and Art domains.
• Using a Gaussian distribution for low-frequency spectrum noise generation yields the best performance, outperforming uniform and random noise distributions.
• The dynamic low-frequency transform (ALOFT) significantly improves model generalization compared to image-level (Mixup, CutMix) and feature-level (MixStyle, DSU) augmentation methods.
• The model maintains strong performance even with a small mask ratio (0.5), indicating effective data augmentation without overfitting.
• Frequency analysis confirms that MLP-based models better capture global structure, which contributes to their superior generalization in domain generalization tasks.