[Paper Review] IntraQ: Learning Synthetic Images with Intra-Class Heterogeneity for Zero-Shot Network Quantization

Yunshan Zhong, Mingbao Lin+5 more|arXiv (Cornell University)|Nov 17, 2021

Advanced Neural Network Applications39 references4 citations

TL;DR

IntraQ proposes a novel zero-shot quantization method that enhances synthetic image quality by preserving intra-class heterogeneity—diverse object scales and positions, class-aware feature distribution, and soft label priors. By integrating local object reinforcement, marginal distance constraints, and soft inception loss, IntraQ achieves a 9.17% top-1 accuracy gain over SOTA on ImageNet when quantizing MobileNetV1 to 4-bit.

ABSTRACT

Learning to synthesize data has emerged as a promising direction in zero-shot quantization (ZSQ), which represents neural networks by low-bit integer without accessing any of the real data. In this paper, we observe an interesting phenomenon of intra-class heterogeneity in real data and show that existing methods fail to retain this property in their synthetic images, which causes a limited performance increase. To address this issue, we propose a novel zero-shot quantization method referred to as IntraQ. First, we propose a local object reinforcement that locates the target objects at different scales and positions of the synthetic images. Second, we introduce a marginal distance constraint to form class-related features distributed in a coarse area. Lastly, we devise a soft inception loss which injects a soft prior label to prevent the synthetic images from being overfitting to a fixed object. Our IntraQ is demonstrated to well retain the intra-class heterogeneity in the synthetic images and also observed to perform state-of-the-art. For example, compared to the advanced ZSQ, our IntraQ obtains 9.17\% increase of the top-1 accuracy on ImageNet when all layers of MobileNetV1 are quantized to 4-bit. Code is at https://github.com/zysxmu/IntraQ.

Motivation & Objective

• Address the performance gap in zero-shot quantization (ZSQ) caused by synthetic images lacking intra-class diversity.
• Identify that existing ZSQ methods fail to preserve intra-class heterogeneity in synthetic data, leading to suboptimal model performance.
• Develop a data synthesis strategy that explicitly models object scale and position variation within the same class to improve feature diversity.
• Introduce a marginal distance constraint to prevent feature clustering and maintain class-wide feature spread in latent space.
• Propose a soft inception loss with prior label injection to generate more complex, realistic scenes without overfitting to a single object.

Proposed method

• Apply local object reinforcement to position target objects at varying scales and spatial locations in synthetic images, increasing intra-class variation.
• Introduce a marginal distance constraint that enforces feature embeddings of the same class to be distributed across a coarse region, avoiding feature concentration.
• Design a soft inception loss that uses soft labels as prior knowledge to guide image generation, promoting diversity in scene composition.
• Combine the three components—local object reinforcement, marginal distance constraint, and soft inception loss—within a unified optimization framework for synthetic image generation.
• Use the resulting synthetic images to fine-tune quantized networks without accessing real training data, enabling zero-shot quantization.
• Leverage pre-trained full-precision models and batch normalization statistics to guide the optimization process, ensuring compatibility with existing quantization pipelines.

(a) Real data

Experimental results

Research questions

• RQ1 Why do existing zero-shot quantization methods underperform despite using synthetic data?
• RQ2 To what extent does intra-class heterogeneity in real data influence the performance of quantized models?
• RQ3 Can explicit modeling of object scale and position variation in synthetic images improve zero-shot quantization accuracy?
• RQ4 How does enforcing feature distribution across a coarse region via marginal distance constraint affect model generalization?
• RQ5 Does using soft label priors in image generation lead to more diverse and realistic synthetic images compared to hard label approaches?

Key findings

• IntraQ achieves a 9.17% top-1 accuracy improvement over the advanced DSG+IL method when quantizing MobileNetV1 to 4-bit on ImageNet.
• The ablation study confirms that combining all three components—local object reinforcement, marginal distance constraint, and soft inception loss—yields the highest accuracy of 66.47% on 4-bit ResNet-18.
• Local object reinforcement alone boosts accuracy from 63.38% (ZeroQ+IL baseline) to 66.14%, demonstrating its significant impact on intra-class diversity.
• The marginal distance constraint effectively prevents feature clustering, resulting in more dispersed and class-discriminative features in latent space.
• Feature visualization via t-SNE confirms that IntraQ-generated images preserve intra-class heterogeneity, with features spread across a coarse region, unlike the dense clusters seen in ZeroQ and DSG.
• IntraQ outperforms state-of-the-art ZSQ methods across multiple bit-width settings, especially in ultra-low-precision scenarios like 4-bit quantization.

(b) ZeroQ