[Paper Review] Generative AI-Based Probabilistic Constellation Shaping With Diffusion Models

Mehdi Letafati, Samad Ali+1 more|arXiv (Cornell University)|Nov 15, 2023

Music and Audio Processing4 citations

TL;DR

This paper proposes a novel diffusion model-based approach for probabilistic constellation shaping in wireless communications, leveraging the denoise-and-generate mechanism of denoising diffusion probabilistic models (DDPM) to align transmitter and receiver symbol reconstruction. It achieves a 30% improvement in cosine similarity and a threefold gain in mutual information over DNN-based benchmarks for 64-QAM, with robust performance under low-SNR and non-Gaussian noise conditions.

ABSTRACT

Diffusion models are at the vanguard of generative AI research with renowned solutions such as ImageGen by Google Brain and DALL.E 3 by OpenAI. Nevertheless, the potential merits of diffusion models for communication engineering applications are not fully understood yet. In this paper, we aim to unleash the power of generative AI for PHY design of constellation symbols in communication systems. Although the geometry of constellations is predetermined according to networking standards, e.g., quadrature amplitude modulation (QAM), probabilistic shaping can design the probability of occurrence (generation) of constellation symbols. This can help improve the information rate and decoding performance of communication systems. We exploit the ``denoise-and-generate'' characteristics of denoising diffusion probabilistic models (DDPM) for probabilistic constellation shaping. The key idea is to learn generating constellation symbols out of noise, ``mimicking'' the way the receiver performs symbol reconstruction. This way, we make the constellation symbols sent by the transmitter, and what is inferred (reconstructed) at the receiver become as similar as possible, resulting in as few mismatches as possible. Our results show that the generative AI-based scheme outperforms deep neural network (DNN)-based benchmark and uniform shaping, while providing network resilience as well as robust out-of-distribution performance under low-SNR regimes and non-Gaussian assumptions. Numerical evaluations highlight 30% improvement in terms of cosine similarity and a threefold improvement in terms of mutual information compared to DNN-based approach for 64-QAM geometry.

Motivation & Objective

• To explore the potential of generative AI, specifically diffusion models, in physical layer signal design for wireless communications.
• To address the limitations of discriminative models in modeling complex signal distributions and enabling end-to-end generative design.
• To improve spectral efficiency and robustness in wireless systems by learning optimal symbol probability distributions through generative modeling.
• To achieve network resilience and out-of-distribution robustness under low-SNR and non-Gaussian noise conditions.
• To establish a new paradigm of AI-native wireless systems by unifying transmitter and receiver symbol reconstruction via reciprocal generative learning.

Proposed method

• Utilizes denoising diffusion probabilistic models (DDPM) to generate constellation symbols by reversing a noise-adding process, mimicking receiver-based symbol reconstruction.
• Trains the DDPM to learn the underlying distribution of constellation symbols, enabling probabilistic shaping through noise-to-symbol generation.
• Aligns the transmitter's symbol generation with the receiver’s denoising process, creating a reciprocal understanding that minimizes reconstruction mismatches.
• Employs a latent diffusion architecture with residual U-Net encoders and decoders to model high-dimensional signal distributions efficiently.
• Uses a noise schedule and reverse diffusion process to iteratively denoise random noise into structured constellation points.
• Evaluates performance using mutual information and cosine similarity as metrics, comparing against DNN-based and uniform shaping benchmarks.

Experimental results

Research questions

• RQ1 Can diffusion models be effectively applied to probabilistic constellation shaping in wireless communication systems?
• RQ2 How does a DDPM-based approach compare to DNN-based and uniform shaping in terms of mutual information and symbol reconstruction similarity?
• RQ3 To what extent does the proposed method maintain performance under low-SNR and non-Gaussian noise conditions?
• RQ4 Can the generative nature of diffusion models enhance network resilience and out-of-distribution robustness in wireless systems?
• RQ5 Does the reciprocal design—where the transmitter emulates the receiver’s denoising process—lead to improved system performance?

Key findings

• For 64-QAM, the proposed DDPM-based scheme achieves a 30% improvement in cosine similarity compared to the DNN-based benchmark.
• The method demonstrates a threefold improvement in mutual information over the DNN-based approach for 64-QAM under the same conditions.
• The generative AI-based system maintains high performance across multiple SNR levels, with median and maximum performance metrics consistently outperforming uniform shaping.
• Under out-of-distribution noise (Laplacian and exponential), the DDPM-based approach shows no performance degradation, while the DNN benchmark degrades significantly.
• The system exhibits robustness to non-Gaussian noise, with performance remaining stable even when noise distribution deviates from the assumed Gaussian model.
• The proposed method achieves better performance with higher-order constellations (e.g., 64-QAM), unlike conventional approaches that degrade under such conditions.