[Paper Review] Vector Quantized Semantic Communication System

Qifan Fu, Huiqiang Xie+3 more|arXiv (Cornell University)|Sep 23, 2022

Neural Networks and Applications6 citations

TL;DR

This paper proposes VQ-DeepSC, a deep learning-enabled vector quantized semantic communication system for image transmission that uses multi-scale semantic embedding spaces and adversarial training to compress and transmit image features as discrete indices. The system achieves superior robustness to BPG and comparable MS-SSIM performance to DeepJSCC, demonstrating strong compatibility with digital communication systems.

ABSTRACT

Although analog semantic communication systems have received considerable attention in the literature, there is less work on digital semantic communication systems. In this paper, we develop a deep learning (DL)-enabled vector quantized (VQ) semantic communication system for image transmission, named VQ-DeepSC. Specifically, we propose a convolutional neural network (CNN)-based transceiver to extract multi-scale semantic features of images and introduce multi-scale semantic embedding spaces to perform semantic feature quantization, rendering the data compatible with digital communication systems. Furthermore, we employ adversarial training to improve the quality of received images by introducing a PatchGAN discriminator. Experimental results demonstrate that the proposed VQ-DeepSC is more robustness than BPG in digital communication systems and has comparable MS-SSIM performance to the DeepJSCC method.

Motivation & Objective

• To address the limited work on digital semantic communication systems by proposing a deep learning-based framework compatible with digital transmission.
• To reduce transmission overhead by leveraging multi-scale semantic feature extraction and vector quantization.
• To improve image reconstruction quality through adversarial training using a PatchGAN discriminator.
• To ensure compatibility with digital communication systems by mapping semantic features to discrete indices via shared embedding spaces.
• To evaluate robustness against channel variations and compare performance with established codecs like BPG and DeepJSCC.

Proposed method

• A U-Net-based encoder-decoder architecture extracts multi-scale semantic features from input images.
• Vector quantization maps extracted semantic features to discrete indices using shared semantic embedding spaces at transmitter and receiver.
• Four multi-scale embedding spaces with dimensions K₁=128, K₂=256, K₃=512, and K₄=1024 are used to support hierarchical feature representation.
• A PatchGAN discriminator is employed in adversarial training to enhance the perceptual quality of reconstructed images.
• The system uses 1×1 convolutional layers for feature compression and LDPC channel coding with adaptive modulation for digital transmission.
• Training employs Adam optimizer with specific learning rates and betas, and uses a GAN-based loss to improve image fidelity.

Experimental results

Research questions

• RQ1 Can a vector quantized semantic communication system achieve better robustness than traditional codecs like BPG in digital wireless channels?
• RQ2 How does the use of multi-scale semantic embedding spaces affect image reconstruction quality and transmission efficiency?
• RQ3 To what extent does adversarial training with a PatchGAN discriminator improve the perceptual quality of reconstructed images?
• RQ4 How does VQ-DeepSC perform in comparison to the state-of-the-art DeepJSCC method in terms of MS-SSIM under varying SNR and channel conditions?
• RQ5 How sensitive is the performance of VQ-DeepSC to LDPC blocklength and modulation order compared to BPG?

Key findings

• VQ-DeepSC achieves a cliff point at approximately 1 dB lower SNR than BPG in AWGN channels when using LDPC with 1/2 rate and BPSK modulation.
• At SNR > 5 dB over Rician fading channels, VQ-DeepSC achieves MS-SSIM above 0.8, comparable to DeepJSCC.
• With similar compression ratios, VQ-DeepSC using 32-QAM achieves higher MS-SSIM than BPG using 16-QAM at low SNR regimes.
• VQ-DeepSC3 shows significantly less sensitivity to LDPC blocklength than BPG, with a performance gain of approximately 8 dB for BPG when increasing blocklength from 20 to 64800 bits.
• The system outperforms UNet-DeepJSCC in certain SNR regimes by adjusting coding rates and modulation, demonstrating improved robustness.
• The combination of multi-scale embedding spaces and adversarial training enables high-quality image reconstruction with low transmission overhead.