[Paper Review] WITT: A Wireless Image Transmission Transformer for Semantic Communications

Ke Yang, Sixian Wang+4 more|arXiv (Cornell University)|Nov 2, 2022

Wireless Signal Modulation Classification4 citations

TL;DR

This paper proposes WITT, a vision Transformer-based joint source-channel coding framework for wireless image transmission that leverages Swin Transformers to capture long-range dependencies and a Channel ModNet to adapt latent representations to varying channel states. WITT achieves superior performance over CNN-based methods, especially at high resolutions and low SNR, with gains in PSNR, MS-SSIM, and bandwidth efficiency across AWGN and Rayleigh fading channels.

ABSTRACT

In this paper, we aim to redesign the vision Transformer (ViT) as a new backbone to realize semantic image transmission, termed wireless image transmission transformer (WITT). Previous works build upon convolutional neural networks (CNNs), which are inefficient in capturing global dependencies, resulting in degraded end-to-end transmission performance especially for high-resolution images. To tackle this, the proposed WITT employs Swin Transformers as a more capable backbone to extract long-range information. Different from ViTs in image classification tasks, WITT is highly optimized for image transmission while considering the effect of the wireless channel. Specifically, we propose a spatial modulation module to scale the latent representations according to channel state information, which enhances the ability of a single model to deal with various channel conditions. As a result, extensive experiments verify that our WITT attains better performance for different image resolutions, distortion metrics, and channel conditions. The code is available at https://github.com/KeYang8/WITT.

Motivation & Objective

• To overcome the performance degradation of CNN-based joint source-channel coding (JSCC) in high-resolution image transmission.
• To enhance representation capacity in wireless image transmission by replacing CNNs with vision Transformers.
• To enable a single deep learning model to adapt to diverse wireless channel conditions without retraining.
• To achieve better end-to-end transmission performance in terms of PSNR, MS-SSIM, and bandwidth efficiency.
• To integrate global attention mechanisms with practical wireless constraints such as power and fading.

Proposed method

• WITT employs a hierarchical Swin Transformer encoder with patch embedding and patch merging across multiple stages to extract multi-scale, long-range features.
• A Channel ModNet is inserted after the final encoder layer to rescale latent representations based on real-time channel state information (CSI).
• The model uses a learnable scaling mechanism in the Channel ModNet that adapts to varying SNR levels without retraining.
• The latent features are projected to a fixed dimension via a fully connected layer and normalized for power constraints before analog transmission over a fading channel.
• The symmetric decoder reconstructs images using up-sampling, patch division, and Swin Transformer blocks with the same Channel ModNet for adaptation.
• The system is trained end-to-end with a reconstruction loss minimizing pixel-wise error under a power constraint.

Experimental results

Research questions

• RQ1 Can a vision Transformer backbone significantly improve end-to-end image transmission performance compared to CNNs, especially at high resolutions?
• RQ2 How can a single deep learning model effectively adapt to varying wireless channel conditions without retraining?
• RQ3 To what extent does incorporating global attention via Swin Transformers improve transmission robustness and quality under fading channels?
• RQ4 How does WITT compare to classical separation-based schemes (e.g., BPG + LDPC) in terms of PSNR, MS-SSIM, and bandwidth efficiency?
• RQ5 Can the Channel ModNet enable a unified model to cover a wide SNR range (e.g., 1–13 dB) while maintaining high performance?

Key findings

• WITT achieves significantly higher PSNR and MS-SSIM than CNN-based JSCC across all image resolutions, with performance gaps widening at higher resolutions.
• On the CIFAR10 dataset, WITT outperforms the CNN-based ADJSCC by 1.2 dB in PSNR at 13 dB SNR over AWGN and by 1.8 dB in MS-SSIM at the same SNR.
• For high-resolution images (DIV2K), WITT maintains strong performance even at low SNR (e.g., 1 dB), while CNN-based JSCC degrades rapidly and falls behind separation-based schemes.
• WITT achieves comparable or better performance than BPG + LDPC across all CBRs, with a 1.5 dB coding gain advantage over CNN-based JSCC at low CBR (1/16) on Rayleigh fading channels.
• WITT reduces FLOPs by 2.5× compared to ADJSCC (198 G vs. 511 G) while achieving faster inference (116 ms vs. 155 ms) despite a larger model size.
• Visual reconstructions from WITT show fewer block artifacts and higher texture fidelity, especially at low bandwidth, confirming perceptual superiority.