[Paper Review] Communication Algorithms via Deep Learning

Hyeji Kim, Yihan Jiang+4 more|arXiv (Cornell University)|Feb 1, 2018

Error Correcting Code Techniques22 references48 citations

TL;DR

This paper proposes a deep learning approach to automatically discover near-optimal decoding algorithms for sequential codes like convolutional and turbo codes using recurrent neural networks (RNNs). By training RNN decoders on the additive white Gaussian noise (AWGN) channel, the method achieves near-optimal bit and block error rates, generalizes across signal-to-noise ratios and block lengths, and demonstrates robustness to non-Gaussian and bursty noise, outperforming traditional decoders in challenging conditions.

ABSTRACT

Coding theory is a central discipline underpinning wireline and wireless modems that are the workhorses of the information age. Progress in coding theory is largely driven by individual human ingenuity with sporadic breakthroughs over the past century. In this paper we study whether it is possible to automate the discovery of decoding algorithms via deep learning. We study a family of sequential codes parameterized by recurrent neural network (RNN) architectures. We show that creatively designed and trained RNN architectures can decode well known sequential codes such as the convolutional and turbo codes with close to optimal performance on the additive white Gaussian noise (AWGN) channel, which itself is achieved by breakthrough algorithms of our times (Viterbi and BCJR decoders, representing dynamic programing and forward-backward algorithms). We show strong generalizations, i.e., we train at a specific signal to noise ratio and block length but test at a wide range of these quantities, as well as robustness and adaptivity to deviations from the AWGN setting.

Motivation & Objective

• To investigate whether deep learning can automate the discovery of efficient decoding algorithms for classical sequential codes.
• To evaluate if RNN-based neural decoders can generalize across varying signal-to-noise ratios (SNR) and block lengths.
• To assess robustness of learned decoders under deviations from the AWGN model, such as bursty or heavy-tailed noise.
• To compare neural decoder performance against state-of-the-art conventional decoders (e.g., Viterbi, BCJR, Turbo) in both standard and non-standard channel conditions.
• To explore whether neural networks can implicitly learn complex decoding algorithms like Viterbi and BCJR through end-to-end training without architectural priors.

Proposed method

• The authors use RNN architectures to model sequential decoding, parameterizing codes via recurrent structures that naturally support variable-length inputs and diverse coding rates.
• They train the RNN decoder end-to-end on synthetic data generated from known convolutional and turbo codes over the AWGN channel at a specific SNR and block length.
• Generalization is tested across a wide range of SNRs and block lengths (up to 10,000 bits), with performance evaluated via bit error rate (BER) and block error rate (BLER).
• Robustness is evaluated under non-AWGN conditions, including t-distributed noise and bursty noise models, where the noise variance increases temporarily.
• The neural decoder is fine-tuned on bursty noise models to adapt to interference scenarios like co-channel radar or OFDM inter-cell interference.
• Interpretability is explored via positional likelihood ratio analysis, comparing neural decoder confidence estimates with those of the BCJR algorithm.

Experimental results

Research questions

• RQ1 Can RNN-based neural decoders achieve near-optimal performance on standard AWGN channels for convolutional and turbo codes without architectural priors?
• RQ2 To what extent can a single RNN decoder generalize across different SNRs and block lengths after training on a single configuration?
• RQ3 How does a learned neural decoder perform under non-Gaussian and bursty noise models compared to conventional decoders?
• RQ4 Can the neural decoder adapt to channel variations without retraining, especially when the noise model deviates from the training assumption?
• RQ5 Does the neural decoder’s internal confidence estimation (via LLR outputs) reflect more robust error propagation behavior than classical algorithms like BCJR?

Key findings

• The RNN decoder achieves bit error rate (BER) and block error rate (BLER) performance within 0.1 dB of the optimal Viterbi and BCJR decoders on the AWGN channel.
• The model generalizes effectively across SNR and block length, maintaining near-optimal performance even when tested at 10,000-bit block lengths after training on 100-bit sequences.
• The neural decoder outperforms the standard Turbo decoder under t-distributed noise (ν=3), particularly due to more conservative and stable likelihood estimates that reduce error propagation.
• In bursty noise scenarios, the fine-tuned neural decoder (bursty-neural-decoder) outperforms both the standard Turbo decoder and its heuristic counterparts (erasure- and saturation-thresholding) at σ²_b = 3.5.
• The neural decoder exhibits more modest confidence estimates in likelihood ratios compared to BCJR, which reduces error cascade effects and improves robustness under noise model shifts.
• Positional analysis shows that while both neural and BCJR decoders make errors concentrated around the 50th bit in bursty noise, the neural decoder’s lower confidence magnitudes prevent widespread error propagation.