[Paper Review] Introducing 4D Geometric Shell Shaping for Mitigating Nonlinear Interference Noise

Sebastiaan Goossens, Yunus Can Gültekin+5 more|arXiv (Cornell University)|Jun 7, 2022

Optical Network Technologies50 references4 citations

TL;DR

This paper introduces 4D Geometric Shell Shaping (4D-GSS), a novel framework for designing non-uniform, multidimensional constellations that enhance nonlinear tolerance in optical fiber systems. By optimizing for mutual information, 4D-GSS achieves a 3% reach increase over PM-16QAM and outperforms both PM-16QAM and probabilistically shaped PM-16QAM in high-launch-power, nonlinear fiber channels, while maintaining low complexity through structured shell-based design.

ABSTRACT

Four dimensional geometric shell shaping (4D-GSS) is introduced as an approach for closing the nonlinearity-caused shaping gap. This format is designed at the spectral efficiency of 8 b/4D-sym and is compared against polarization-multiplexed 16QAM (PM-16QAM) and probabilistically shaped PM-16QAM (PS-PM-16QAM) in a 400ZR-compatible transmission setup with high amount of nonlinearities. Reach increase and nonlinearity tolerance are evaluated in terms of achievable information rates and post-FEC bit-error rate. Numerical simulations for a single-span, single-channel show that 4D-GSS achieves increased nonlinear tolerance and reach increase against PM-16QAM and PS-PM-16QAM when optimized for bit-metric decoding (RBMD). In terms of RBMD, gains are small with a reach increase of 1.7% compared to PM-16QAM. When optimizing for mutual information, a larger reach increase of 3% is achieved compared to PM-16QAM. Moreover, the introduced GSS scheme provides a scalable framework for designing well-structured 4D modulation formats with low complexity.

Motivation & Objective

• To close the nonlinearity-caused shaping gap in high-capacity optical fiber communication by designing 4D constellations with enhanced nonlinear tolerance.
• To develop a scalable, low-complexity framework for generating well-structured 4D modulation formats using geometric shaping principles.
• To evaluate the performance of 4D-GSS against conventional PM-16QAM and probabilistically shaped PM-16QAM in a 400ZR-compatible, high-nonlinear transmission environment.
• To investigate the impact of constellation structure and binary labeling on bit-metric decoding (R_BMD) and mutual information (MI) performance.

Proposed method

• The 4D-GSS framework designs constellations as shells in a 4D I-Q space, with points grouped by distance from the origin to enable structured, non-uniform spacing.
• The method uses geometric shaping to create constellations that are not Cartesian products of lower-dimensional constellations, enabling improved nonlinear interference noise (NLIN) mitigation.
• Optimization is performed using two metrics: bit-metric decoding rate (R_BMD) and mutual information (MI), with constraints to reduce complexity and ensure structural regularity.
• Binary labeling is applied to the 4D-GSS constellations, with bits assigned to control orthant selection, shell selection, and symmetric point selection within shells.
• Simulations are conducted in a single-span, single-channel 400ZR-compatible setup with high launch power to emphasize nonlinear effects.
• Performance is evaluated using achievable information rate (AIR), post-FEC bit-error rate, and metrics such as PAPR and kurtosis.

Experimental results

Research questions

• RQ1 Can a 4D geometric shaping framework be designed to outperform conventional PM-16QAM in nonlinear fiber channels?
• RQ2 How does optimizing for mutual information (MI) versus bit-metric decoding rate (R_BMD) affect the reach and performance of 4D-GSS constellations?
• RQ3 To what extent does the proposed GSS structure support good binary labeling, and how does poor labeling impact performance?
• RQ4 Can 4D-GSS achieve significant nonlinear tolerance gains while maintaining low implementation complexity?

Key findings

• When optimized for mutual information (MI), 4D-GSS-4 achieves a 3% reach increase over PM-16QAM in a 400ZR-compatible, high-nonlinear fiber setup.
• When optimized for bit-metric decoding (R_BMD), 4D-GSS-4 achieves a 1.7% reach increase over PM-16QAM, indicating a performance gap due to suboptimal binary labeling.
• The MI-optimized 4D-GSS-4 constellation shows a 33% lower PAPR and 6% lower kurtosis than PM-16QAM, contributing to improved nonlinear tolerance at high launch powers.
• The bit-wise mutual information (MI) analysis reveals that bit b7 in 4D-GSS-4 has the lowest MI, indicating a labeling inefficiency that limits performance gains.
• Despite the labeling penalty, 4D-GSS-4 outperforms w4-256 at high launch powers (>14 dBm), where w4-256’s MI rapidly degrades due to high PAPR and kurtosis.
• The framework enables a scalable, low-complexity design of 4D constellations with structured shells, offering a promising path for future high-spectral-efficiency, nonlinear-tolerant optical transmission.