Capacity-Approaching Transmission over 6375 km Using Hybrid Quasi-Single-Mode Fiber Spans

Increasing the capacity and spectral efficiency of high speed optical transmission over transoceanic lengths of single-mode fiber is a great challenge due to limited signal to noise ratio at the receiver. Capacity can be optimized by tailoring modulation formats. However, beyond 6 bits/symbol in four-dimensional space, it is still not clear if high dimensional modulation formats can outperform two-dimensional counterparts. In this work, using mutual information and generalized mutual information (GMI) capacity analyses of various modulation formats, it is shown that a signal constellation can be geometrically shaped to approximate an optimal Gaussian distribution with equiprobable signaling, thus approaching the Shannon limit closer than the standard square quadrature amplitude modulation (QAM). By reviewing the design rule of amplitude-phase shifted keying (APSK), gray-mapping 64APSK is found, being only 0.5 dB away in theory at a 8.4 b/s/Hz target spectral efficiency (SE). An experimental comparison verifies that 64APSK is about 0.5 dB better than 64QAM at <9 b/s/Hz four-dimenional GMI capacity, and have ∼3.6 dB optical signal-to-noise ratio implementation penalty compared with the Shannon limit at the target SE. Using these findings, 168 channels modulated with 24.8 Gbaud 64APSK are successfully transmitted over 6375 km using low-density parity check (23 090, 16 163, 0.7) codes for bit error correction. An SE of 8.3 b/s/Hz is achieved at the total C-band capacity of 34.9 Tb/s after nonlinearity compensation.