Performance of coherent optical PSK systems with Costas loop in receiver for nonlinear second-order PLL model

Using the nonlinear second-order PLL (phase-locked loop) model, the authors consider the performance of the coherent optical PSK (phase-shift keying) systems with Costas loop in receiver, for the first time. The shot noise of the corresponding PIN photodiodes is described through the signal-to-noise ratio in the loop bandwidth, and the laser phase noise is described through the normalized frequency fluctuations instead of the phase ones. The analysis is carried out for both homodyne and heterodyne types of receivers. The theory and results presented in this article can be applied to the analysis and optimization of performance in regions where the linear PLL model is not sufficient.