A terahertz (THz) photonic crystal fiber (PCF) with an ultra-wide bandwidth and single-polarization-single-mode (SPSM) operation is designed and analyzed. Two air slots are introduced in the core region and epsilon-near-zero (ENZ) material is deposited in four specific air holes in the cladding of the PCF. The design achieves significantly different electric (E)-field distributions of the X-polarized (XP) and Y-polarized (YP) modes. The E-field components overlapping the ENZ material are attenuated because it is lossy. Gain material is then deposited in a rectangular slot in the core center to provide amplification of the E-field components overlapping this gain region. Changing the dimensions of the PCF modifies the amplification and attenuation rates to the wanted XP mode, the unwanted YP mode, and any unwanted higher order (HO) modes. The amplification of the wanted mode and the attenuation of the unwanted modes are maximized through optimization. The result is a PCF with an ultra-wide SPSM spectrum of 0.53 THz, from 1.00 to 1.53 THz. The minimum loss difference (MLD) across this bandwidth between the wanted mode and any unwanted modes is over 7.4 dB/cm. To the best of our knowledge, this is the widest SPSM bandwidth of a PCF fiber reported in THz regime.