This work outlines a design process to optimize concentrating photovoltaic (CPV) system performance based on the average irradiance characteristics of the installation site. As the geometric concentration of an optical element increases, its acceptance angle generally decreases. In locations with significant diffuse irradiance, the optical efficiency of the installed CPV system will be reduced. The exact relationship between irradiance conditions, acceptance angle, and optical efficiency is quantified with a radiometric model for two- and three-dimensional concentrating systems. Expected power output of the installation is calculated using on-site optical efficiency and global irradiance. The price of a CPV system is explored by using acceptance angle and geometric concentration to find a ratio between PV and concentrator material usage. With system cost and power output quantified, overall performance of CPV systems is discovered and compared, allowing one to choose the most effective design. The entire methodology is explored using an example that highlights the impact of site irradiance on the final design decision.