Weak gravitational lensing surveys using photometric redshifts can have their cosmological constraints severely degraded by errors in the photo-z scale. We explore the cosmological degradation versus the size of the spectroscopic survey required to calibrate the photo-z probability distribution. Previous work has assumed a simple Gaussian distribution of photo-z errors; here, we describe a method for constraining an arbitrary parametric photo-z error model. As an example we allow the photo-z probability distribution to be the sum of N_G Gaussians. To limit cosmological degradation to a fixed level, photo-z distributions comprised of multiple Gaussians require up to a 5 times larger calibration sample than one would estimate from assuming a single-Gaussian model. This degradation saturates at N_G ≈ 4 in the simple case where the fiducial distribution is independent of N_G. Assuming a single Gaussian when the photo-z distribution has multiple parameters underestimates cosmological parameter uncertainties by up to 35%. The size of required calibration sample also depends on the shape of the fiducial distribution, even when the rms photo-z error is held fixed. The required calibration sample size varies up to a factor of 40 among the fiducial models studied, but this is reduced to a factor of a few if the photo-z error distributions are forced to be slowly varying with redshift. Finally, we show that the size of the required calibration sample can be substantially reduced by optimizing its redshift distribution. We hope this study will help stimulate work on better understanding of photo-z errors.