Powerful constraints on theories can already be inferred from existing CMB anisotropy data. But performing an exact analysis of available data is a complicated task and may become prohibitively so for upcoming experiments with 10⁴ pixels. We present a method for approximating the likelihood that takes power spectrum constraints, e.g., "band-powers," as inputs. We identify a bias which results if one approximates the probability distribution of the band-power errors as Gaussian—as is the usual practice. This bias can be eliminated by using specific approximations to the non-Gaussian form for the distribution specified by three parameters (the maximum likelihood or mode, curvature or variance, and a third quantity). We advocate the calculation of this third quantity by experimenters, to be presented along with the maximum-likelihood band-power and variance. We use this non-Gaussian form to estimate the power spectrum of the CMB in 11 bands from multipole moment = 2 (the quadrupole) to = 3000 from all published band-power data. We investigate the robustness of our power spectrum estimate to changes in these approximations as well as to selective editing of the data.