THE PHOTON UNDERPRODUCTION CRISIS

By virtue of its physical and chemical simplicity, the intergalactic medium (IGM) serves as an exquisite calorimeter, recording the instantaneous ionizing emissivity and heat produced by cosmic sources at each epoch. At z < 6 the IGM is highly ionized (Gunn & Peterson 1965), with a fluctuating residue of neutral hydrogen: the Lyα forest (Lynds 1971). After nearly two decades, the Lyα forest remains the most well-understood and robust prediction of cosmological hydrodynamic simulations (e.g., Cen et al. 1994; Zhang et al. 1995; Hernquist et al. 1996; Miralda-Escudé et al. 1996; Rauch et al. 1997). This robustness arises because the Lyα forest is dominated by gas at moderate overdensity; gas that traces the dark matter (with only mild impact by pressure forces) and whose temperature is governed by the simple processes of photoionization heating and adiabatic cooling (e.g., Weinberg 1998; Peeples et al. 2010a). It is this simplicity that makes the calorimeter reliable: IGM models suggest the neutral fraction of gas at the cosmic mean density at z ∼ 3 is n_HI/n_H ∼ 10^−5.5 (e.g., Kollmeier et al. 2003), and this low neutral fraction must be maintained by the background of photoionizing radiation produced by quasars and star-forming galaxies (Miralda-Escude & Ostriker 1990; Haardt & Madau 1996), possibly augmented by other undiscovered sources.

Determining the intensity of the ultraviolet background (UVB)—specifically the hydrogen photoionization rate, Γ_HI, is non-trivial. Because of its low surface brightness, the metagalactic UVB is not directly measured but is inferred through multiple independent channels, such as the quasar proximity effect (e.g., Bechtold et al. 1987; Scott et al. 2002) or the low surface brightness emission from the outskirts of galactic disks (e.g., Adams et al. 2011). The measurements are intrinsically difficult and subject to significant uncertainties and challenges (e.g., anisotropic QSO radiation, uncertain local gas densities). Alternatively, one can predict the intensity and spectrum of the UVB by synthesizing measurements of all possible sources of ionizing flux and the absorption and re-emission of UV radiation by the IGM and high column density absorbers (Haardt & Madau 1996, 2001, hereafter, HM01). While this procedure relies on a host of observational inputs, the most uncertain is the fraction f_esc of ionizing photons that escape from star-forming galaxies. Direct (and difficult) ionizing continuum measurements suggest that f_esc ∼ 10% at z ∼ 3–4 (e.g., Shapley et al. 2006; Vanzella et al. 2010) and is substantially lower at z < 1 (e.g., Bridge et al. 2010; Barger et al. 2013).

Exploiting our theoretical understanding of the IGM provides a third avenue for probing Γ_HI. By forcing a match between the (more easily observed and well understood) opacity of the Lyα forest and that from a theoretical IGM (typically taken directly from simulations) we have an independent determination that can be compared with both indirect measurements and the predicted UVB. There has historically been excellent agreement between the predicted UVB and the value inferred from the mean opacity (Davé & Tripp 2001). It was precisely the comparison between the predicted and observed forest opacity that provided strong arguments (now confirmed) for a "high" baryon density and low associated deuterium abundance (Rauch et al. 1997; Weinberg et al. 1997).

In this Letter, we demonstrate that this excellent agreement no longer holds at low redshift. Specifically, we will show a factor of five discrepancy between the Γ_HI predicted by the most sophisticated model of UVB evolution (Haardt & Madau 2012; hereafter HM12) and the value required to reproduce observed properties of the Lyα forest. We show in Figure 1 the predicted Γ_HI from HM12 (black solid line) compared to observational determinations. The dashed line shows an independent model of the UVB from Faucher-Giguère et al. (2009), which overall is quite similar to that of HM12. The large open star is the value reported here.

Zoom In Zoom Out Reset image size

We take advantage of new measurements of the column density distribution (CDD) of the low-redshift Lyα forest from Cosmic Origins Spectrograph (COS) observations (Danforth et al. 2014) to determine Γ_HI by comparison with cosmological simulations of the IGM. We will further show that our conclusions would be very similar if we instead use the mean decrement as our observational measure.

After describing the cosmological simulation that we use (Section 2) and inferring the value of Γ_HI required to match the observed CDD (Section 3), we discuss (in Section 4) possible resolutions to the discrepancy between our inferred Γ_HI and the value predicted by HM12, the "photon underproduction crisis" (PUC) of our title. None of these resolutions alone appears entirely satisfactory. The most exciting possibility is that this discrepancy is probing exotic sources of ionizing photons or novel heating mechanisms in the diffuse IGM operating far above the usual theoretical expectations.

The simulations in this Letter are performed using a modified version of the Gadget2.0 smoothed particle hydrodynamics (SPH)+nbody code (Springel 2005; Oppenheimer & Davé 2008). We include radiative cooling from primordial composition gas and metals assuming ionization equilibrium. Our main simulation is performed within a box of 50 h⁻¹ Mpc on a side (co-moving) with 2 × 576³ particles and LCDM cosmological parameters Ω_m = 0.25, Ω_Λ = 0.75, Ω_b = 0.044, h = 0.70. Our principal results are obtained from the z = 0.1 output of this simulation.

We stress at the outset that the results presented here should be robust to the adopted simulation code. At the physical scales and conditions of the Lyα forest, numerical issues of co-existing multi-phase media and under-or-over resolved hydrodynamic instabilities do not play a significant role. While our simulation adopts the momentum-driven wind ("vzw") formalism (described in detail by Oppenheimer & Davé 2008), and these winds can significantly heat the IGM close to galaxies, the overall Lyα forest properties of our simulations are largely insensitive to the adopted galactic wind prescription (e.g., Kollmeier et al. 2006; McDonald et al. 2006; Davé et al. 2010).

We extract 2500 spectra from our simulation boxes along a fixed grid using the Specexbin software package (Oppenheimer & Davé 2006). At each pixel position in the simulated sightline, the physical properties of the gas are computed (density, temperature, velocity, and metallicity) by considering the contribution of each overlapping SPH particle. The "physical space" spectra are converted to redshift space by incorporating the gas peculiar velocity, thermal motions, and the Hubble flow along the line of sight. The spectra are convolved with COS resolution and noise is added to produce signal-to-noise ratio (S/N) = 100. We use AutoVP (Davé et al. 1997) to analyze individual absorption systems in a way to mimic, as closely as possible, the corresponding observational procedure.

Figure 2 shows the CDD, defined here to be the mean number of absorbers per logarithmic interval of column density per unit redshift path length, for the simulated Lyα forest created with the HM12 and HM01 backgrounds. The COS CDD measurements from Danforth et al. (2014) are shown as magenta symbols. Earlier CDD measurements by (Lehner et al. 2007; green symbols) are in excellent agreement at column densities above 10¹³ cm⁻² and are likely affected by incompleteness in the lowest column density bin. The simulated IGM is a thicker beast with the HM12 UVB determination, overpredicting the observed CDD by a factor of ≈3.3 over the column density range 10¹³–10¹⁴ cm⁻². With the HM01 background, the simulated CDD is slightly but consistently above the COS measurements.

Zoom In Zoom Out Reset image size

For a highly ionized system in photoionization equilibrium, the neutral column density is inversely proportional to the photoionization rate, N_HI∝1/Γ_HI. The slope of the simulated and observed CDDs in Figure 2 is approximately $N_{\rm HI}^{-0.75}$, so reducing the amplitude of the CDD by a factor of 3.3 requires lowering the column densities of individual systems by a factor of 3.3^1/0.75 ≈ 5. The blue histogram in Figure 2 shows the CDD computed from the simulation with the HM12 UVB increased in amplitude by a factor of five, and the agreement with the Danforth et al. (2014) data is now very good. There is some tension in the highest column density bin, where line saturation effects are beginning to become important and details of fitting algorithms, observational noise, and spectral resolution may play a larger role.

The mean flux decrement for the simulated spectra is 〈D〉 = 0.05, 0.024, and 0.018 for the HM12, HM01, and boosted HM12 backgrounds, respectively. These can be compared to the value of 〈D〉 = 0.020 ± 0.003 found by Kirkman et al. (2007) from observed Lyα forest spectra at low redshift. The mean decrement results lead to exactly the same conclusion as the CDD analysis, requiring a factor of ≈5 boost in the amplitude of Γ_HI relative to the HM12 prediction. The CDD comparison has the virtue of focusing on systems that are the most physically simple¹² (i.e., moderate overdensity of ≈5–20 at z = 0) and the most straightforward to measure observationally, but the similarity of the results demonstrates the robustness of the conclusion.

The large mismatch between the low-redshift photoionization rate predicted by HM12 and inferred from matching the observed CDD challenges¹³ our current understanding of the sources of the UVB, the physical state of the IGM, or both. We now discuss a number of possible resolutions to this discrepancy.

4.1. Galaxy Escape Fraction

4.2. Quasar Emissivity

4.3. Mean Free Path

4.4. Extra Heating of the IGM

It is possible that there is a source of IGM heating that is not accounted for by our simulations. In conventional models, the temperature–density relation of the diffuse IGM is determined by the balance between heating by photoionization and adiabatic cooling by the expansion of the universe (e.g., Katz et al. 1996; Hui & Gnedin 1997). For temperatures T ≲ 2 × 10⁵ K, the hydrogen recombination coefficient scales as T^−0.7, so a hotter IGM produces a thinner Lyα forest. The H i column density of a given system scales as T^−0.7Γ_HI⁻¹, so a factor of five increase in Γ_HI achieves the same effect as a factor of 5^1/ 0.7 ∼ 10 increase in temperature.

To examine the amount of heating required to resolve this discrepancy, we have carried out the simple experiment of boosting the temperature of all SPH particles in the simulation by a factor of four before extracting Lyα forest spectra with the HM12 UVB, and we find that the resulting CDD is similar to but slightly above that of the HM01 model shown in Figure 2, as expected by comparing 4^0.7 = 2.6 to the factor of 3.7 difference in Γ_HI between HM01 and HM12.

A harder ionizing background spectrum leads to a hotter IGM temperature because the average residual energies of photoelectrons are higher. However, producing even a factor of four increase in temperature would require an implausibly hard UVB spectral shape, so the extra heating solution would require some mechanism other than photoionization. Broderick et al. (2012) have proposed one such mechanism, powered by TeV gamma-ray emission from blazars. These high energy gamma rays annihilate and pair-produce through interactions with extragalactic background photons. In principle, this process can drive a plasma instability, which locally dissipates the (high) energy of the produced pair, thereby heating the low-density IGM. This heating results in an inverted temperature–density relation in the forest because it deposits more energy per particle (Puchwein et al. 2012).

To investigate the potential impact of blazar heating, we have performed a suite of simulations implementing the "intermediate" prescription for blazar heating proposed by Puchwein et al. (2012), and confirmed that we reproduce their results for the T–ρ relation at z = 3 and z = 0.¹⁵ This simulation was performed at slightly lower resolution, with 2 × 384³ particles in a 48 h⁻¹ Mpc box, and we ran a matched simulation without blazar heating for comparison. The blazar heating has the anticipated effect of thinning out the Lyα forest at low redshift for both the HM12 and HM01 UVB models as shown in Figure 3. The shallower CDD for the blazar simulation is a result of the inverted ρ–T relation, which reduces the neutral fraction in the lower density gas responsible for lower column density absorbers. This highlights the potential to use the low-redshift Lyα forest CDD to probe the impact of blazar contributions to the heating of the low-density IGM. However, blazar heating is clearly insufficient to reconcile the HM12 background with the observed CDD. If we instead apply the HM01 background to the blazar simulation we get reasonably good agreement with the Danforth et al. (2014) CDD, but most of this improvement comes from the larger Γ_HI of HM01.

Zoom In Zoom Out Reset image size

4.5. Density Structure of the IGM

4.6. New Sources of Ionizing Photons

The factor of five discrepancy between the value of Γ_HI required to match cosmological models of the z = 0 IGM to the observed mean decrement and CDD and that predicted by state-of-the-art models for the evolution of the extragalactic UVB (HM12) highlights a significant gap in our current understanding of the sources of the UV background or the structure of the IGM, or both. We have discussed a number of possible resolutions, no one of which appears satisfactory. The least radical solution is to increase the mean f_esc at low redshift such that galaxies dominate the emissivity and simultaneously boost the quasar emissivity, though both of these changes oppose our current understanding of these sources. For the undaunted, extra photons from decaying dark matter or a drastic change to the physical structure of the IGM as predicted by LCDM may also be the resolution to the PUC.

We thank Rik Williams and Andy Gould for helpful discussions and comments. We thank the NSF (OIA-1124453), NASA (NNX12AF87G, NNX10AJ95G, and HST-AR-13262), and the Ahmanson Foundation for grant support.