MAPPING AND SIMULATING SYSTEMATICS DUE TO SPATIALLY VARYING OBSERVING CONDITIONS IN DES SCIENCE VERIFICATION DATA

, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , and

Published 2016 October 17 © 2016. The American Astronomical Society. All rights reserved.
, , Citation B. Leistedt et al 2016 ApJS 226 24 DOI 10.3847/0067-0049/226/2/24

0067-0049/226/2/24

ABSTRACT

Spatially varying depth and the characteristics of observing conditions, such as seeing, airmass, or sky background, are major sources of systematic uncertainties in modern galaxy survey analyses, particularly in deep multi-epoch surveys. We present a framework to extract and project these sources of systematics onto the sky, and apply it to the Dark Energy Survey (DES) to map the observing conditions of the Science Verification (SV) data. The resulting distributions and maps of sources of systematics are used in several analyses of DES–SV to perform detailed null tests with the data, and also to incorporate systematics in survey simulations. We illustrate the complementary nature of these two approaches by comparing the SV data with BCC-UFig, a synthetic sky catalog generated by forward-modeling of the DES–SV images. We analyze the BCC-UFig simulation to construct galaxy samples mimicking those used in SV galaxy clustering studies. We show that the spatially varying survey depth imprinted in the observed galaxy densities and the redshift distributions of the SV data are successfully reproduced by the simulation and are well-captured by the maps of observing conditions. The combined use of the maps, the SV data, and the BCC-UFig simulation allows us to quantify the impact of spatial systematics on N(z), the redshift distributions inferred using photometric redshifts. We conclude that spatial systematics in the SV data are mainly due to seeing fluctuations and are under control in current clustering and weak-lensing analyses. However, they will need to be carefully characterized in upcoming phases of DES in order to avoid biasing the inferred cosmological results. The framework presented here is relevant to all multi-epoch surveys and will be essential for exploiting future surveys such as the Large Synoptic Survey Telescope, which will require detailed null tests and realistic end-to-end image simulations to correctly interpret the deep, high-cadence observations of the sky.

Export citation and abstract BibTeX RIS

1. INTRODUCTION

The Dark Energy Survey (DES, The Dark Energy Survey Collaboration 2005) began in 2012 and over at least five seasons will observe ∼5000 square degrees of the Southern sky in five optical bands (grizY). When completed, DES will cover a volume of the universe up to 20 times greater than the Sloan Digital Sky Survey (SDSS, Gunn et al. 2006), which is the largest optical survey to date. Hence, DES will provide an enormous legacy data set that will be useful in a range of astrophysical and cosmological studies. It is thus essential to develop approaches to robustly analyze DES data while accounting for statistical and systematic uncertainties.

The primary science goal of DES is to uncover the nature of dark energy using a combination of cosmological observables. In addition to expansion rate measurements using supernova light curves, DES will rely on probes of the growth rate such as the clustering and gravitational lensing of galaxies and clusters of galaxies. Exploiting these observables to probe dark energy requires exquisite control over the spatial coverage and calibration of the survey. Spatial fluctuations in the depth or quality of the data (e.g., the properties of the sky noise, the photometry, or galaxy ellipticity measurements) can impact the galaxy catalogs and lead to systematic biases in cosmological analyses. All ongoing and future surveys will be limited by our ability to identify and mitigate such systematics.

Establishing an exhaustive list of the sources of potential systematics in cosmological measurements is beyond the scope of this paper. However, it is worth recalling that systematics in clustering and cosmic shear studies are mostly rooted in astrophysical foregrounds (extinction by dust or obscuration by bright stars), observing conditions (e.g., seeing, sky noise, airmass), or processing and calibration (such as the quality of the photometry or the point-spread function). These systematics affect the probability of detecting sources and their properties, yielding nontrivial distortions in the reduced data, in particular for the galaxy catalogs. In DES, various efforts are dedicated to modeling or capturing the complicated transfer function connecting the raw data to the final galaxy catalogs. For instance, the Ultra Fast Image simulator (UFig, Bergé et al. 2013) is used to create simulated DES images which are then processed in a manner similar to the real data. This approach has been employed, e.g., to characterize systematics in shear measurements (Bruderer et al. 2015). UFig was also interfaced with the BCC N-body simulations (Busha et al. 2013) by Chang et al. (2014) in order to forward-model the survey transfer function with the known underlying astrophysics and cosmology. In this paper, we test this transfer function and investigate how well BCC-UFig is able to reproduce the physical characteristics (e.g., redshift distributions) and systematics (e.g., spurious galaxy density fluctuations) found in the DES Science Verification (SV) data. By contrast, Balrog 45 (Suchyta et al. 2016, used in Melchior et al. 2015) takes the approach of populating real DES images with simulated galaxies in order to measure the effective transfer function of the survey. These complementary efforts will be improved in the coming years to fully exploit DES data.

The observing conditions and astrophysical foregrounds unavoidably vary across the survey footprint (e.g., nightly variations of seeing or color reddening by Galactic dust). This paper focuses on mapping these sources of systematics onto the sky. This operation is analogous to the construction of foreground templates for the analysis of cosmic microwave background data (e.g., Tegmark 1997; Slosar et al. 2004; Ade et al. 2014). Such templates have been used in numerous analyses of single-epoch surveys like SDSS, in particular, for galaxy and quasar clustering measurements (e.g., Tegmark et al. 1998; Scranton et al. 2002; Ho et al. 2012; Ross et al. 2012; Leistedt et al. 2013; Agarwal et al. 2014; Leistedt & Peiris 2014), and are being used in analyses of SV data (e.g., Becker et al. 2015; Jarvis et al. 2015; Vikram et al. 2015; Crocce et al. 2016; Giannantonio et al. 2016). More generally, templates of potential sources of systematics can be used to carry out null tests with the data or model their contamination. As detailed below, multi-epoch surveys such as DES require a dedicated projection framework. In addition, the extracted observing conditions can be incorporated in image simulations to mimic the survey properties.

This paper is organized as follows. In Section 2, we present a scheme to map multi-epoch survey data onto the sky and apply it to DES–SV data. We present and analyze the resulting maps of sources of observational systematics. In Section 3, we use these maps to analyze the SV data and the BCC-UFig simulations, and show the impact of observational systematics on the measured galaxy densities and on the redshift distributions inferred using photometric redshifts. In Section 4, we conclude and discuss the impact and future extensions of this work.

2. MAPPING THE PROPERTIES OF DES–SV IMAGES

2.1. Geometrical Projection

Mapping potential sources of systematics, such as observing conditions, is a routine operation in modern galaxy surveys. For the SDSS, this mapping was relatively straightforward since SDSS was a single-epoch survey. Therefore, direct mapping between sky position and images could be established46 (e.g., Ross et al. 2011, 2012; Leistedt & Peiris 2014). In other words, any of the properties of SDSS images (e.g., seeing) directly project onto the sky. This is no longer the case for DES, which is a multi-epoch survey where several single-epoch CCD images are processed and stacked into "coadd" images from which galaxies and stars are then extracted. The nominal depth in the main DES survey requires up to 10 tilings in each band, while deeper regions require an order of magnitude more (i.e., in the SN fields, which are dedicated to the DES supernova program). The coadding process is performed in non-overlapping regions called "tiles," which are 0.75 × 0.75 deg2 squares constructed to cover and uniquely decompose the entire DES footprint. As a consequence of the multi-epoch nature of DES, there is not a unique value of, e.g., seeing at each sky position, but rather a distribution of values corresponding to the coadded single-epoch images. This is illustrated in Figure 1, which shows the footprints and properties of a set of single-epoch images used in an arbitrary coadd, that is, part of the DES–SV data (described in the next section). The seeing, airmass, and background noise (as well as many other properties not shown here) exhibit strong fluctuations and correlations. Combined with the nontrivial geometrical overlap between these images, this demonstrates the need for a flexible projection framework. In the standard processing pipeline, these images are processed and coadded in tiles (black line of Figure 1) with the DESDM software, using the software packages described in Sevilla et al. (2011), Desai et al. (2012), and Mohr et al. (2012)47 The operations performed in these codes unavoidably mix the image properties across the coadds and affect the properties of the detected sources. The geometry of the DECam focal plane—a hexagonal shape with 62 science CCDs (Honscheid et al. 2008; Flaugher et al. 2015)—may also be imprinted in the reduced data. Therefore, one would like to access the full distribution of the single-epoch properties and connect it to the coadds, catalogs, and sky coordinates.

Figure 1.

Figure 1. Upper panel: geometrical projection of the i-band single-exposure images coadded in an arbitrary tile of the DES–SV data (black contour). The colors correspond to different single-epoch pointings, with the relevant CCDs shown as individual rectangles. Lower panels: properties of the same set of CCDs exhibiting significant variations and correlations. The nontrivial, spatially varying geometrical overlap and image properties will result in spatially varying systematics when analyzing the galaxy catalogs.

Standard image High-resolution image

To construct sky maps of the single-epoch properties, we proceed as follows. We first connect the single-epochs and coadds, and keep track of which images were processed and coadded by the DESDM software. We then resolve the geometry of all images so that a given position on the sky is connected to a single coadd image and to a set of single-exposure CCDs. This is realized by accessing the images individually and using the WCS48 transformations to convert local image coordinates into equatorial coordinates. We also make sure these transformations match the procedures used in the DES software.49 Finally, we employ the HEALPix pixelization (Górski et al. 2005) and connect the tree of geometrically resolved images to HEALPix pixels on the sky. All HEALPix maps shown in this paper use ${N}_{\mathrm{side}}=4096$, providing a pixel area of 0.74 arcmin2.

The previous construction gives access to the full joint distribution of single-epoch and coadd image properties on the sky. This is a complicated object since each HEALPix pixel contains a vector of image properties. As mentioned before, a crucial product is the projection of this joint distribution into scalar sky maps. This requires the computation of one value (such as a summary statistic) per pixel, e.g., compressing the vector of seeing values in each pixel into mean, median, standard deviation, or even minimum and maximum values. This process can be done for any quantity of interest with arbitrary weights. This is how any potential source of spatial systematics arising from single-epoch images can be mapped onto the sky. Figure 2 shows the result of projecting some of the properties of the images of Figure 1. We see that the geometry of the CCDs as well as the relative orientations of the focal planes for the various exposures strongly affect the coverage and mean properties of the survey.

Figure 2.

Figure 2. Projection of the properties of the single-epoch images of Figure 1, showing how the time fluctuations and correlations are converted into spatial fluctuations. ADUs are Analog-Digital Units. The total exposure time is proportional to the CCD count; each image was taken with a 90 s exposure time.

Standard image High-resolution image

2.2. Application to DES–SV Data

SV data refers to the testing data acquired between 2012 November and 2013 February, processed by the "SVA1" version of the DESDM pipeline (B. Yanny et al. 2015, in prepration) and consisting of 858 coadd tiles, 665 of which have data in all five grizY bands. The SV data cover more than 300 deg2 in total, split into contiguous regions of interest: the large SPT-E and SPT-W regions (≈200 and 50 deg2, respectively), the RXC J2248, Bullet, and El Gordo known rich clusters (≈10 deg2 each), COSMOS (≈6 deg2), and the Supernovae fields SN-E, SN-X, SN-S, and SN-C (≈10 deg2 each). The footprint of DES–SV is shown in Figure 3 with the various fields labeled.

Figure 3.

Figure 3. DES–SV footprint partitioned into several discontinuous regions, with the largest being the SPT-E and W fields (≈200 and 50 deg2, respectively). The small red regions contain objects where spectroscopic redshifts are available, which are used to train the photometric redshift estimation codes, as discussed in Section 3.

Standard image High-resolution image

To create maps for the full SV data set, we create a symbolic tree of CCD and coadd images, resolve their geometries, and project them into HEALPix maps. However, unlike the illustration shown in Figures 1 and 2, we now crop the projection to each tile (the black contour of Figure 1) since the DESDM software separately processes tiles using the stacks of CCD images. We perform this projection in the full SV area and stitch the projected coadds to assemble the full SV footprint. In terms of outputs, we project the following quantities in the five grizY bands: airmass, seeing, sky brightness, sky sigma (defined later in this paragraph), and exposure time. These can all affect the quality of the photometric measurements (see, e.g., Li et al. 2016). We compress the multi-epoch information into average and total maps (e.g., mean seeing and total exposure time). For the former, a natural choice would be to take the uniformly weighted average in each HEALPix pixel. However, this choice is probably too simplistic because, in practice, images are coadded using weights derived from the flux variance. More precisely, the DESDM pipeline provides a "weight" or "variance" map for each single-epoch image, which is the inverse variance of the flux in each pixel, a quantity coined "sky sigma" and denoted by σi in the remainder of this paper. For an image i and a given pixel, the value of sky sigma depends on a number of parameters, including the flux itself, the gain of the amplifier, the readout noise, the bias correction, and the flat-fielding. These quantities are measured by DESDM and yield an estimate of a variance map for each image. Single-epoch images are coadded using these maps such that the coadded flux is the weighted average over all exposures,

Equation (1)

in each coadd pixel, where ${w}_{i}={({p}_{i}^{2}{\sigma }_{i}^{2})}^{-1}$. The extra pis are rescaling factors to enforce a common photometric calibration to the single-epoch fluxes. They read

Equation (2)

where mZi is the zero-point magnitude of the single-epochs and mZ is that of the coadd image. The variance of the total flux in each pixel of the coadd image is given by

Equation (3)

A detailed discussion of these quantities is beyond the scope of this paper, but we note that ${\sigma }_{{\rm{tot}}}^{2}$ is proportional to the magnitude limit of the survey. In the above formulae, we omitted the pixel indexing in σi, but the coadding and the evaluation of ${\sigma }_{{\rm{tot}}}^{2}$ must be performed pixel by pixel across the coadd image. The technicalities of this process (including the projection and coadding) are handled by the SWARP software (Bertin et al. 2002). However, the projection formalism presented above can be used to quickly estimate ${\sigma }_{{\rm{tot}}}^{2}$ (and for example construct approximate magnitude limit maps). For this purpose, we compute an average sky sigma per single-epoch CCD image, defined as the pixel average of σi across the CCD. Rather than computing σi and ${\sigma }_{{\rm{tot}}}^{2}$ per pixel, we only need to calculate σi per CCD and ${\sigma }_{{\rm{tot}}}^{2}$ in the distinct regions of image overlap, as shown in Figure 1. Averaging sky sigma across CCDs yields a significant reduction of the complexity of the full projection, which needs to be performed for the five bands for a number of quantities of interest, using several hundred thousand single-epoch images. Finally, any quantity of interest can be averaged using the same weights wi (which we call "sky sigma weights"), which is more useful than the unweighted average. The effective seeing of the coadd images is better approximated by the sky-sigma-weighted mean since the coadds are based on these weights.

Using CCD-averaged quantities significantly speeds up the projection and leads to a more flexible and faster algorithm for producing maps of quantities of interest with various weighting schemes (e.g., mean, sky sigma, or exposure weighted mean, median, etc.). The relative difference between maps constructed with and without the CCD-averaging approximation is of the order of 1%–3%, which is well below the precision needed for using such maps in large-scale null tests. Sub-CCD fluctuations might prove useful for small-scale null tests and can be projected using MANGLE (Hamilton & Tegmark 2004; Swanson et al. 2008). However, the present work shows that large-scale fluctuations are correctly captured by using CCD-averaged properties. Our tests indicate that the quality of this approximation improves with the survey area and number of exposures.

A number of maps were constructed for the DES–SV data in order to capture the spatial fluctuations of the observing conditions and other observational quantities. These maps are used in numerous SV analyses to perform spatial null tests with the data (e.g., Becker et al. 2015; Jarvis et al. 2015; Vikram et al. 2015; Crocce et al. 2016; Giannantonio et al. 2016). Figure 4 shows some of the main maps for the i band: the total exposure time, the mean sky sigma, and total sky sigma, as well as the minimum, maximum, and mean seeing. All of the quantities were calculated according to the previous scheme, i.e., the weighted average method and the sky sigma weights, with the exception of the mean sky sigma maps. This is because the weighted sky sigma is equivalent to the total sky sigma described above. Showing both maps sheds light on the difference between adding the noise properties linearly or in quadrature. In the following, we analyze these maps and detail the implications for the analyses of SV data. We focus on the SPT-E and W regions since they are the largest contiguous regions of data.

Figure 4.

Figure 4. Maps of some of the main observational quantities (potential sources of systematics) in the SPT-E and W fields (top and bottom of each sub-panel). The HEALPix maps are produced at ${N}_{\mathrm{side}}=4096$, where each pixel is the mean value of the observed ${N}_{\mathrm{side}}=16384$ sub-pixels, in order to obtain more accurate values near the edges of the survey.

Standard image High-resolution image

2.3. Analysis of the DES–SV Observing Conditions

The maps shown in Figure 4 exhibit significant structure and features on all scales, mostly because DES data have three intrinsic scales on which their properties can vary: the size of the DECam focal plane (2.2 deg diameter field of view), the coadd tile (0.75 × 0.75 deg2), and the single CCD (0.3 × 0.15 deg2). In spite of the random offsets and overlap of the focal plane when obtaining images and coadding them, these three scales become imprinted in the projected observing conditions. For example, the focal plane geometry is clearly visible in the total sky sigma maps in a number of regions. This is due to a significantly lesser or greater number of observations, or to their respective noise levels (sky sigma). Also, the mean seeing map is affected by outliers, i.e., by extreme (low or high) values of seeing in the set of single-epochs, as shown in the min/max maps in the bottom of Figure 4. The rectangular CCD geometry is also visible in the maps, especially near the edges. In addition, the observing properties of the 62 CCDs in a given single-epoch are very correlated since they experience quasi-identical observing conditions. By contrast, correlations between exposures are due to proximity in time, for example, if the observations were taken the same night. Finally, the tile edges are particularly visible in truncated regions or due to applying different zero-point magnitudes (e.g., the center of SPT-W, or the sharp transition in the upper part of SPT-E).

To identify which scales may be affected by the features described above, we compute the full-sky angular power spectra of the maps in Figure 4 (the full SV, not only the SPT-E and W regions). The results are shown in Figure 5; all of the spectra are made dimensionless and normalized such that ${\sum }_{{\ell }}{C}_{{\ell }}=1$ to clarify the comparison. As seen before, all of the maps exhibit significant power on all scales. The labels show which multipole ranges correspond to the typical scales of the SV fields, DECam focal plane, tiles, and CCDs. It is important to note that many of the features of Figure 5 are due to the sky coverage (i.e., the footprint) of the SV, not the correlations in the observed regions. This is emphasized by an extra line showing the power spectrum of the DES–SV footprint mask. Here, we do not deconvolve the effect of the mask on the power spectra because it typically redistributes the power between the modes. In the pseudo-spectrum estimation method, this deconvolution assumes flat priors on the power spectra, while quadratic maximum likelihood estimators can incorporate more flexible priors on the power spectra (see, e.g., Leistedt et al. 2013). This deconvolution would significantly affect the observed power spectra due to the small sky coverage of SV data. By contrast, not deconvolving the mask enables one to separate the scales affected by the survey coverage and by the observing conditions. The significant power in the ${\ell }\in [0,200]$ range is mostly due to the size and shape of the SV fields (all fields except SPT-E and W have approximately the size of the focal plane). In the other power spectra, any power in excess of the black line is due to structure within the fields, i.e., to the features described previously. As expected, airmass and seeing maps mostly have additional power on small scales. However, the sky sigma maps have much more power on all scales, particularly around the focal plane and coadd scales.

Figure 5.

Figure 5. Full-sky angular power spectra of some of the i-band observational systematics shown in Figure 4. Prior to power spectrum estimation, the maps were divided by their average values in order to obtain dimensionless Cs. The power spectrum of the DES–SV coverage mask (presented in E. Rykoff et al. 2015, in preparation; Crocce et al. 2016) is also shown in black. Any excess of power relative to the mask implies structure and features in the maps which can yield non-trivial contamination and systematics in the galaxy catalogs. We also indicate the characteristic scales affected by the geometry of the SV survey and DECam instrument.

Standard image High-resolution image

As seen in Figure 4, the maps of the observing conditions are correlated. Figure 6 shows the Pearson correlation coefficients of the DES–SV maps in the gri bands (calculated for the full SV area). These spatial correlations have two origins: time correlations between observations made closely spaced in time, and physical correlations between some of the properties. For example, the noise level and seeing are correlated.

Figure 6.

Figure 6. Correlation coefficients between some of the maps produced for the DES–SV data.

Standard image High-resolution image

In conclusion, the observing conditions fluctuate significantly over a wide range of scales and may affect the properties of the galaxies detected in DES coadd images. Any resulting spurious spatial correlations that propagate into the galaxy catalogs will need to be detected and eliminated. Typical techniques to mitigate these effects in clustering analyses include modeling the survey window function (e.g., Maddox et al. 1996; Blake et al. 2010), or using cross-correlations (Scranton et al. 2002; Ross et al. 2011, 2012; Ho et al. 2012; Crocce et al. 2016) or mode-projection (Leistedt et al. 2013; Leistedt & Peiris 2014) to correct or mask the spatial modes affected by the observing conditions. It is important to note that these approaches require the availability of accurate templates of the sources of systematics, which were precisely constructed in this section. We now turn to a concrete example of the use of these templates.

3. APPLICATION TO BCC-UFIG

BCC-UFig (Chang et al. 2014) is a framework of image-level simulations of the DES–SV data. It relies on the Ultra Fast Image Generator (UFig, Bergé et al. 2013) and the BCC cosmological simulations (the Blind Cosmology Challenge, Busha et al. 2013) in order to obtain realistic images of a galaxy survey simulated with a known cosmological model. BCC-UFig covers the SPT DES–SV region, and consists of 480 coadd images in the griz bands and 432 in the Y band. As detailed in Chang et al. (2014), these images were processed using the same software packages as the DESDM SVA1 pipeline. In this paper, we exploit the fact that the simulated BCC-UFig images integrate some of the actual observing conditions of the DES–SV data. In particular, the simulated coadd images incorporate the median values of the seeing, limiting magnitude, and magnitude zero point of the true DES–SV images. These quantities were obtained from the products presented in the previous section (i.e., maps of the median observing conditions, as opposed to the mean maps shown in Figure 4) by averaging each map over the surface of the tiles. The result of this smoothing is shown in Figure 7, and we comment on its effect on the galaxy catalogs below. The fact that the BCC-UFig is based on simulated coadd images and not on single-epoch images is the main difference from the real SV data. However, as discussed below, BCC-UFig reproduces most of the spatial systematics found in the data and relevant to clustering and weak-lensing analyses because these are due to fluctuations in observing conditions at scales larger than the coadds.

Figure 7.

Figure 7. Maps of the i-band SV observing conditions incorporated in the BCC-UFig simulation, obtained by smoothing the maps of Figure 4 in tiles. The analysis mask shows the region considered when measuring the redshift distributions and galaxy number densities presented below.

Standard image High-resolution image

Survey simulations like BCC-UFig can be used to test analysis techniques and pipelines in the presence of realistic systematics. For example, Chang et al. (2014) used BCC-UFig to compare the performances of various star-galaxy classifiers and to study the evolution of the observed galaxy and stellar densities as a function of some of the observing conditions (depth, seeing, Galactic latitude). Such tests cannot be performed at high significance in the real data due to the small size and sky coverage of the sample of spectroscopically confirmed galaxies (based on the COSMOS and SN fields, shown in red in Figure 3).

In this paper, we produce galaxy catalogs based on the BCC-UFig and compare them with the SV data catalogs. We mostly attempt to mimic the galaxy catalogs used in the clustering and cross-correlation analyses of SV Crocce et al. (2016) and Giannantonio et al. (2016). We first construct a multi-band catalog by cross-matching the positions of the objects detected in the griz bands. We then remove all objects with extreme fluxes or colors: x > 30 and $x-y\gt 3$ or $x-y\lt -1$, where x and y are mag_auto magnitudes in the griz bands measured by SExtractor. To select galaxies in this catalog, we use the "modest" classifier. As described in Chang et al. (2014) and Soumagnac et al. (2013), objects are labeled as galaxies by this classifier if they do not satisfy any of the following criteria: (mag_auto_i < 18 and class_star > 0.3), (spreadmodel_i + 3*spreadmodel_err_i < 0.003), or (mag_auto_i < 21 and mag_psf_i > 30). Finally, we only consider objects with 18 < mag_auto_i < 22.5 in the SPT-E region, and we split this galaxy sample into redshift bins using photometric redshifts. We now investigate the realism of these galaxy samples, first in terms of their redshift distributions.

3.1. Photometric Redshifts and Redshift Distributions

Photometric redshifts (photo-z)—redshifts estimated from broadband fluxes and colors—are one of the main sources of uncertainties in imaging surveys, and it is essential to reproduce this aspect of the data with BCC-UFig galaxies. We employ three photo-z codes: BPZ (Benítez 2000; Coe et al. 2006), TPZ (Carrasco Kind & Brunner 2013, 2014), and ANNz2 (Sadeh et al. 2015). These rely on very distinct algorithms that were tested on early SV data in Sánchez et al. (2014). They have also been used in the main SV clustering and cosmic shear analyses (Becker et al. 2015; Jarvis et al. 2015; Crocce et al. 2016; Giannantonio et al. 2016). For details on the three codes and an updated comparison using SV data, we refer the reader to Bonnett et al. (2015). Here, we only provide a brief summary of the three algorithms and focus on comparing the redshift distributions inferred from the SV data and the BCC-UFig simulation.

BPZ is a Bayesian template fitting photo-z code that relies on a set of calibrated template spectra which are redshifted and converted into template colors using the DES filters. This code computes a posterior probability for the redshift of each object, given its observed colors and errors, by fitting for all templates and marginalizing over the choice of template. By contrast, TPZ and ANNz2 are machine learning codes that must be trained on a representative sample of the data to infer a set of heuristic rules (i.e., a flexible data-driven model) to compute the redshift from the observed photometric colors. TPZ is a publicly available code50 based on prediction trees and random forests, while ANNz2 uses a combination of machine learning algorithms, including neural networks and k-nearest neighbors. The three photo-z codes deliver a redshift probability distribution function (PDF) and a photo-z point estimate, usually measured as the mean or the mode of the PDF.

We employ the BPZ, TPZ, and ANNz2 algorithms that were trained and calibrated on the SV data, more specifically, on the sample of galaxies presented in Bonnett et al. (2015) for which spectroscopic redshifts are also available (about 46,000 galaxies). This sample is shown in red in Figure 3 and was used to calibrate the BPZ template prior and train the TPZ and ANNz2 methods. Note that we only use mag_auto magnitudes and colors with BPZ and ANNz2, and we only include the magnitude errors in the training of TPZ (where they are used to perturb the magnitudes when re-training the prediction trees, in order to obtain reliable redshift posterior PDFs).

Following most analyses of SV (e.g., Crocce et al. 2016; Giannantonio et al. 2016), we create five BCC-UFig redshift samples by selecting objects with photometric redshift falling in a top hat window of size Δz = 0.2 in the range 0.2 < z < 1.2. We use the ANNz2 photo-z point estimates to bin our data in the redshift ranges, i.e., to select the objects that fall in each redshift bin. We then reconstruct the N(z) by stacking the redshift PDFs of the selected objects for the three codes. Figure 8  shows the redshift distributions of the SV data samples compared with their BCC-UFig counterparts. We recall that the BCC-UFig and SV data were subject to the same color and quality cuts, and were restricted to the same portion of the sky: the SPT-E region analysis mask shown in Figure 7. Hence, the inferred redshift distributions should match relatively well since the colors of the BCC-UFig galaxies were shown to correctly match those of the data in Chang et al. (2014). This is confirmed by Figure 8: when comparing the left and right panels, the features and relative amplitudes between the N(z) inferred from the three codes are very similar.

Figure 8.

Figure 8. Redshift distributions of the SV data (left) and BCC-UFig (right) catalogs obtained using the photometric redshift estimation methods trained on a spectroscopic sample of galaxies (see text for details). These are for the SPT-E fields shown above and are normalized such that $\int N(z){dz}=1$. By comparison, the variance obtained by randomly splitting the redshift samples is of the order of 0.01 in all redshift bins.

Standard image High-resolution image

An important difference between the left and right panels is that the true redshift distributions can be calculated for the BCC-UFig and be compared with the distributions inferred using photometric redshifts. Analyzing the detailed performance of the photo-z codes is beyond the scope of this paper; a full investigation in the context of the weak-lensing SV data samples is presented in Bonnett et al. (2015). However, the results of Figure 8 show that most of the features of data redshift distributions are recovered in the simulation. For instance, BPZ yields wider N(z)s than machine learning methods, but is less accurate near z ∼ 0.4 due to the layout of the DES grizY filters and the limitations of the set of template spectra. Also, the redshift distributions inferred by TPZ are narrower than the true underlying distribution. These features persist when selecting galaxies with BPZ or TPZ photo-z point estimates. Selecting with ANNz2 minimizes the width of the inferred N(z) from the three methods and reduces the amount of low-redshift outliers in the third bin.

The comparison of true and inferred redshift distributions is not trivial with the SV data given the small sample sizes of spectroscopically confirmed galaxies, especially at high redshift. For this reason, a realistic survey simulation like BCC-UFig is a powerful tool for testing critical analysis stages, such as photometric redshift estimation, in regimes that are difficult to explore with the data. More specifically, Figure 8 demonstrates that the features seen in the redshift distributions calculated for the SV data are compatible with and well-reproduced by the BCC-UFig simulation.

We now challenge an assumption made above (and in current SV analyses): the fact that the redshift distributions can be spatially averaged over a large area without accounting for systematics (here the entire SPT-E region). While the analysis of SV data is restricted to the most uniform regions, as shown by the analysis mask in Figure 7, these include unavoidable residual depth and quality fluctuations. The mean N(z) is the main quantity of interest for cosmological analyses. However, its variance due to statistical and systematic uncertainties must be evaluated in order to assess the robustness of theoretical clustering and gravitational lensing predictions using the N(z). As we will see below, the statistical fluctuations are small, as expected from the area and number density of objects. However, it is essential to test for residual spatial systematics in the inferred redshift distributions.

Here, we focus on testing the variability of the N(z) distributions due to residual depth fluctuations and observational systematics in the SPT-E region. For each of the quantities presented in the previous section (e.g., seeing), we compute the median value and use it to split each redshift sample into two subsamples. These cover different regions of the sky observed under different conditions. We compute the difference between the redshift distributions corresponding to the two patches, i.e., taking the redshift distribution of the galaxies in the region where the systematic is above the median value, and subtracting that of the galaxies in the other region. Figures 9 and 10 show these differences for the i-band seeing and exposure time, respectively; these are two significant sources of spatially varying depth in the SV data (e.g., Crocce et al. 2016). Importantly, the variance obtained by randomly splitting the redshift samples (instead of splitting based on observing conditions) is of the order of 0.01 in all redshift bins.

Figure 9.

Figure 9. Difference between the redshift distributions for mean seeing bins, i.e., computed for the good (low seeing) and bad (high seeing) regions.

Standard image High-resolution image
Figure 10.

Figure 10. Same as Figure 9, but for exposure time. However, note that the fluctuations go in the opposite direction since we compute low minus high values, which corresponds to bad and good regions for exposure time.

Standard image High-resolution image

These figures indicate that the N(z) differences are significant compared to the sample variance. This is as expected since good regions (e.g., low seeing or high exposure time) have lower noise and better photometry. As a consequence, the photo-z codes will have better overall quality and yield narrower redshift PDFs. Therefore, the derived redshift distributions when selecting objects in top hat redshift windows will be more accurate. In our difference convention, this translates into a positive bump surrounded by wells in Figure 9 (since we take the difference between low seeing minus high-seeing regions), and the opposite in Figure 10 (where we compute low exposure time minus high exposure time). This is indeed observed in most bins, even though this depends on the details of the photometric redshift estimation.

While the observed N(z) fluctuations are significant compared to the sample variance, they are small compared to the overall amplitudes shown in Figure 8 (less than 5% in all the cases we tested). This can be seen not only in the histograms of the true redshifts from BCC-UFig, but also in the N(z) inferred from the photo-z codes. In fact, in the right panels corresponding to BCC-UFig, these distributions follow the fluctuations of the true redshifts. This is not the case in all of the panels because low- and high-redshift objects suffer from other issues that make the comparison difficult. In particular, we did not re-weight the redshift distributions to adjust the color distributions of the training, validation, and data samples, as in Bonnett et al. (2015) and Sánchez et al. (2014). Such corrections do not affect the comparison between the data and the simulation.

This analysis provides an estimate of the order of magnitude of the N(z) fluctuations due to observational sources of systematics and spatially varying depth in the SV data. Provided that the mean N(z) is properly characterized, these fluctuations will not bias the cosmological analyses. However, because they are due to residual spatial systematics, they may cause other types of contamination in the galaxy catalogs. This is shown in Crocce et al. (2016) and in the next section, where seeing is found to spuriously correlate with the SV data and contaminate the clustering measurements.

3.2. Spatial Null Tests

We now turn to the spatial properties of the BCC-UFig redshift samples. Figure 11 shows the average galaxy density measured in the previous redshift samples as a function of a few sources of systematics (median exposure time, seeing, and sky sigma). We create these data points by jointly analyzing HEALPix maps (at ${N}_{\mathrm{side}}=4096$) of the galaxy redshift bins (SV data and BCC-UFig) and the maps of observing conditions presented in the previous sections. Prior to estimation, all maps are divided by their mean values, so that the observables are dimensionless and concentrated near the central values (1, 1) in the panels of Figure 11. The dynamical range explored by the galaxy densities in each panel depends on the observational quantity under consideration. For example, normalized seeing values are mostly concentrated between 0.9 and 1.1, while exposure times span a wider range, as can be verified in Figure 4. The error bars are obtained by jack-knife re-sampling in 50 sky regions, which is possible thanks to the large number of objects (greater than 104 in each region).

Figure 11.

Figure 11. Changes in the galaxy number densities (ngals) relative to the mean ($\overline{n}$) in each redshift bin, as a function of some observational properties, also normalized to their mean values. The other properties were found to cause negligible differences in the galaxy number densities. Both the SV data (black circles) and BCC-UFig (red squares) exhibit similar fluctuations, which are small but significant compared to the sample variance, calculated using jack-knife re-sampling in 50 sky regions.

Standard image High-resolution image

Analogous galaxy density measurements are shown in Suchyta et al. (2016) and Crocce et al. (2016) using the SV data. Figure 11 shows very similar trends and amplitudes despite using different maps (the median maps instead of the weighted mean maps). The most significant fluctuations are due to the r- and i-band seeing, particularly in the first and last redshift bins, in agreement with what is found in Suchyta et al. (2016) and Crocce et al. (2016). Other observational properties create similar but smaller fluctuations. Remarkably, the BCC-UFig redshift samples exhibit similar galaxy density fluctuations in most bins. In particular, Figure 11 shows that the characteristic features of the seeing and sky sigma fluctuations as a function of galaxy density are reproduced by BCC-UFig. This demonstrates that the simulation succeeds in capturing some of the galaxy density fluctuations caused by the systematics considered here. The remaining qualitative and quantitative discrepancies are likely due to the approximations adopted in the simulation. The most significant effect is likely the incorporation of observing conditions at the tile level instead of the single-epoch images: the current implementation limits the spatial resolution of systematics to relatively large scales.

More generally, it is interesting to quantify the extent to which the maps capture depth fluctuations in the data. This is because the effects described above—spurious spatial variations in the redshift distributions and galaxy densities—are usually corrected for or marginalized over in cosmological analyses. This is either done at the level of the survey window function or in the measured power spectra or correlation functions. We do not attempt to develop and validate such a model since this must be done in the context of a specific analysis at hand (e.g., clustering), which is beyond the scope of this paper. However, we demonstrate that the maps trace the main sources of systematics by showing that they strongly correlate with depth fluctuations and stellar contamination. Figure 12 shows the Pearson correlation coefficient of some relevant observing condition maps with the following: (1) a map of the stars misclassified as galaxies (by the "modest" classifier) in the BCC-UFig galaxy sample described above; (2) maps of the average i-band magnitude errors in the BCC-UFig and SV data ("Gold" catalog, see Crocce et al. 2016) in mag_auto_i magnitude bins. Figure 12 shows that the exposure time and total sky sigma maps strongly correlate with the magnitude errors in all bands and magnitude bins, in both the data and the simulation, demonstrating that the maps capture most of the depth fluctuations. In fact, a depth map of the SVA1 "Gold" catalog was constructed using the method described in detail in E. Rykoff et al. (2015, in preparation). Briefly, a coarse depth map is first constructed by fitting the magnitude–magnitude error relation of galaxies, exploiting the fact that the magnitude errors satisfy ${\sigma }_{m}\propto {\sigma }_{F}/F$, where F and σF are the galaxy flux and its standard deviation. This relation depends on the local limiting magnitude of the survey, which can be estimated in coarse HEALPix pixels where there are enough galaxies to obtain precise limiting magnitude estimates (but at low spatial resolution). This map is then refined by constructing a data-driven model of the depth based on the maps of the observing conditions presented here, which are available at very high resolution (using machine learning algorithms, see E. Rykoff et al. 2015, in preparation). The maps were also used in Crocce et al. (2016) to build a linear model of the spurious correlations observed in the angular correlation functions and correct for them.

Figure 12.

Figure 12. Pearson correlation coefficients between observing conditions, stars missclassified as galaxies in the BCC-UFig reduced data, and mean i-band magnitude errors (magerr_auto_i) in both the BCC-UFig and SVA1-Gold galaxy catalogs (in magnitude bins, using mag_auto_i). This shows that the maps of the observing conditions are significantly correlated with the stellar contamination and depth fluctuations in the SV data and simulations, therefore capturing the main sources of spatial systematics present in the galaxy samples.

Standard image High-resolution image

Note that the correlations between the noise and the magnitude errors are less significant in the simulation than in the data. This is due to the approximation highlighted previously: BCC-UFig is based on simulated coadd images, not on simulated single-epochs. Hence, systematics at scales smaller than the coadds are not resolved. The previous section showed that this approximation correctly reproduced systematics in the galaxy densities and redshift distributions, which are due to large-scale fluctuations of the observing conditions (e.g., seeing). However, fluctuations in the noise and depth can be significant on sub-coadd scales. This explains why the correlation coefficient between the map of total sky sigma and the magnitude errors in BCC-UFig is less significant than what was found in the data.

Finally, as shown in Figure 12, the observing condition maps correlate with the stellar contamination in the BCC-UFig samples. This test cannot be performed with the SV data since object types are only available for a small sample of spectroscopically confirmed sources, restricted to a small region of the sky, as shown in the previous sections. A large, realistic simulation such as BCC-UFig allows us to confirm that the observing conditions trace the main sources of spatial systematics in the galaxy samples, and can be used to model and remove them in cosmological studies.

4. CONCLUSIONS AND OUTLOOK

We detailed a method to extract and project the properties of multi-epoch galaxy surveys onto the sky, making use of the properties of the images and the HEALPix pixelization. We applied this technique to the DES–SV data and mapped the main sources of observational systematics, including the average properties of seeing, airmass, and sky sigma. These maps will be made publicly available in the forthcoming DES data releases, and are currently used in analyses of SV data (e.g., Becker et al. 2015; Jarvis et al. 2015; Vikram et al. 2015; Crocce et al. 2016; Giannantonio et al. 2016).

High-resolution maps of the observing conditions can be used as templates to identify, model, and mitigate spatial systematics or residual contamination in the data. As an illustration, we measured the galaxy densities and redshift distributions of DES–SV photometric redshift galaxy samples, and showed that they were significantly affected by the observational conditions of the survey due to depth and seeing fluctuations. These systematics are correctly mitigated in current SV analyses thanks to the sky masks and corrections to the two-point correlation measurements validated by stringent null tests (see, e.g., Becker et al. 2015; Crocce et al. 2016; Giannantonio et al. 2016). However, it will be increasingly difficult to keep them under control in future studies. For instance, restrictive sky masks can remove unreliable regions but often discard hard-won data and do not alleviate the need to treat spatial systematics in the retained regions. As the depth and sensitivity of the survey increase, these systematics will become increasingly significant compared to statistical errors. The power of masking or current correction techniques is also limited since they rely on templates and contamination models which are not validated against simulations.

One approach to resolve these issues, i.e., to assess the significance of systematics and validate the techniques to mitigate them, is to resort to realistic image simulations. All of the tests and analyses in this paper were performed in parallel on galaxy samples obtained by processing the BCC-UFig in the same way as the SV data and applying the same quality and selection cuts. These simulated galaxy samples include spatial systematics since the image simulations incorporate the actual SV observing conditions. Even with the approximation of simulating coadd images instead of single-epochs, we found that the principal effects of spatial systematics observed in the galaxy densities and redshift distributions were successfully reproduced by the BCC-UFig galaxy samples. Furthermore, the data and the simulation agreed quantitatively in many cases, showing that the current BCC-UFig simulation, even with known limitations, is sufficiently realistic to study a range of effects. The availability of the ground truth in the simulation (e.g., the true redshifts) allowed us to quantify the significance of the systematic density and redshift fluctuations for the first time, and to demonstrate that the observing condition maps capture systematics such as depth fluctuations and stellar contamination. Pursuing this route will be essential for the future DES studies, since these fluctuations will have to be carefully characterized and mitigated. Future versions of the BCC-UFig simulation will be more realistic and reproduce spatial systematics at higher resolution. Combining them with high-resolution maps of the observing conditions and the effective transfer function measured by Balrog (Suchyta et al. 2016) will allow us to fully exploit the potential of DES data for cosmological studies. These complementary avenues will be essential to correctly interpret the deep, high-cadence data delivered by the Large Synoptic Survey Telescope (LSST), where both the statistical power and the impact of the observing conditions will be increased by many orders of magnitude (e.g., LSST Science Collaboration et al. 2009; Jee & Tyson 2011; LSST Dark Energy Science Collaboration 2012; Carroll et al. 2014).

B.L., H.V.P., and F.E. are supported by STFC and the European Research Council under the European Community's Seventh Framework Programme (FP7/2007-2013)/ERC grant agreement No. 306478-CosmicDawn. This work was supported in part by National Science Foundation grant No. PHYS-1066293 and the hospitality of the Aspen Center for Physics. C.C., A.R., A.A., and C.B. are supported by in part the Swiss National Foundation grants 200021-149442 and 200021-143906. F.S. acknowledges financial support provided by CAPES under contract No. 3171-13-2. M.L. is partially supported by FAPESP and CNPq. We acknowledge use of the HEALPix software package (Górski et al. 2005).

We are grateful for the extraordinary contributions of our CTIO colleagues and the DES Camera, Commissioning, and Science Verification teams in achieving the excellent instrument and telescope conditions that have made this work possible. The success of this project also relies critically on the expertise and dedication of the DES Data Management organization.

This paper has gone through internal review by the DES collaboration. The DES and Fermilab publication numbers are DES-2015-0098 and FERMILAB-PUB-15-310-A-AE, respectively.

Funding for the DES Projects has been provided by the U.S. Department of Energy, the U.S. National Science Foundation, the Ministry of Science and Education of Spain, the Science and Technology Facilities Council of the United Kingdom, the Higher Education Funding Council for England, the National Center for Supercomputing Applications at the University of Illinois at Urbana-Champaign, the Kavli Institute of Cosmological Physics at the University of Chicago, the Center for Cosmology and Astro-Particle Physics at the Ohio State University, the Mitchell Institute for Fundamental Physics and Astronomy at Texas A&M University, Financiadora de Estudos e Projetos, Fundação Carlos Chagas Filho de Amparo à Pesquisa do Estado do Rio de Janeiro, Conselho Nacional de Desenvolvimento Científico e Tecnológico and the Ministério da Ciência, Tecnologia e Inovação, the Deutsche Forschungsgemeinschaft and the Collaborating Institutions in the DES. The DES data management system is supported by the National Science Foundation under Grant Number AST-1138766.

The Collaborating Institutions are Argonne National Laboratory, the University of California at Santa Cruz, the University of Cambridge, Centro de Investigaciones Enérgeticas, Medioambientales y Tecnológicas-Madrid, the University of Chicago, University College London, the DES-Brazil Consortium, the University of Edinburgh, the Eidgenössische Technische Hochschule (ETH) Zürich, Fermi National Accelerator Laboratory, the University of Illinois at Urbana-Champaign, the Institut de Ciències de l'Espai (IEEC/CSIC), the Institut de Física d'Altes Energies, Lawrence Berkeley National Laboratory, the Ludwig-Maximilians Universität München and the associated Excellence Cluster universe, the University of Michigan, the National Optical Astronomy Observatory, the University of Nottingham, The Ohio State University, the University of Pennsylvania, the University of Portsmouth, SLAC National Accelerator Laboratory, Stanford University, the University of Sussex, and Texas A&M University.

The DES participants from Spanish institutions are partially supported by MINECO under grants AYA2012-39559, ESP2013-48274, FPA2013-47986, and Centro de Excelencia Severo Ochoa SEV-2012-0234. Research leading to these results has received funding from the European Research Council under the European Union's Seventh Framework Programme (FP7/2007-2013) including ERC grant agreements 240672, 291329, and 306478.

Footnotes

  • 45 
  • 46 

    With the exception of the Stripe 82 region, the deeper multi-epoch program of SDSS, and the small zones of overlap between the single-epoch images.

  • 47 

    Including SCAMP (astrometry, Bertin 2006), SWARP (image coaddition, Bertin et al. 2002), PSFEx (modeling of the point-spread function, Bertin 2011), and SExtractor (object detection and measurement, Bertin & Arnouts 1996).

  • 48 

    WCS refers to the World Coordinate System of the FITS format (Calabretta & Greisen 2002).

  • 49 

    In particular, DES images make use of the WCS TPZ projection, built on the standard TAN projection and adding general polynomial corrections.

  • 50 
Please wait… references are loading.
10.3847/0067-0049/226/2/24