THE GALAXY ENVIRONMENT OF A QSO AT z ∼ 5.7

Observations of the highest redshift (z ≳ 6) quasars indicate that they contain supermassive black holes with masses >10⁹ M_☉ (e.g., Jiang et al. 2007; Kurk et al. 2007; De Rosa et al. 2011). Detecting such massive black holes less than a gigayear after the big bang is challenging for structure-formation models. Some models propose that supermassive black holes are hosted in high-density peak dark matter halos (e.g., Volonteri & Rees 2006). Numerical models predict that the most massive dark matter halos at z ∼ 6 will evolve into massive >10¹⁴–10¹⁵ M_☉ clusters in the local universe (e.g., Springel et al. 2005).

At redshifts of 2 < z < 5 significant galaxy overdensities or protoclusters have been found around more than a half dozen luminous radio galaxies, confirming the idea that luminous active galactic nuclei (AGNs) pinpoint dense regions in the early universe (see, e.g., Venemans et al. 2007b, and references therein). Nevertheless, the environments associated with other classes of AGNs, such as optically selected quasars, are currently less well constrained.

Recently, there have been a number of studies finding contradictory results regarding the galaxy environment of quasars at z ∼ 2–5. Cantalupo et al. (2012), in a narrow-band imaging survey around a quasar at z = 2.4, found a much larger number of Lyman alpha emitters (LAEs) than in blank-field Lyα surveys. They suggest that this overdensity may be fully explained by quasar fluorescence, which boosts gas-rich but intrinsically faint LAEs, increasing the number of detectable objects. Francis & Bland-Hawthorn (2004) carried out deep narrow-band observations centered on a z = 2.168 quasar. They were expecting to see 6–25 Lyα-fluorescent clouds and tens of normal LAEs based on similar surveys. However, they did not detect any Lyα emission. In light of this result, Bruns et al. (2012) developed a semi-analytical model to interpret the observations. They concluded that the intense ultraviolet emission of the quasar may be suppressing the star formation in galaxies that are situated close to the quasar. This is consistent with the findings of Kashikawa et al. (2007) that LAEs are clustered around a quasar at z = 4.87, but avoid it in its immediate vicinity (∼4.5 comoving Mpc). Swinbank et al. (2012), on the other hand, reported a galaxy overdensity an order of magnitude higher than that which might be expected in the field within 8.2 comoving Mpc of a quasar at z = 4.528. The same authors did not find clear evidence of overdensities in the fields of two quasars at z ∼ 2.2. Very recently, Husband et al. (2013) noted that the luminous quasars in their sample at z ∼ 5 were typically found in overdense regions. Nevertheless, they mentioned that even the richest quasar environment they studied was no richer than other structures identified in blank fields.

Since the discovery of z ∼ 6 quasars, several groups have tried to identify galaxy overdensities associated with them. The results were as puzzling as those at lower redshifts. No unambiguous relation has been found between galaxy overdensities and z ∼ 6 quasars, mainly due to the difficulty of finding galaxies with accurate redshifts at z ∼ 6. So far, these efforts were mainly based on studies of continuum i-dropout galaxies, characterized by a large magnitude difference between the i and z bands (i.e., using the same technique that was used to detect the quasars). This technique probes a redshift range of approximately Δz ≈ 1, which is large enough to identify galaxies that are not physically related to the quasar. Willott et al. (2005) carried out a survey with Gemini of three z > 6.2 quasars. They found no evidence for an overdensity of i-dropouts in the 27 arcmin² field surrounding the quasars. Stiavelli et al. (2005), however, came to a different conclusion for one of these fields. They observed a small field around SDSS J1030 + 0524 at z = 6.28 with the Advanced Camera for Surveys (ACS) of the Hubble Space Telescope (HST) and found more than twice the number of dropouts with i₇₇₅ − z₈₅₀ > 1.5 expected from statistics obtained by GOODS. The ACS field used in this study only surveys the region close to the quasar, but the observations are more sensitive to faint galaxies. Stiavelli et al. (2005) explain that the difference is that their data set is deeper and the majority of the excess sources are fainter than the limiting magnitude of Willott et al. (2005). Zheng et al. (2006), using ACS observations, found an overdensity in the field of a radio-loud quasar at z = 5.8: the surface density of 1.3 < i − z < 2.0 sources was about six times higher than the number expected from ACS deep fields. Kim et al. (2009) studied i-dropout galaxies in five fields centered on z ∼ 6 quasars using ACS and reported that regions near quasars are sometimes overdense and sometimes underdense. From a theoretical perspective, Overzier et al. (2009), using semi-analytic galaxy models in combination with the dark matter Millennium simulation (Springel et al. 2005; De Lucia & Blaizot 2007), showed that the lack of neighboring galaxies as found by Willott et al. (2005) and Kim et al. (2009) was not inconsistent with quasars occupying massive halos due to a combination of depth, field-of-view, and projection effects. However, the same simulations also predicted that there were regions much more overdense than seen around any z ∼ 6 quasar observed to date. Utsumi et al. (2010) found an overdensity of i-dropout galaxies around a quasar at z = 6.43, but at the same time these objects avoided the center near the quasar (∼15 comoving Mpc), similar to what Kashikawa et al. (2007) found at lower redshift. At high redshift, it is very hard to obtain spectra of faint galaxies, but there are a few cases at z ∼ 5, based on available spectroscopy, where quasars are located in regions with an overdensity of galaxies (Capak et al. 2011; Walter et al. 2012; Husband et al. 2013). It is worth noting that in some of these cases, the quasars do not reside in the center of the overdensities but at distances of ∼5–15 comoving Mpc. It is important to note that a few galaxy overdensities or protoclusters have been discovered serendipitously in random fields (e.g., Ouchi et al. 2005 at z ∼ 5.7; Toshikawa et al. 2012 at z ∼ 6; Trenti et al. 2012 at z ∼ 8), suggesting that not all overdensities host AGNs, although this could also be explained by the duty cycle of AGN activity.

In summary, current dropout studies at z ∼ 6 give partly contradictory results. An efficient alternative is to search for LAEs in a narrow redshift range near the quasar using narrow-band filters. However, mainly due to the extremely low density of z ∼ 6 quasars of ∼6 × 10⁻¹⁰ Mpc⁻³ (Fan et al. 2004), until recently no high-redshift quasars were known to have a redshift that shifts the Lyα line into a region of the optical spectrum that is devoid of bright sky emission lines. These atmospheric windows, for example, around 8160 Å and 9120 Å, have successfully been used by blank field Lyα surveys to search for galaxies at z = 5.7 and z = 6.6 (e.g., Hu et al. 1999, 2002; Ouchi et al. 2005, 2008). So far, the only study using narrow-band imaging to detect the Lyα-emitting halo and possible companions around quasars at z > 6 was carried out using the HST by Decarli et al. (2012). Even though the goal of these observations was to detect the Lyα halo around the quasars, they did not find any companions in the immediate vicinity of two z > 6 quasars. This latter result was not unexpected because of the small field-of-view covered by their study (∼1 arcmin²).

In the present work, we use deep narrow- and broad-band imaging to study the environment of the broad-absorption line quasar ULAS J0203+0012 (hereafter J0203; Venemans et al. 2007a; Jiang et al. 2008; Mortlock et al. 2009), one of the first z ∼ 6 quasars known with a redshift that shifts the Lyα line into an atmospheric window that allows for deep narrow-band imaging. We assume the quasar redshift of z = 5.72 determined by Mortlock et al. (2009), which is based on the N v, Si iv, C iv, and C iii lines from a combined optical and near-infrared spectrum. However, we have to keep in mind that any redshift in the range of 5.70 < z < 5.74 is consistent with their data. Ryan-Weber et al. (2009), using a Keck/NIRSPEC spectrum, independently confirmed this quasar, reporting a redshift of z = 5.706 based on a broad emission feature that they presume to be C iv λ 1549.062. In any case, the uncertainty in redshift does not shift the Lyα line outside the narrow-band filter used in our work.

In Section 2 we describe the data used in this study, photometry, and object detection. Section 3 describes how LAE and dropout or Lyman break galaxy (LBG) candidates were selected. In Section 4 we present our results, including a number count comparison with blank fields, photometric properties of the LAE sample, and our estimation of the black hole mass of the quasar. Finally, in Section 5 we discuss the results and present our conclusions.

In this paper, all magnitudes are given in the AB system and are corrected for Galactic extinction (Schlafly & Finkbeiner 2011). We employ a ΛCDM cosmology with H₀ = 70 km s⁻¹ Mpc⁻¹, Ω_M = 0.3, and Ω_Λ = 0.7, which yields an age of the universe of 0.976 Gyr and a spatial scale of 39.9 kpc arcsec⁻¹ in comoving units at z = 5.7.

The field centered on the quasar J0203 at z = 5.72 was observed during 2010 November–December. Narrow- and broad-band imaging was carried out with the FOcal Reducer/low dispersion Spectrograph 2 (FORS2; Appenzeller & Rupprecht 1992) using the red sensitive detector consisting of two 2k × 4k MIT CCDs at the Very Large Telescope (VLT). The pixels were 2 × 2 binned to decrease the readout time and noise, giving a spatial scale of 0.25 arcsec pixel⁻¹. The field-of-view was 68 × 68. The field was observed in the narrow-band filter (FILT_815_13, λ_c = 8150 Å, Δλ = 130 Å, hereafter NB) and the broad-band filters Z (Z_GUNN, λ_c = 9100 Å, Δλ = 1305 Å) and R (R_SPECIAL, λ_c = 6550 Å, Δλ = 1650 Å). The filter transmission curves are shown in Figure 1. This filter set allowed us to sample both LAEs in a redshift range of 5.66 ≲ z ≲ 5.75 with the narrow-band filter and LBGs in a broader redshift range 5.2 ≲ z ≲ 6.8 with the broad bands (5.2 ≲ z ≲ 5.8, if we add the narrow-band filter to the selection).

Zoom In Zoom Out Reset image size

The individual exposure times in NB, Z, and R were 800 s, 110 s, and 240 s per pointing, respectively. The individual exposures were shifted by ∼10'' with respect to each other to facilitate the removal of bad pixels and flat field errors associated with a fixed position of the CCD pixel. The total integration times were 6 hr in NB, 1.5 hr in Z, and 48 minutes in R.

Standard data reduction was performed, which consisted of bias subtraction, flat fielding, sky subtraction, image alignments, and stacking. We calculated the photometric zero points by using the magnitudes and colors of stellar objects in the Sloan Digital Sky Survey (SDSS) catalog (Abazajian et al. 2009). With this procedure, the flux losses for point sources are corrected, which is appropriate for this study since we are interested in high-redshift LAEs that are expected to be unresolved. The accuracy of the photometric zero points was 0.04, 0.04, and 0.05 for the NB, Z, and R frames, respectively.

The area covered by the reduced images was ∼44.4 arcmin². The seeing for the final NB, Z, and R images was 084, 086, and 076, respectively. The 5σ limiting magnitudes of the reduced images with a 15 diameter aperture were NB = 25.34, Z = 25.14, and R = 26.29.

In order to determine appropriate colors, the NB and R images were convolved to match the PSF of the Z image, the image with the worst seeing, using the IRAF task psfmatch. The NB total magnitudes were calculated from the unconvolved image.

The source catalog was created running SExtractor (Bertin & Arnouts 1996) in dual image mode with the narrow-band frame as the detection image. Aperture magnitudes were calculated using a 15 diameter aperture (∼1.75 ×FWHM of the seeing). The chosen aperture was a good balance between optimizing the signal-to-noise (S/N) photometry of point sources and minimizing the amount of flux outside the aperture (Labbé et al. 2003). This size assured that at least 70% of the flux of a point source was inside the aperture. We also tested larger apertures (e.g., 18 diameter) and the results did not change, but the S/N of the objects slightly decreased. Magnitudes of objects not detected or fainter than 2σ limiting magnitudes either in Z or R were replaced by the corresponding 2σ limiting magnitude. Finally, we only considered objects with S/N greater than 4, objects with NB > 18, and with SExtractor flag ⩽4 in order to eliminate objects flagged as truncated (too close to an image boundary), incomplete, or corrupted. The final catalog contained 2424 objects.

In order to conclude whether or not an overdensity around the quasar was present, we needed a comparison field obtained at similar depth in a region that did not target a z ≳ 5.7 quasar. For LAEs we were able to use results from the literature (see Section 4.1.1), whereas for LBGs there were no R−Z dropout searches with large spectroscopic follow-up in the literature. We performed our own selection of R−Z dropouts using the public catalogs of the Subaru Deep Field (SDF) imaging survey (Kashikawa et al. 2004), which (after removing low-quality regions) had an effective area of ∼876 arcmin². The SDF had several characteristics that made it a suitable comparison field for our study. (1) It is a large field, which helps to reduce cosmic variance. (2) It has R and Z bands covering similar wavelength ranges compared to the ones in FORS2. (3) There are 42 spectroscopically confirmed i-dropout galaxies discovered in different studies (Nagao et al. 2004, 2005, 2007; Ota et al. 2008; Jiang et al. 2011, 2013; Toshikawa et al. 2012), which can be directly used to quantify the accuracy of our LBG selection.

The confirmed i-dropout galaxies in the SDF were originally selected in the Z-band and all of them have a redshift z > 5.9 (with one exception at z = 5.762). For that reason, these galaxies would either not be detected or have negligible flux in our NB filter. Thus, in order to perform a consistent comparison, we created a second catalog for the J0203 field this time using the Z-band frame as the detection image (see Section 4.1.2).

Figure 2 shows the two-color diagram of Z − NB and R − Z for the objects detected in the narrow-band filter and summarizes the selection criteria for LAE and dropout candidates.

Zoom In Zoom Out Reset image size

3.1. LAEs

3.2. Lyman Break Galaxies

Dropout galaxies are also known as LBGs, given that they are selected using the Lyman break technique (Steidel et al. 1996). The basis of this method is the fact that hydrogen is very effective at absorbing radiation at wavelengths shorter than 912 Å (the Lyman limit). Therefore, radiation with λ ≲ 912 Å is strongly suppressed by intergalactic and interstellar absorption, so a very small fraction of these photons will reach us, forming the so-called Lyman Break. At high-redshifts, the Lyα forest becomes so optically thick that a large fraction of the light coming from λ_rest = 912 Å to λ_rest = 1216 Å is also absorbed. At this point the Lyman break selection becomes, in effect, a selection of objects with a sharp break at λ_rest = 1216 Å.

Dropout candidates were selected with the following criteria.

• 1.  
  Continuum break. A prominent break in the continuum is expected for high-redshift galaxies due to the hydrogen absorption. We require a continuum break of R − Z > 2.0. Additionally, in an attempt to bias our high-redshift candidates toward a redshift closer to the redshift of the quasar, we require a break between the flux in the NB and R filters: R − NB > 1.5. However, this last criterion has a small effect and excludes only one candidate.
• 2.  
  Significance of the break. To avoid contamination by objects that only satisfy the color criteria due to photometric errors, we require: $\left| (R - Z)\right| > 2.5 \sqrt{\vphantom{A^A}\smash{\hbox{${\sigma ^{2}_{R} + \sigma ^2_{Z}}$}}}$.
• 3.  
  Faintness. Since high-redshift galaxies are expected to be faint, we impose a minimum magnitude of Z > 21.

As shown in Figure 2, there are eight LBG candidates satisfying the criteria.

The coordinates of the LAE and LBG candidates and their projected distances to the quasar are presented in Table 1. Figure 3 shows the color image of the field overlaid with the positions of the LAE and LBG candidates, and the quasar. Postage-stamp images of these objects are shown in Figure 4.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Table 1. Coordinates of the Candidates of Figure 3 and their Projected Distances with Respect to the Quasar

Object	R.A.	Decl.	Distance	Distance
	(J2000.0)	(J2000.0)	(arcmin)	(cMpca)
LAE 1	02:03:38.23	+00:11:09.9	1.97	4.65
LAE 2	02:03:21.02	+00:10:40.2	3.37	7.96
Drop 1	02:03:27.15	+00:10:32.7	2.34	5.52
Drop 2	02:03:19.76	+00:11:25.8	3.33	7.85
Drop 3	02:03:37.12	+00:15:29.9	3.24	7.64
Drop 4	02:03:34.04	+00:14:34.2	2.13	5.02
Drop 5	02:03:39.69	+00:15:35.0	3.60	8.49
Drop 6	02:03:39.92	+00:13:38.5	2.21	5.22
Drop 7	02:03:42.04	+00:12:56.3	2.46	5.78
Drop 8	02:03:32.71	+00:09:58.6	2.51	5.92

Note. ^aComoving Mpc at z = 5.7.

Download table as:  ASCIITypeset image

4.1. Blank Field Comparison

4.1.1. LAEs

The selection criteria used in the present work (see Section 3.1) is close to that used by Ouchi et al. (2005, 2008). Thus, it is natural to use their LAE sample for comparison. Their observations cover a larger area and reach fainter luminosities than this work: the area they imaged is 1.04 deg² on the sky and a 5σ limiting magnitude of NB = 26.0 (λ_c = 8150 Å, Δλ = 120 Å). Unlike our field, the Ouchi et al. (2008) sample is not centered on a quasar, which is why we consider it a blank field. However, even in blank fields, protoclusters can exist. In fact, Ouchi et al. (2005) detected two overdensities that could be clusters in a formation phase.

In Figure 5, we show our cumulative number of LAEs and also the numbers from Ouchi et al. (2008) scaled to our area. Our results are in good agreement with the expected number from Ouchi et al. (2008). Figure 6 shows the distribution on the sky of the 401 LAEs at z = 5.7 ± 0.05 detected by Ouchi et al. (2008). Masked regions due to bright stars or image artifacts are shown in red. The dashed square in the bottom left corner represents the effective size of the FORS2 field-of-view used in this work. With the goal of estimating the probability of detecting, in a blank field, the number of LAEs that we find in this work, we placed 100,000 FORS2 fields-of-view at random positions in the Ouchi et al. (2008) field. Only fields where less than 10% of the region is masked out were considered. We counted how many LAEs fell in each FORS2 field and the result is summarized in the histogram of Figure 7. Even considering that Ouchi et al. (2008) would have detected more galaxies since they were sensitive to fainter magnitudes, we find from Figure 7 that the number of LAEs in our field-of-view is consistent with the most typical number expected in their blank field, i.e., one or two galaxies.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Our results also compare with the study by Hu et al. (2010), which is based on a sample of 88 spectroscopically confirmed LAEs at z ∼ 5.7 in an area of 1.16 deg². Figure 5 shows their cumulative number of LAEs scaled to our area. The fact that their numbers are lower than the numbers of the Ouchi et al. (2008) sample could be explained by the fact that Hu et al. (2010) could not spectroscopically confirm approximately half of their photometric candidates. Nevertheless, our results are still consistent, within the errors, with the number expected from Hu et al. (2010). We conclude that there is no overdensity of LAEs in the quasar field.

4.1.2. LBGs

As stated in Section 3.2 and shown in Figures 3 and 4, we found eight objects satisfying our LBG criteria. These objects were detected in the NB filter and our criteria place them in the redshift range: 5.2 ≲ z ≲ 5.8. However, we cannot follow the same approach if we want to exploit the information of the 42 confirmed i-dropout galaxies in our comparison field (SDF), due to the fact that essentially all of these confirmed galaxies have a redshift z > 5.9 (except one that has a redshift of z = 5.762, ID = 1 in Toshikawa et al. 2012) and would not be detected in our NB filter.

We consider the SDF to be a blank field because it does not contain known z ≳ 5.7 quasars. The only caveat is that it has been claimed that the SDF contains two overdensities. One is at z ∼ 4.9 (Shimasaku et al. 2003), from which we expect minimal contamination due to our selection criteria. The other one is a protocluster at z ∼ 6 (Ota et al. 2008; Toshikawa et al. 2012), which in principle could complicate our analysis.

The creation of the source catalogs for both the quasar and comparison fields was carried out in the same manner as for the J0203 field in Section 2, but this time using the Z-frame as the detection image. We applied the same selection criteria as in Section 3.2, except for the narrow-band constraints. Additionally, since the depth of the fields are different, in order to make a consistent comparison, we have constrained our LBG candidates to have a magnitude brighter than the 5σ limiting magnitude in the quasar field, i.e., Z < 25.14. Since all the confirmed members of the protocluster at z ∼ 6 have magnitudes Z > 25.5 (Toshikawa et al. 2012), our Z < 25.14 selection cut should atleast prevent us from dealing with a large fraction of the protocluster galaxies.

Figures 8 and 9 present the color–magnitude diagram and the selection criteria of the Z-band selected objects in the J0203 and SDF fields respectively. In red circles we show the LBG candidates detected in these fields. There were 20 LBG candidates in the quasar field, and this sample contained the 8 NB-selected LBGs from Section 3.2. We found 370 LBG candidates in the SDF. Scaling the LBG candidates in the SDF to our effective area, the expected number of candidates is ∼19. Additionally, we placed 100,000 FORS2 fields-of-view at random positions in the SDF. Counting how many LBG candidates fell in each FORS2 field resulted in a Gaussian distribution with mean μ = 16 and standard deviation σ = 5. We concluded that the density of LBG candidates in the quasar field is consistent with that of a blank field.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Figure 9 also presents the compilation of known i-dropout galaxies in the SDF. With our depth we could not detect any of the 42 spectroscopically confirmed galaxies. We were only able to recover the brightest candidate of Nagao et al. (2007), a source that could not be confirmed or ruled out as a galaxy at 6.0 < z < 6.5 by its spectrum. Additionally, if we used the extra information of the SDF provided by the i band, we noticed that the majority of the objects that we selected as possible LBGs seemed to be contaminants with a rather shallow slope in the R−i, i−Z colors instead of having a sharp break.

From this analysis, we concluded that our LBG sample is very likely to be highly contaminated. We therefore cannot reliably estimate the (over)density of LBGs around our quasar. In the following sections we thus focus primarily on the LAE results.

4.2. Photometric Properties of the LAEs

We assume a simple model where the LAE spectrum consists of the Lyα line with flux F_Lyα, a continuum flux density redward of Lyα with strength C and a power law slope β (f_λ∝λ^β). Then, the flux density in the NB (f_{λ,  NB}) and Z (f_{λ, Z}) can be written as:

$$\begin{equation} f_{\lambda ,{\scriptsize NB}} = C \lambda _{{\rm eff}, {\scriptsize NB}}^\beta + F_{{\rm Ly}\alpha } / \Delta \lambda _{\scriptsize NB}} \end{equation} \tag{ 1 }$$

$$\begin{equation} f_{\lambda ,Z} = C \lambda _{{\rm eff}, Z}^\beta , \end{equation} \tag{ 2 }$$

with λ_{eff, i} the effective wavelength of the i filter and Δλ the width of the filter. Dunlop et al. (2012) found that high redshift galaxies (5 < z < 7) have an average slope of β ≃ 2 and this does not show a significant trend with either redshift or M_UV. Assuming a flat continuum, i.e., β = −2, we solve the previous 2 × 2 linear system and estimate the line flux and continuum flux density.

To calculate L_Lyα and equivalent width, we use the relations:

$$\begin{equation} L_{{\rm Ly}\alpha } = 4\pi d_L^2(z=5.7) F_{{\rm Ly}\alpha } \end{equation} \tag{ 3 }$$

$$\begin{equation} {\rm EW}_{{\rm obs}} = \frac{F_{{\rm Ly}\alpha }}{C(\lambda _{{\rm Ly}\alpha }(1 + z)^\beta)} \end{equation} \tag{ 4 }$$

with d_L the luminosity distance and λ_Lyα the wavelength of the Lyα line. The rest-frame equivalent width is given by EW₀ = EW_obs/(1 + z).

To calculate star formation rates (SFRs), we follow the approach of Ouchi et al. (2008):

$$\begin{equation} \mbox{SFR}(M_\odot \mbox{ yr}^{-1}) = \frac{L_{{\rm Ly}\alpha }}{1.1 \times 10^{42} \,\mathrm{erg}\, \mathrm{s}^{-1}} \end{equation} \tag{ 5 }$$

where they used the relation of Hα luminosity and star formation rate (Kennicutt 1998) and assumed the standard case B recombination factor of 8.7 for the Lyα/Hα luminosity ratio (Brocklehurst 1971).

We calculate the input parameters for the filters used in this project, taking into account both the filter curves and the quantum efficiency of the CCD: λ_{eff,  NB} = 8150.2 Å, λ_{eff, Z} = 9180.2 Å, and Δλ_NB = 119 Å. The star formation rates and rest-frame equivalent widths for our two LAE candidates from Section 3.1 are $(\mbox{SFR},\, {\rm EW}_0) = (6.0 \pm 1.3 \; M_\odot \, \mathrm{yr}^{-1},\, 47^{+222}_{-16} \; \mathring{\rm{A}})$ and (SFR, EW₀) = (1.7 ± 0.9 M_☉ yr⁻¹, ⩾ 14 Å) for LAE 1 and LAE 2, respectively (see Figures 3 and 4).

The properties derived for LAE 1 are within the expected range for typical LAEs (see Ouchi et al. 2008, SFR = $6.2^{+2.7}_{-1.9} \; M_\odot \, \mathrm{yr}^{-1}$ for L* LAEs at z = 5.7), whereas for LAE 2, the SFR is lower than for typical LAEs and the equivalent width is only a lower limit.

4.3. Black Hole Mass of the Quasar

We present a deep study of the LAE and R − Z dropout population centered on the z = 5.72 quasar J0203 with an estimated black hole mass of ∼2.5 × 10⁹ M_☉. The redshift of the quasar enables a LAE narrow-band search in its vicinity. We detect 2 LAE and 20 R − Z dropout candidates (the number of dropout candidates decreases to 8 if we use the narrow band for detection and selection). The LAE sample spans a very narrow redshift range around the quasar: 5.66 < z < 5.75. On the other hand, the R − Z dropout sample spans a larger redshift range as is typical for dropout selections: 5.2 ≲ z ≲ 6.8 (5.2 ≲ z ≲ 5.8, if the narrow-band filter is added to the selection). Comparing our LAE counts with the luminosity functions of two LAE surveys that do not target a quasar (Ouchi et al. 2008; Hu et al. 2010), we find that our number counts are consistent with what is observed in blank fields.

The number of R − Z dropout candidates in the field of J0203 is consistent with the expected number from the SDF blank field. However, we find that with our depth and using only two broad bands for the selection, it is very likely that our candidates are highly contaminated. This implies that we cannot establish a reliable estimate of the (over)density of LBGs around J0203.

There are various ways to explain the non-detection of an overdensity of LAEs around the quasar.

• 1.  
  Low number statistics may prevent us from detecting a possible overdensity of emission line galaxies. If there were an overdensity it would be at least a factor of ∼3–5, based on environmental studies of radio galaxies. Since we are studying only one quasar, it could be that we are targeting a rare source and that other quasars still harbor overdensities in their surroundings. This is certainly a possibility, but we believe that it is unlikely given the relatively low counts of LAEs and LBGs toward other z ∼ 6 quasars studied to date.
• 2.  
  The lack of neighbors in the immediate vicinity could be due to mergers of galaxies in the halo that hosts the quasar. However, we believe that such mergers should not prevent us from detecting an overdensity based on the protoclusters found around radio galaxies (e.g., Venemans et al. 2007b) and the recently spectroscopically confirmed LBGs in the vicinities of quasars at z ∼ 5 found by Husband et al. (2013), which in principle should have been affected in the same manner.
• 3.  
  The strong ionizing radiation from the quasar may prevent galaxy formation in its surroundings. There are several studies that support this claim (e.g., Bruns et al. 2012; Francis & Bland-Hawthorn 2004) and therefore it would be important to understand and quantify this effect in order to consider it in simulations. Indeed, our results present evidence to support this idea because none of our candidates are at a distance closer than ∼4.5 comoving Mpc (∼0.7 physical Mpc), as can be seen in Figure 3 and Table 1. This is similar to the results presented by Kashikawa et al. (2007) and Utsumi et al. (2010), i.e., that galaxies avoid the vicinity of a quasar within distances of ∼4.5 comoving Mpc and ∼15 comoving Mpc, respectively.Yet, the quasar ionization hypothesis fails to explain why apparent overdensities are found in the vicinities of other high-redshift quasars (e.g., Stiavelli et al. 2005). Further, we believe that is hard to reconcile a hypothesis in which the radiation affects the environment isotropically with the unified AGN model where quasars have two collimated beams. However, contrary to the ionization hypothesis, Cantalupo et al. (2012) suggest that the radiation of the quasar may enhance the number of LAEs in its surroundings. They show that gas-rich objects with little or no associated star formation, known as proto-galactic clouds or dark galaxies, can be detected thanks to the Lyα fluorescence induced by a nearby quasar. One of the main characteristics of these sources is a high Lyα equivalent width (EW₀ > 240 Å). A detailed comparison with Cantalupo et al. (2012) is challenging because of the different redshifts and because the Lyman limit luminosity of our quasar, calculated following Hennawi et al. (2006), is about one order of magnitude smaller than the luminosity of the hyperluminous quasar (HLQSO) observed by Cantalupo et al. (2012). Additionally, since J0203 is a broad absorption line quasar (Mortlock et al. 2009), it is possible that the amount of escaping ionizing radiation is lower than expected in typical quasars.Nevertheless, in our study, we do not find such fluorescent clouds. It should be noted that in our work, LAE 1 has an ${\rm EW}_0 = 47^{+222}_{-16}$ Å which has a 4% probability of having EW₀ > 240 Å based on the uncertainties of the measured fluxes and LAE 2 only has a lower limit of EW₀ ⩾ 14 Å.
• 4.  
  High-redshift quasars may not reside in the most massive halos. It is expected that the relation between black hole mass and halo mass that arises from combining the local relations between black hole and bulge mass on the one hand, and the relation between galaxy and halo mass on the other, is significantly different (both in slope and in scatter) at high redshift. Evidence for this has been found recently at z ∼ 2.7 by Trainor & Steidel (2012) who studied fifteen HLQSOs (≳ 10¹⁴ L_☉; M₁₄₅₀ ≃ −30) that are associated with galaxy overdensities. The authors conclude that HLQSOs do not require environments very different from their much less luminous quasar counterparts and are not being hosted by rarer dark matter halos. However, if this applies to the z ∼ 6 quasars as well, one would have to explain their extremely low space density. One possibility is that, due to selection effects, the bright quasars thus far discovered are biased toward more average-sized halos that are inhabited by very massive black holes.
• 5.  
  High-redshift quasars may not always reside in highly overdense, large-scale environments. All observational evidence considered, there appears to be a trend for z ∼ 6 quasars to sit in low-to-average galaxy environments on megaparsec scales. ΛCDM predictions have shown that the most massive halos at high redshift do not invariably grow into the most massive ones at low redshift. Moreover, the most massive present-day structures (clusters) originated primarily from regions that were overdense on very large scales (tens of megaparsec) at z ∼ 6 (De Lucia & Blaizot 2007; Overzier et al. 2009; Angulo et al. 2012). Although few studies have probed out to such large radii and with significant sensitivity, the lack of companion galaxies on megaparsec scales may indicate that the quasars do not pinpoint progenitors of the most massive clusters even if the quasars themselves are hosted by very massive halos. It is also possible that quasars reside in large-scale overdense environments, but just not in the center of them. Studies with larger fields-of-view are required to test this hypothesis.

One question that emerges naturally is why radio-galaxy-based protocluster surveys have been much more successful than any quasar-based protocluster survey. From a theoretical perspective, N. Fanidakis & A. Orsi (2013, in preparation) suggest that radio galaxies and quasars are observational manifestations of different accretion states, which are determined by the accretion and black hole properties and the large scale environment. Quasars accrete vast amounts of gas and form in the gas-rich environments of intermediate mass (∼10¹² M_☉) dark-matter halos (Fanidakis et al. 2013). Radio galaxies, in comparison, are powered by rapidly spinning black holes that typically inhabit the centers of the most massive (≳ 10¹³ M_☉) dark matter halos in the universe and accrete low-density gas (for the properties of their AGN model; see Fanidakis et al. 2011, 2012). The largest concentrations of baryons are expected to be found within these halos and thus, radio galaxies can be used to pinpoint the location of large galaxy overdensities. The suggestion of a discrete halo environment of quasars and radio galaxies is also supported by clustering analysis of AGNs in the low-redshift universe (0 < z < 2; e.g., Ross et al. 2009; Donoso et al. 2010; Wake et al. 2011).

More detailed studies of the environment and of the host galaxy of high-redshift quasars are needed to clarify the picture of overdensities around quasars. Future observational campaigns targeting high-redshift quasars either at z = 5.7, z = 6.6, or z = 7.0 (corresponding to the few gaps of OH night-sky emission bands where narrow-band filters can be most effectively targeted to detect Lyα emission) will be crucial to increase the sample and to quantify possible object-to-object variations.

Based on observations made with ESO Telescopes at the La Silla Paranal Observatory under program ID 385.A-0030(A). E.B. thanks N. Fanidakis, J. Hennawi, and M. Tanaka for useful discussions about this work, M. Maseda for proof reading a preliminary version of this manuscript, and the IMPRS for Astronomy & Cosmic Physics at the University of Heidelberg. The plots in this publication were produced using Matplotlib (Hunter et al. 2007, http://www.matplotlib.org).

Facility: VLT:Antu (FORS2) - Very Large Telescope (Antu)