ALMA IMAGING OF GAS AND DUST IN A GALAXY PROTOCLUSTER AT REDSHIFT 5.3: [C ii] EMISSION IN "TYPICAL" GALAXIES AND DUSTY STARBURSTS ≈1 BILLION YEARS AFTER THE BIG BANG

We have detected 1.0 mm continuum emission toward the z = 5.3 SMG AzTEC-3 (Figure 1). From two-dimensional Gaussian fitting to a data set averaged over the line-free spectral windows, we find a continuum flux density of 6.20 ± 0.25 mJy (Table 1) and a deconvolved continuum source size of 040$^{+0{\buildrel{\prime\prime}\over{.}} 04}_{-0{\buildrel{\prime\prime}\over{.}} 04}$ × 017$^{+0{\buildrel{\prime\prime}\over{.}} 08}_{-0{\buildrel{\prime\prime}\over{.}} 17}$ (2.5 × 1.1 kpc² at z ≃ 5.3; Table 2).¹⁶

Zoom In Zoom Out Reset image size

Table 1. [C ii] and Continuum Properties of Sources in the z = 5.3 Protocluster

Target	Redshift	SC ii	dvC ii	IC ii	$L^{\prime }_{\rm C\,\scriptsize{II}}$	LC ii	S1.0 mm	log10(LC ii/LFIR)
		(mJy)	( km s−1)	(Jy km s−1)	(1010 K km s−1 pc2)	(109 L☉)	(mJy)
AzTEC-3	5.2988 ± 0.0001	18.4 ± 0.5	421 ± 19	8.21 ± 0.29	3.05 ± 0.11	6.69 ± 0.23	6.20 ± 0.25	–3.40
LBG-1	5.2950 ± 0.0002	8.99 ± 0.73	218 ± 24	2.08 ± 0.18	0.77 ± 0.07	1.69 ± 0.15	<0.45	>(–2.50)
							<0.15–0.24a,b	>(–2.03)–(–2.22)c
LBG-2				<0.21b,d	<0.08b,d	<0.17b,d	<0.24b
LBG-3				<0.21b,d	<0.08b,d	<0.17b,d	<0.24b

Notes. All quoted upper limits are 3σ. ^aIndividual limits for subcomponents LBG-1a, LBG-1b, and LBG-1c. ^bPoint source limits in mJy beam⁻¹, Jy beam⁻¹ km s⁻¹, or equivalent. ^cAssuming that the far-infrared continuum emission is not resolved by our observations. ^dAssuming the same line width and redshift as measured for LBG-1. Assuming an average or median of the line width of the subcomponents of LBG-1 would result in ∼10%–15% lower limits. We consider this difference in limits negligible compared to other sources of uncertainty.

Download table as:  ASCIITypeset image

Table 2. Derived [C ii] and Continuum Properties

Target	d([C ii])	dphys([C ii])	d(FIR)	dphys(FIR)	SFRFIRa	ΣSFR	$M_{\rm dyn}^{\rm [C{\scriptsize{II}}]}$b
		(kpc × kpc)		(kpc × kpc)	(M☉ yr−1)	(M☉ yr−1 kpc−2)	(1010 M☉)
AzTEC-3	063$^{+0{\buildrel{\prime\prime}\over{.}} 09}_{-0{\buildrel{\prime\prime}\over{.}} 09}$ × 034$^{+0{\buildrel{\prime\prime}\over{.}} 10}_{-0{\buildrel{\prime\prime}\over{.}} 15}$	3.9 × 2.1	040$^{+0{\buildrel{\prime\prime}\over{.}} 04}_{-0{\buildrel{\prime\prime}\over{.}} 04}$ × 017$^{+0{\buildrel{\prime\prime}\over{.}} 08}_{-0{\buildrel{\prime\prime}\over{.}} 17}$	2.5 × 1.1	1100	530	9.7
LBG-1	121$^{+0{\buildrel{\prime\prime}\over{.}} 41}_{-0{\buildrel{\prime\prime}\over{.}} 69}$ × 095$^{+0{\buildrel{\prime\prime}\over{.}} 68}_{-0{\buildrel{\prime\prime}\over{.}} 37}$	7.5 × 5.9	undetected	⋅⋅⋅	<18–54c	⋅⋅⋅	5.0
LBG-2	undetected	⋅⋅⋅	undetected	⋅⋅⋅	<28d	⋅⋅⋅	⋅⋅⋅
LBG-3	undetected	⋅⋅⋅	undetected	⋅⋅⋅	<28d	⋅⋅⋅	⋅⋅⋅

Notes. ^aDetermined from L_FIR, assuming a Chabrier (2003) initial mass function. ^bDerived assuming an isotropic virial estimator (e.g., Engel et al. 2010), and the galaxy size along the major axis. ^cThe lower of the limits assumes that all SFR_FIR is concentrated toward one component, the higher limit assumes that it is spatially resolved and spread over all components (LBG-1a, LBG-1b, and LBG-1c). ^dPoint source limits, assuming the same redshift and SED shape as for LBG-1.

Download table as:  ASCIITypeset image

The two pointings also covered five Lyman-break galaxies with photometric redshifts close to (or colors consistent with) z = 5.3, with COSMOS optical IDs 1447523, 1447526 (Ilbert et al. 2009; "LBG-2" and "LBG-3" in the following; ∼17 north-east and ∼22 northwest of AzTEC-3), and 1447524 (a triple of sources ∼15'' southeast of AzTEC-3, targeted by the second pointing; "LBG-1" below and in Figure 1; see C11 for more details on the source identifications). Besides color information, LBG-1 has an optical spectroscopic redshift of z_spec = 5.300 that is likely valid for all of its three components (LBG-1a, LBG-1b, and LBG-1c below), and LBG-3 has a photometric redshift of z_phot = 5.269 (C11). No 1.0 mm continuum emission is detected toward any of these galaxies, with 3σ upper limits of 0.24 mJy beam⁻¹ for LBG-2 and LBG-3 (conservative limits are quoted due to potential sidelobe residuals from AzTEC-3), and 0.45 mJy for LBG-1 (or 0.15–0.24 mJy beam⁻¹ for the three LBG subcomponents when accounting for potential source overlap at the current spatial resolution). Based on their small sizes in HST/ACS F814W imaging data (Figure 1), we assume in the following that LBG-2 and LBG-3 are spatially unresolved by our ALMA observations. Given the spatial separation of the three components of LBG-1 (Figure 1) and the spatial extent of the [C ii](²P_3/2→²P_1/2) emission (see below), we assume that the continuum emission in this source is resolved over ∼3 beams for the above estimate. We adopt this spatially integrated limit (rather than more sensitive limits for individual or stacked LBG components) in the following, to enable comparison with existing resolution-limited photometry at other wavelengths (see, e.g., Figure 2).

Zoom In Zoom Out Reset image size

We have detected strong [C ii](²P_3/2→²P_1/2) line emission toward AzTEC-3 (Figure 3). From two-dimensional Gaussian fitting to the line emission, we find a deconvolved [C ii] source size of 063$^{+0{\buildrel{\prime\prime}\over{.}} 09}_{-0{\buildrel{\prime\prime}\over{.}} 09}$ × 034$^{+0{\buildrel{\prime\prime}\over{.}} 10}_{-0{\buildrel{\prime\prime}\over{.}} 15}$ (3.9 × 2.1 kpc²; Table 2). The [C ii] emission is compact over the entire velocity range, without evidence for a significant shift in the centroid position between velocity channels (Figures 3–5). The [C ii] peak position is consistent with those of the CO and FIR continuum emission (R10), as well as with the peak of the optical emission longward of observed-frame ∼2 μm (Figure 2). There is tentative evidence in different velocity intervals for emission that may extend beyond the central, compact component that dominates the [C ii] emission, but only at low flux density levels. The most prominent of these features is a faint [C ii] emission component in the blue line wing that extends northeast from the center of the galaxy (Figure 6, left). This component could correspond to a tidal feature, a close galaxy companion in a minor merger event, or a [C ii] outflow (or inflow). A similar, but less significant feature is seen in the red [C ii] line wing (Figure 6, right). Observations at higher spatial resolution are required to further resolve the detailed structure of the interstellar medium in this massive starburst galaxy.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

We have also detected [C ii](²P_3/2→²P_1/2) emission toward the triple Lyman-break galaxy system LBG-1 (Figure 7). The emission is spatially resolved on a scale of 121$^{+0{\buildrel{\prime\prime}\over{.}} 41}_{-0{\buildrel{\prime\prime}\over{.}} 69}$ × 095$^{+0{\buildrel{\prime\prime}\over{.}} 68}_{-0{\buildrel{\prime\prime}\over{.}} 37}$ (7.5 × 5.9 kpc²; Table 2), covering all three optical emission regions (LBG-1a, LBG-1b, and LBG-1c; Figure 7). The [C ii] emission thus is clearly resolved toward the three components, which is seen more prominently in different velocity bins along the line emission (Figures 7 and 8). No [C ii](²P_3/2→²P_1/2) emission is detected toward LBG-2 or LBG-3.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

The [C ii](²P_3/2→²P_1/2) line parameters and redshifts for all detected galaxies were determined with four parameter Gaussian fits to the line and continuum emission spectra (Table 1). The [C ii] emission in AzTEC-3 has a peak flux density of 18.4 ± 0.5 mJy at a line FWHM of 421 ± 19 km s⁻¹, corresponding to an integrated line flux of 8.21 ± 0.29 Jy km s⁻¹, and peaks at a redshift of z = 5.2988 ± 0.0001 (Figure 9, left). The [C ii] emission in LBG-1 has a peak flux density of 8.99 ± 0.73 mJy at a line FWHM of 218 ± 24 km s⁻¹, corresponding to an integrated line flux of 2.08 ± 0.18 Jy km s⁻¹, and peaks at a central redshift of z = 5.2950 ± 0.0002 (Figure 10, left). The central velocities and line shapes of the [C ii] emission are different at the optical positions of the three regions (Figure 10, middle). The three components LBG-1a, b, and c peak at velocities of −49 ± 18, −3 ± 12, and +25 ± 16 km s⁻¹ relative to the central redshift, with line FWHM of 93 ± 32, 152 ± 29, and 250 ± 41 km s⁻¹, respectively (Table 3; Figure 10, middle). LBG-1a is the faintest and narrowest component, and thus, does not show a clearly spatially separated peak in the velocity-integrated [C ii] map (Figure 7). However, the [C ii] line emission clearly extends toward LBG-1a in maps of narrower velocity bins (Figure 8). No [C ii] emission is detected toward LBG-2 and LBG-3 down to 3σ limits of 0.21 and 0.21 Jy km s⁻¹, assuming the same line width and redshift as measured for LBG-1. We also stacked the spectra of LBG-2 and LBG-3, which did not result in a detection either (Figure 10, right). We caution that this stack only provides a $\sqrt{2}$ deeper [C ii] limit if these galaxies are at a common redshift.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

We have successfully detected the OH(²Π_1/2 J = 3/2→1/2) doublet toward AzTEC-3 (Figure 9, middle). We measure peak flux densities of 1.46 ± 0.22 and 2.07 ± 0.22 mJy for the two components of the Λ-doublet, at a common line FWHM of 384 ± 43 km s⁻¹ for each component. This corresponds to an integrated line flux of 1.44 ± 0.13 Jy km s⁻¹ (Table 4). This 163 μm OH feature thus has a higher line flux than any CO lines detected in this source, carrying almost 20% of the flux of the [C ii](²P_3/2→²P_1/2) line. The central velocity of the feature is shifted by –109 ± 19 km s⁻¹ relative to the [C ii] line. This may indicate that the OH emission is associated with an outflow, but better sensitivity and higher spatial resolution observations are required to further investigate such a scenario.

Table 4. Line Fluxes and Luminosities in AzTEC-3

	Iline	$L^{\prime }_{\rm line}$	Ref.
	(Jy km s−1)	(1010 K km s−1 pc2)
CO(J = 2→1)	0.23 ± 0.03	5.84 ± 0.78	1
CO(J = 5→4)	0.92 ± 0.09	3.70 ± 0.37	1
CO(J = 6→5)	1.36 ± 0.19	3.82 ± 0.54	1
CO(J = 16→15)	<0.22a	<0.09a	2
OH(2Π1/2 J = 3/2→1/2)	1.44 ± 0.13	0.57 ± 0.05	2
[C ii](2P3/2→2P1/2)	8.21 ± 0.29	3.05 ± 0.11	2

Notes. All quoted upper limits are 3σ. ^aAssuming the same width as measured for the CO(J = 5→4) line (R10). Does not account for any uncertainty due to prior subtraction of OH emission (the closer OH component peaks ∼580 km s⁻¹ redwards of the line). References. (1) R10; (2) this work.

Download table as:  ASCIITypeset image

Our observations also covered the CO(J = 16→15) line in AzTEC-3, but no emission was detected (Figure 9, right). We place a 3σ limit of <0.22 Jy km s⁻¹ on the strength of the line, assuming the same width as that of the CO(J = 5→4) line (487 ± 58 km s⁻¹; R10). This limit is 4–6 ×  lower than the fluxes measured in the CO(J = 5→4) and CO(J = 6→5) lines, suggesting that no strong component with high CO excitation is present.

Based on the [C ii] redshift, we have used a recent data set (PI: Riechers) from the Karl G. Jansky Very Large Array (VLA) to search for CO(J = 2→1) line emission toward LBG-1. We do not detect any signal down to an approximate 3σ limit of <0.03 Jy km s⁻¹, assuming the same width and redshift as for the [C ii] line (Figure 11; these data will be described in detail in a future publication, D. A Riechers et al. 2015, in preparation). Given this non-detection, we do not consider the limits on the OH(²Π_1/2 J = 3/2→1/2) and CO(J = 16→15) lines from the ALMA data to be constraining for this source. Given the nondetections of [C ii], we do not extract limits on these lines for LBG-2 and LBG-3 either.

Zoom In Zoom Out Reset image size

3.4.1. AzTEC-3

3.4.2. Lyman-Break Galaxies

Given the detailed available photometry from COSMOS, we match galaxy templates to the observed SED of LBG-1 (see Figure 12, left). Fits are obtained by normalizing all templates to the observed Subaru i-band flux of the LBG (i.e., rest-frame ultraviolet light). Differences in photometry shortward of the Lyα line are not considered further in the comparison, since the nearby galaxy templates do not account for intergalactic medium absorption at z = 5.3 due to the Gunn–Peterson effect (Gunn & Peterson 1965). The ALMA continuum limit for LBG-1 is inconsistent with the SED shapes of spiral, starburst, and dust-obscured galaxies in the nearby universe, but is consistent with the flatter SED shapes typically observed in nearby dwarf galaxies. Similar results are found for LBG-2 and LBG-3 when assuming the same redshift as for LBG-1 (Figure 12, middle and right). To determine the L_FIR of LBG-1, we thus integrated the dwarf galaxy templates over their infrared peaks.²⁰ This results in L_FIR limits of <1.1–3.4 × 10¹¹ L_☉.²¹ We conservatively scale the highest of these templates to the ALMA limits in the following. We then assume LBG-2 and LBG-3 to have the same redshift and SED shape as LBG-1 to determine limits on their L_FIR.

Zoom In Zoom Out Reset image size

Based on the upper limit for the rest-frame 157.7 μm continuum flux density, we assume standard relations and the same dust absorption coefficient as for AzTEC-3 to place constraints on the dust mass of LBG-1. Assuming a dust temperature of T_dust = 30 K and an opacity power law index of β = 1.5 yields M_dust<3.1–9.4 × 10⁷ M_☉,²² where the range indicates the difference between point source and extended source limits.

We derive line luminosities from the observed [C ii] intensities using standard relations (e.g., Solomon & vanden Bout 2005; Carilli & Walter 2013). We assume a Chabrier (2003) stellar initial mass function (IMF) to derive star formation rates from L_FIR. For AzTEC-3, we use the measured continuum size to estimate the average star formation rate surface density. Based on the line width and [C ii] galaxy sizes measured along the major axis, we then use an isotropic virial estimator (e.g., Engel et al. 2010) to derive dynamical masses for all [C ii]-detected sources (see Table 2).

Despite its high L_FIR, both the continuum and [C ii] line emission in the massive starburst galaxy AzTEC-3 are fairly compact. The bulk of the atomic gas as traced by [C ii] is distributed over a region of only ≲2 kpc radius.²³ In spite of the high signal-to-noise ratio (and thus, high centroid precision in the velocity channels in Figure 5) of our [C ii] detection, there is no evidence for a significant velocity gradient on >1 kpc scales for the bulk of the emission. This suggests that the comparatively large velocity widths of the atomic and molecular lines are dominantly supported by emission from highly dispersed gas (the median [C ii] velocity dispersion at the spatial resolution of our observations is ∼315 km s⁻¹), rather than ordered rotation. Only at faint levels do we find evidence for tidal structure or outflowing/infalling streams of gas. AzTEC-3 thus appears to be considerably more compact than some other z > 4 SMGs such as GN20 or HDF 850.1 (∼4 times larger in the FIR continuum; Younger et al. 2008; Carilli et al. 2010; Walter et al. 2012a; Neri et al. 2014), but it is more comparable in extent to the z = 6.34 starburst HFLS3 (3.4 kpc × 2.9 kpc diameter in [C ii], 2.6 kpc × 2.4 kpc diameter in the rest-frame 157.7 μm continuum; Riechers et al. 2013).

Based on the size of the far-infrared continuum emission (∼2.1 kpc²) and the star formation rate of 1100 M_☉ yr⁻¹, we find a high star formation rate surface density of Σ_SFR = 530 M_☉ yr⁻¹ kpc⁻² and a L_FIR surface density of Σ_FIR = 0.5 × 10¹³ L_☉ kpc⁻². This is close to the theoretically predicted Eddington limit for starburst disks that are supported by radiation pressure, and is consistent with theories for so-called maximum starbursts (Elmegreen 1999; Scoville 2003; Thompson et al. 2005). Comparable, compact "hyper-starbursts" were found in the z = 6.42 quasar host galaxy J1148+5251, and in the z = 6.34 dusty galaxy HFLS3 (Σ_SFR = 1000 and 600 M_☉ yr⁻¹ kpc⁻²; Walter et al. 2009; Riechers et al. 2013). Nearby, comparable Σ_SFR are only found in the very centers of GMCs or in the nuclei of ultra-luminous infrared galaxies like Arp 220.

The picture of a relatively compact, warm starburst is consistent with the molecular gas excitation in AzTEC-3, which is comparable to that in the similarly compact and warm z = 6.34 starburst HFLS3 (Riechers et al. 2013), but considerably higher than in other, less compact z > 4 SMGs like GN20 and HDF 850.1 (R10; Carilli et al. 2010; Walter et al. 2012a) and in typical z ∼ 2–3 SMGs (e.g., Riechers et al. 2011b, 2011c). The gas excitation in AzTEC-3, however, is lower than in quasar host galaxies, where the active galactic nucleus (AGN) may contribute to the excitation of high-level CO lines (e.g., Riechers et al. 2006, 2009, 2011a; Weiß et al. 2007). In particular, the nondetection of CO(J = 16→15) emission is consistent with no luminous AGN component contributing to the gas heating, in agreement with previous models of the CO excitation (R10). This picture is also consistent with the relatively low L_C ii/L_FIR ratio of 4.0 × 10⁻⁴. This value is comparable to that of HFLS3 (5.4 × 10⁻⁴; Riechers et al. 2013), and lower than in other, less compact z > 4 SMGs like HDF 850.1 (1.7 × 10⁻³; Walter et al. 2012a), but higher than in some quasar host galaxies (e.g., 1.9 × 10⁻⁴ for J1148+5251; Walter et al. 2009). It is in agreement with the (L_C ii/L_FIR)–Σ_IR relation for nearby infrared-luminous galaxies (Diaz-Santos et al. 2013). This ratio suggests the presence of a stronger-than-average far-UV radiation field due to a warm, dense, compact starburst with high Σ_SFR, but gives no direct indication for the presence of an obscured AGN. Given the inferred compactness and high density of the gas and dust, it cannot be ruled out that the optical depth of the dust in AzTEC-3 at rest-frame 157.7 μm is considerable, and thus, that L_C ii is comparatively low due to extinction. Using the L_CO(1–0) found from CO excitation modeling to CO(J = 2→1) and higher-J lines (R10), we find L_C ii/L_CO(1–0) ≃ 2070, which is by ∼50% lower than the value found for HFLS3 (L_C ii/L_CO(1–0) ≃ 3050; Riechers et al. 2013). This would be consistent with a higher dust optical depth in AzTEC-3 compared to HFLS3, resulting in a reduced [C ii] line luminosity. From our SED modeling, we find a dust optical depth of τ_157.7 μm = 1.28 ± 0.04 at the wavelength of the [C ii] emission for AzTEC-3, which is higher than what is found for HFLS3 (τ_157.7 μm≲ 1; Riechers et al. 2013). This suggests that, in contrast to HFLS3, the dust in AzTEC-3 is at least moderately optically thick toward [C ii] line emission. Extinction of the [C ii] line by dust would be consistent with the finding that this line may be somewhat narrower than the CO lines (421 ± 19 versus 487 ± 58 km s⁻¹; R10). However, it is important to point out that internal, differential gas excitation between different components within the galaxy could easily account for such a small, barely significant difference in line width as well. We do not correct the [C ii] line luminosity for extinction, but we note that the above picture would remain essentially unchanged for even a factor of a few higher L_C ii.

The observed $L^{\prime }_{\rm line}$ ratio, and thus, the brightness temperature (T_b) ratio between the [C ii](²P_3/2→²P_1/2) and CO(J = 2→1) lines is 0.52 ± 0.07 (Table 4). Previous two-component CO excitation modeling suggests optical depths of τ_CO 2–1 = 1.5 and 35.9 for the low- and high-excitation gas components that are estimated to contribute 27.5% and 72.5% to the CO(J = 2→1) line luminosity, respectively (obtained from the models shown by R10), which implies that the CO emission is optically thick. Assuming that the [C ii] and CO emission emerge from regions of similar surface area, the similarity in observed T_b would suggest that the [C ii] line emission either is optically thick as well, or alternatively, that it has an intrinsically higher line excitation temperature than the CO emission ($T_{\rm ex}^{\rm CO\,2\hbox{--}1}$ = 18.3 and 44.9 K at kinetic temperatures of T_kin = 30 and 45 K for the low- and high-excitation gas components in the model of R10). The latter may be expected if a large fraction of the [C ii] emission is associated with PDRs (e.g., Stacey et al. 2010).

Finally, the detection of strong OH(²Π_1/2 J = 3/2→1/2) emission at almost 20% of the [C ii] line luminosity is also consistent with a warm starburst with an intense radiation field. The high upper level energy and high critical density of this transition (E_up/k_B ≃ 270 K and n_crit ≃ 10^8.6 cm⁻³, respectively) makes collisional excitation unlikely, but instead indicates the presence of a strong infrared radiation field. This is consistent with what was found for the z = 6.34 dusty starburst HFLS3, the only other high-z galaxy detected in the 163 μm OH doublet to date (Riechers et al. 2013). The slight blueshift of the OH feature relative to the CO and [C ii] lines in AzTEC-3 would be consistent with the presence of a molecular outflow.

We find a molecular gas mass fraction of f_gas = $M_{\rm H_2}$/M_dyn ≃ 55%, with ∼10% of M_dyn being due to stellar mass. Taken at face value, this would indicate that ≲35% of the mass of AzTEC-3 in the central ∼4 kpc are due to dark matter. Conversely, even if no dark matter were to be present, these considerations place an upper limit on the CO luminosity to gas mass conversion factor of α_CO < 1.3 M_☉ (K km s⁻¹ pc²)⁻¹, which is ∼3×  lower than the Galactic α_CO.²⁴ Using α_CO = 0.8 (<1.3) M_☉ (K km s⁻¹ pc²)⁻¹, we find a gas-to-dust mass ratio of $M_{\rm H_2}$/M_dust ≃ 200 (<330), i.e., ∼3×  higher than in HFLS3 when assuming the same α_CO (Riechers et al. 2013). These values are higher than the average of 120 ± 28 found for the nuclei of nearby infrared-luminous galaxies, but within the range of values measured for individual sources (29–725; Wilson et al. 2008).

AzTEC-3, as the most massive, most intensely star-forming galaxy in the protocluster, thus is a compact, gas-dominated galaxy that hosts a maximal starburst. Its relative compactness, and implied high Σ_SFR appear not to be a common feature among z > 4 SMGs, but are comparable to the most extreme other cases known. In particular, the z = 4.05 galaxy GN20, which is also associated with an overdensity of galaxies (Daddi et al. 2009), is significantly more extended by a factor of a few and has lower Σ_SFR and lower molecular gas excitation (e.g., Carilli et al. 2010). The z = 5.18 SMG HDF 850.1 is also associated with an (less pronounced) overdensity of galaxies, is significantly more extended by a factor of a few, and has lower Σ_SFR and lower molecular gas excitation than AzTEC-3 (Walter et al. 2012a). Assuming that these systems are the progenitors of the same population of massive central cluster galaxies, AzTEC-3 thus appears to be in a different phase of its evolution. This makes it particularly interesting to further understand how this may be connected to the properties of its environment.

We solidly detect and spatially resolve [C ii] emission in the "typical" z > 5 star-forming system LBG-1. The atomic gas is spatially and dynamically resolved toward all three optically identified sources. We resolve at least two major velocity gradients between the different components (Figure 7). The smaller gradient appears to be associated with the northern component LBG-1c. The larger gradient appears to extend from the southern component LBG-1a across the middle component LBG-1b toward LBG-1c (blue- to redshifted emission), but it may mask smaller-scale gradients associated with the individual components at the present spatial resolution. In particular, there is tentative evidence for smaller-scale velocity gradients with opposing directions, pointing from the "overlap" region in between LBG-1a and LBG-1b towards the centers of the two optical sources, but higher spatial resolution is required to confirm and more clearly separate these smaller-scale gradients.²⁵ This system thus may represent a merger of three Lyman-break galaxies, or some other form of complex, clumpy, extended system in formation. The dynamical mass of the triple system is about half that of the SMG AzTEC-3 (Table 2), showing that LBG-1 contributes a significant fraction of the mass bound in galaxies to the protocluster environment.

The [C ii] emission integrated over all components is ∼4×  fainter than in the SMG AzTEC-3, suggesting that each of the components is typically by about an order of magnitude fainter than the SMG. There is no evidence for continuum emission at the position of LBG-1, suggesting that it is at least ≳15×  fainter (>25–40×  fainter for the individual subcomponents) than the SMG (3σ). Assuming an SED shape similar to AzTEC-3, this would suggest an SFR of SFR_FIR < 80 M_☉ yr⁻¹ (≲25–40 M_☉ yr⁻¹ for the subcomponents). From the [C ii] luminosity, we obtain an estimate of SFR_FIR ∼ 150 M_☉ yr⁻¹, using the relation for dusty starbursts recently suggested by Sargsyan et al. (2012). This is somewhat inconsistent with the continuum-based estimate, unless a bottom-heavy IMF is assumed (e.g., Conroy & van Dokkum 2012). However, despite the limited constraints on the rest-frame far-infrared SED of the source, typical spiral/starburst and dusty galaxy SED templates can be ruled out to fit the SED of LBG-1 (Figure 12). Templates for dwarf galaxies provide a substantially better match to the overall SED, suggesting that LBG-1 is not a (very) dusty system, and providing an upper limit to the SFR of SFR_FIR < 18–54 M_☉ yr⁻¹ (the range represents the difference between point source and extended source limits, since the actual limit depends on the size of the emitting region; see Table 2). We consider this to be the currently best estimate for the FIR SFR in this system. From the range of templates that best represent the data, the possibility of a comparatively low, sub-solar metallicity cannot be excluded as an important reason for the low implied FIR flux of the system. However, the detection of strong 157.7 μm [C ii] emission and the presence of deep interstellar metal absorption features in rest-frame UV spectra (C11) is inconsistent with very low metallicities. We thus consider it most likely that the SED shape is due to the combination of a young starburst and some contribution of optical light from a stellar population that is already in place. In particular, the lack of a strong "bump" at rest-frame optical wavelengths (relative to the strength of the rest-frame UV emission that is dominated by young, massive stars) may indicate the lack of a significant older stellar population. Such a scenario is consistent with what is expected for a young, recently formed galaxy observed at an early cosmic epoch, only 1.1 billion years after the Big Bang. We independently estimated the star formation rate based on the rest-frame UV continuum emission of the galaxy (e.g., Madau et al. 1998), using the SED fitting methods outlined by C11. This yields SFR_UV = 22 M_☉ yr⁻¹. This is comparable to the best SFR_FIR limit, and again consistent with low dust extinction. By assuming SFR_total = SFR_UV+SFR_FIR, we find that at most <45%–71% of the star formation activity in LBG-1 is obscured by dust. The UV-to-optical SED shape is consistent with a dust extinction of only A_V ≃ 0.5 mag (C11). The finding of low dust extinction is consistent with recent estimates for LBGs at comparable and higher redshifts in the rest–frame optical (e.g., Bouwens et al. 2010), suggesting that detection of such galaxies in the FIR continuum may require substantial integration times even with ALMA. This is also consistent with recent, less sensitive far-infrared studies of Lyman-α emitters (Walter et al. 2012b; Gonzalez-Lopez et al. 2014). Given its stellar mass (C11) and total star formation rate, LBG-1 is consistent with being situated on the star-forming "main sequence" at z ≈ 5 (e.g., Speagle et al. 2014).

From the [C ii] luminosity and L_FIR limit, we find an L_C ii/L_FIR limit of >3.1 × 10⁻³ to >9.4 × 10⁻³ (the higher limit applies if the far-infrared continuum emission is not resolved by our observations), i.e., approximately >0.3% to >0.9%. This is consistent with normal, star-forming galaxies at low redshift, but significantly higher than in dusty quasars and extreme starburst galaxies (e.g., Gracia-Carpio et al. 2011; Diaz-Santos et al. 2013). Assuming thermalized excitation between CO(J = 2→1) and CO(J = 1→0), we find a L_C ii/L_CO(1–0) limit of >4600. This is consistent with a comparatively modest UV radiation field strength (e.g., Stacey et al. 2010), comparable to normal, star-forming galaxies in the local universe when accounting for the reduction in CO(J = 2 → 1) luminosity due to the CMB at the redshift of LBG-1.²⁶

Our findings are consistent with the assumption that the protocluster member LBG-1 is a "typical," close to $L^\star _{\rm UV}$ star-forming system at z > 5 with relatively low dust content. LBG-1 appears to be a triple system, but our sensitivity would have been sufficient to pick up a single component with one-third of the [C ii] luminosity within the protocluster. We thus conclude that LBG-2 and LBG-3, which have photometric redshifts/colors consistent with the protocluster environment, either have [C ii] luminosities that are lower than those of the individual components of LBG-1 by at least a factor of three (which would be consistent with their UV/optical properties within the relative uncertainties), or have redshifts outside of our bandpass (dz ≃ 0.04 per 1.875 GHz band). Independent spectroscopic confirmation and/or more sensitive [C ii] observations will be required to distinguish between these possibilities. Note that eight additional LBGs identified as likely protocluster members at larger distances from the center (C11) were not covered by this initial investigation, and thus, could have higher [C ii] and/or FIR luminosities than LBG-1.