Initial Evaluation of SNEMO2 and SNEMO7 Standardization Derived from Current Light Curves of Type Ia Supernovae

In order to address the issue of unmodeled spectral diversity, Saunders et al. (2018) presented the Super-Nova Empirical MOdels (SNEMO), which apply expectation maximization factor analysis (EMFA, a dimensionality reduction algorithm similar to principal component analysis) to optical spectrophotometric time series data obtained by the Nearby Supernova Factory (SNfactory; Aldering et al. 2002). EMFA reduces the dimensionality of the training data set to a predefined number of eigenvectors. In the case of SNEMO, these eigenvectors are a time series of spectra (Saunders et al. 2018, Equations (7) and (10)). Combined, these eigenvectors represent a linear basis from which one can reconstruct any optical SN Ia spectral time series. This method of defining the eigenvectors is similar to the method used to define SALT2's x₁ (Guy et al. 2007, Section 5), however the EMFA algorithm used to obtain the SNEMO components handles missing and noisy data in a different manner and does not use any photometric training data. Additionally, SNEMO does not fit a variable color law and instead assumes a Fitzpatrick & Massa (2007) reddening law (Saunders et al. 2018, Section 3.2). Like SALT2, each of the best-fit model coefficients (or eigenvector projections) describe a certain light-curve shape and can be combined to standardize supernova magnitudes. Unlike some light-curve shape parameters (e.g., ${\rm{\Delta }}{m}_{15}$), these EMFA eigenvectors are pure mathematical constructs and do not necessarily connect to anything physical or intuitive.

SNEMO is a family of models trained on the same data. Saunders et al. (2018) released three variants⁹ : SNEMO2, SNEMO7, and SNEMO15.¹⁰ SNEMO2 is named for its two spectral-temporal eigenvectors; SNEMO7 and SNEMO15 have seven and fifteen eigenvectors, respectively. In addition, each SNEMO model has a color correction curve that is identical to the Fitzpatrick & Massa (2007) reddening law. The "zeroth" eigenvector, describing the mean spectral-temporal evolution, is related to m_B in Equation (1), and its corresponding coefficient used only as an overall scaling factor. The other spectral-temporal and color parameters are combined linearly to standardize SNe Ia.

SNEMO2, which consists of a mean vector, one spectral-temporal component of variation, and a color law, is directly analogous to SALT2, differing only in the training data and methodology used to obtain the model components. SNEMO2 allows for a direct comparison between the SNEMO and SALT2 training methodologies without introducing any more degrees of freedom to the model. The other SNEMO models introduce more parameters to allow the model to capture more of the spectral variation. In the initial release of SNEMO, Saunders et al. (2018) showed that these extra parameters do improve the quality of the model in fitting the diversity of SN Ia behavior.

Using the SNfactory training and a separate SNfactory validation set, SNEMO15 was found to be the model able to capture the most spectral diversity while avoiding overfitting. SNEMO7 was considered to be a model that well-sampled multiband light curves should be able to constrain, while capturing more SN Ia variation than SNEMO2 (or SALT2). In addition, SNEMO7 was determined to be the point of diminishing returns when using Tripp-like linear standardization. It is worth noting that there is evidence for the further consideration of nonlinear spectral behavior (e.g., ejecta velocities) that may require the more descriptive spectral fits obtained with SNEMO15.

For a further understanding of the similarities and difference between SALT2 and SNEMO, we plot the correlations between the model parameters in Figure 1. Figure 13 of Saunders et al. (2018) shows this same plot but for SNEMO2 parameters measured from spectrophotometric data. This work focuses on SNEMO7 parameters derived from photometric data, so Figure 1 uses our nominal SNEMO7 data set. Details on the data and model are described in Section 2.

Zoom In Zoom Out Reset image size

In this work, we perform the initial test of how well SNEMO7 standardizes SNe Ia using only publicly available photometric light-curve data. This goes beyond the spectrophotometric time series data set used in the development and initial testing of the SNEMO models. We include a host stellar mass term as a proxy for any uncorrected SN Ia astrophysical systematics. It is these possible unknown systematics that stand as the largest threat to precision dark energy measurements. Host stellar mass has become a standard proxy since Kelly et al. (2010). However more recent research by Gupta et al. (2011), Hayden et al. (2013), Rigault et al. (2013, 2018), Childress et al. (2013, 2014), Moreno-Raya et al. (2018), Rose et al. (2019), and others, show that alternative astrophysical measurements may better match the true physical mechanism.

We use the following criteria to evaluate SNEMO7's ability to standardize current photometric SNe Ia data sets:

• 1.  
  When applying the model to current light-curve-only data, are the standardization coefficients consistent with those derived from the spectrophotometric time-series training data set?
• 2.  
  How many standardization coefficients are distinguishable from zero?
• 3.  
  What are the correlations between the coefficients? Strong correlations imply that a projection needs to be fit even if the standardization coefficient is consistent with zero.
• 4.  
  Given current data sets, does SNEMO7 reduce the need for unexplained intrinsic scatter in SN Ia (${\sigma }_{\mathrm{unexplained}}$) in the Hubble–Lemaître diagram?
• 5.  
  Does SNEMO7 reduce the correlations with host galaxy properties, such as the one with stellar mass ($\gamma$)? A reduction of these correlations would imply a reduced systematic floor for SN Ia standardization.

These tests do not attempt to validate or characterize SNEMO's ability to fit light curves, but rather focus on questions concerning population-level effects that have the potential to impact cosmological measurements. Characterizing light-curve fits will be done thoroughly in a forthcoming paper (Saunders et al. 2020). We will also not investigate the limits of Tripp-like standardization equations or methods. Finally, we are here only asking how SNEMO fares on these five criteria when given the current quality of SN Ia data sets, not how it performs when given the data quality expected from LSST or WFIRST. These are all important research topics and should be discussed independently.

In Section 2, we discuss the photometric data and the method we used to test SNEMO7, and in Section 3, we discuss our findings and present answers the five questions above. In Section 4, we discuss how these results impact SN Ia cosmology in the systematics dominated era of LSST and WFIRST.

The three released SNEMO models are available in the sncosmo python package¹¹ (version 1.7). We use the mcmc_lc function in that package to find the posterior distribution of the best-fit model coefficients (i.e., the eigenvector projections, c_i) for each SN Ia and estimate their uncertainties (${\sigma }_{i}$) from these posteriors. This function uses Markov Chain Monte Carlo to sample from

$$\begin{eqnarray}&&{\chi }^{2}={\left({{\boldsymbol{f}}}_{\mathrm{obs}}-{{\boldsymbol{f}}}_{\mathrm{mod}}\right)}^{\top }{\left({{\rm{\Sigma }}}_{\mathrm{obs}}+{{\rm{\Sigma }}}_{\mathrm{mod}}\right)}^{-1}({{\boldsymbol{f}}}_{\mathrm{obs}}-{{\boldsymbol{f}}}_{\mathrm{mod}}).\end{eqnarray} \tag{ 2 }$$

This is a function of the model coefficients $(z,{t}_{0},{c}_{i},{A}_{s})$, where ${{\boldsymbol{f}}}_{\mathrm{obs}}(b,p)$ is the flux observed in bandpass b at phase p, and ${{\boldsymbol{f}}}_{\mathrm{mod}}(b,p;z,{t}_{0},{c}_{i},{A}_{s})$ is the flux predicted in bandpass b at phase p, obtained by performing synthetic photometry on the spectral time series model with the given model coefficients. A diagonal covariance matrix whose entries represent the observational uncertainty in each bandpass and phase observed (${{\rm{\Sigma }}}_{\mathrm{obs}}$) is added to the model covariance (${{\rm{\Sigma }}}_{\mathrm{mod}}$) to obtain the full covariance matrix used in the light-curve fits. The population dispersion of a given component (c_i) is normalized to approximately 1, meaning ∼1000 normal SNe Ia should have a c_i range of ∼−3 to 3. Further details on the interpretation of SNEMO parameters can be found in Saunders et al. (2018). In addition to the SNEMO coefficients, the time of maximum brightness is fit along with the model coefficients with wide, uniform priors ((−50, 50) for each of the model coefficients, and $(\min ({t}_{\mathrm{obs}})-20,\max ({t}_{\mathrm{obs}}))$ for the time-of-max). When running the inference, we let the redshift in SNEMO vary within the uncertainty of the measurement (∼0.0001). We also correct for Milky Way dust reddening using the Schlafly & Finkbeiner (2011) maps.

In a SALT2-like analysis, initial light-curve quality cuts based on phase sampling and signal-to-noise are usually applied. We do not yet have a similar heuristic for which light curves are high enough quality to be fit with SNEMO7. Instead, we filter the SNe Ia on the SNEMO7 parameters and uncertainties directly, rather than any other measured properties of the light curves. We define an object to be well fit by SNEMO7 when its eigenvector coefficient values are less than a threshold ($| {c}_{i}| \lt 5$) and the uncertainties on those coefficients are also smaller than another threshold (${\sigma }_{i}\leqslant 2$). The cut on $| {c}_{i}| \lt 5$ is intended to remove large outliers, and the cut on ${\sigma }_{i}\gt 2$ removes SN Ia that have an uncertainty in the best-fit coefficients larger than twice the 1σ population dispersion. A high ${\sigma }_{i}$, e.g., $\gt 2$, represents data that can be fit by a wide range of models, effectively putting no constraint on the true values of the model parameters. We investigate the effect of different ${\sigma }_{i}$ cutoff values on our results in Appendix. These quality cuts remove unconstrained fits without excessively restricting our sample size. This results in a nominal data set of $N=240$.

SNEMO7 does not yet have an uncertainty model (the ${{\rm{\Sigma }}}_{\mathrm{mod}}$ in Equation (2)). A formal uncertainty model describes the regions in parameter space where SNe Ia are more diverse than the model and reduces the impact these regions have on fitting data. This uncertainty model is under development (C. Saunders et al. 2020, in preparation), but in this work we need to look at the effect of treating the model as imperfect. For SALT2, the uncertainty model is partially determined by the statistical uncertainty from their training data set (Guy et al. 2007); the model is more certain in areas that had more training data. For SNEMO, the training data was selected to all have the same rest-frame wavelength coverage. As such, this part of the uncertainty model should be smooth. The other part of the SALT2 uncertainty model describes correlated residuals around the model. We expect this component to be reduced for SNEMO7, as it describes more of the intrinsic SN behavior. Using these assumptions, we investigate the effects of an imperfect model using a simplified uncertainty model. This naive uncertainty model consists of a diagonal covariance matrix with entries given by 1% or 2% of the peak flux value in each band. The formal uncertainty model in development, has more variation in phase than our naive model, but its scale is within this range.

The addition of these uncertainties degrades the coefficient measurement precision (i.e., ${\sigma }_{i}$), therefore reducing the number of SN Ia passing quality cuts. With a 1% naive uncertainty model, the number of SN Ia passing our quality cuts are $N=194$, and dropping to $N=126$ with the 2% uncertainty model. Ultimately, the data sets that survive these cuts are dominated by CSP SNe Ia. Table 1 shows how varying the light-curve fit quality cuts and uncertainty model affects the total number of SNe Ia in our sample.

Several factors contribute to the poor constraints on the model parameters. A large factor is wavelength coverage. The SNEMO model is defined from 3300 to 8600 Å, and any observations in bands with rest-frame wavelengths outside of this range are not used to constrain the model parameters. As an example, we find that all of the SNLS objects that pass our cuts are at redshifts below ∼0.7, which is where the effective wavelength of the r-band falls below the lower bound of the SNEMO wavelength range. The signal-to-noise ratio of the observations or the temporal sampling of the light curves can also have an impact on our ability to constrain the model parameters in the light-curve fits. A full study of these effects is left to future work.

We used the Unified Nonlinear Inference for Type Ia cosmologY (UNITY) framework to estimate the standardization equation, Equation (3) below. UNITY, a Bayesian hierarchical model implemented in Stan (Carpenter et al. 2017) using pystan (Riddel et al. 2018), was developed by Rubin et al. (2015) and further refined by Hayden et al. (2019). A more recent version (UNITY1.2) now includes the capability of modeling Tripp-like standardization equations with an arbitrary number of standardization parameters.¹² Because our focus is on standardization and not cosmology directly, we assume a flat, ΛCDM cosmology with ${{\rm{\Omega }}}_{M}=0.3$.

All spectral energy distribution models considered (SALT2, SNEMO2, and SNEMO7) are unable to achieve a dispersion in distance modulus that is consistent with measurement uncertainties and linear standardization. We model the remaining "unexplained" dispersion with a model parameter: unexplained intrinsic scatter, ${\sigma }_{\mathrm{unexplained}}$. We assume ${\sigma }_{\mathrm{unexplained}}$ describes the width of a Gaussian distribution. More details about this parameter are described in Rubin et al. (2015, Section 2.7).

We use a simple Gaussian mixture model in magnitude for modeling the outlier distribution (see Kunz et al. 2007). Thus, we have no explicit outlier rejection, but as SNe Ia get further from their predicted rest-frame B-band magnitudes, they are more and more likely to be described by the outlier distribution. We fix the width of the outlier Gaussian to 0.5 mag, added in quadrature with the measurement uncertainties, and allow the fraction of SNe Ia in this distribution to be a model parameter. A further explanation is presented in Rubin et al. (Section 2.3 of 2015).

Using the SNEMO7 model with UNITY requires a total of eight standardization coefficients: six for the light-curve-shape eigenvectors, one for the color law, and finally a coefficient describing the effect (if any) of host galaxy stellar mass. These can be combined into a standardized distance modulus equation, following the Tripp convention

$$\begin{eqnarray}&&\mu ={m}_{B}-\left({M}_{B}+\beta {A}_{s}+\gamma m+\sum _{i=1}^{N}{\alpha }_{i}{c}_{i}\right),\end{eqnarray} \tag{ 3 }$$

where μ, m_B, and M_B are the distance modulus, apparent and absolute magnitude, respectively, the same as Equation (1). For SALT2 and SNEMO2, N = 1, but for SNEMO7, N = 6. A_s and β are the color term and color standardization coefficient, respectively. A_s is a spectral variant of the traditional ${A}_{{\rm{V}}}$ extinction. For comparison to SALT2, A_s should be approximately $({R}_{V}+1)c$, meaning that β should be ∼1. Finally, γ is the standardization coefficient applied to the logarithm of the stellar host galaxy stellar mass (m).¹³ The zero-point of m is shifted such that the data set's average is zero, partially decorrelating γ and the absolute magnitude M_B. As this standardization equation uses the same sign for all of the coefficients, these α coefficients have the opposite signs as the one in Equation (1). Due to the small sample sizes, we ran UNITY1.2 without estimating selection effects or calibration offsets between data sets.

In order to test if SNEMO can improve the Hubble–Lemaître diagram unexplained dispersion or reduce the correlations with host galaxy properties, we first need a baseline for our comparison. As such, we use the SALT2.4 version of SALT2 to fit the SNe Ia that passed basic quality cuts for SNEMO2 and SNEMO7. The results were then put into UNITY1.2 to estimate the standardization coefficients of Equation (3). The nominal value for a SALT2 Tripp-like mass standardization parameter, with the data set that can constrain SNEMO7, was found to be $\gamma =-0.01\pm 0.02$. This is in agreement with $\gamma =0.042\pm 0.013$ seen in Sullivan et al. (2010). Note our inflated uncertainties due to the smaller sample size. When looking at the SNe Ia that can constrain SNEMO2, we measure a nominal value of $\gamma =-0.043\pm 0.010$, nearly identical to that of Sullivan et al. (2010). This supports our conclusion that our high threshold for light-curve quality does not introduce large biases in the standardization parameters. The full fit can be seen in Figure 2 with the numerical values presented in Tables 2 and 3.

Zoom In Zoom Out Reset image size

Table 2.  Parameter Estimation Results from UNITY1.2 for SN Ia That Passed SNEMO2 Quality Cuts

	SALT2	SNEMO2
% Error Model		0	1	2
Data set size	867a	885a	886	881
MB	$-{19.193}_{-0.006}^{+0.006}$	$-{19.444}_{-0.010}^{+0.011}$	$-{19.452}_{-0.011}^{+0.011}$	$-{19.455}_{-0.011}^{+0.011}$
${\sigma }_{\mathrm{unexplained}}$	${0.116}_{-0.005}^{+0.006}$	${0.129}_{-0.007}^{+0.007}$	${0.117}_{-0.007}^{+0.007}$	${0.102}_{-0.008}^{+0.008}$
β	${2.85}_{-0.07}^{+0.06}$	${0.88}_{-0.04}^{+0.03}$	${0.91}_{-0.04}^{+0.03}$	${0.91}_{-0.03}^{+0.04}$
${\alpha }_{1}$	$-{0.126}_{-0.006}^{+0.006}$	${0.044}_{-0.007}^{+0.007}$	${0.045}_{-0.007}^{+0.007}$	${0.045}_{-0.008}^{+0.008}$
γ	$-{0.043}_{-0.010}^{+0.010}$	$-{0.039}_{-0.014}^{+0.014}$	$-{0.042}_{-0.014}^{+0.014}$	$-{0.039}_{-0.015}^{+0.014}$
No. of outliers	15 (1.7%)	21 (2.4%)	20 (2.3%)	26 (3.0%)

Notes. The SN Ia used in the SALT2 analysis are the ones that passed the ${\sigma }_{i}\leqslant 2$ for SNEMO2 with no error model and were successfully fit with SALT2. The "No. of outliers" is reported both as an absolute number and a percentage of the data set.

^aFor the same initial data set, SNEMO2 has six additional SNe Ia rejected as cosmological outliers, where as 18 additional SNe Ia that were rejected at the SALT2 light-curve fitting stage.

Download table as:  ASCIITypeset image

Table 3.  Parameter Estimation Results from UNITY1.2 for SN Ia That Passed SNEMO7 Quality Cuts, ${\sigma }_{i}\leqslant 2$.

	SALT2			Saunders et al. (2018)	SNEMO7
% Error Model					0	1	2
Data set size	240	194	126	133	240	194	126
MB	$-{19.197}_{-0.011}^{+0.011}$	$-{19.196}_{-0.012}^{+0.012}$	$-{19.188}_{-0.017}^{+0.017}$	⋯	$-{19.53}_{-0.03}^{+0.02}$	$-{19.52}_{-0.03}^{+0.03}$	$-{19.50}_{-0.05}^{+0.05}$
${\sigma }_{\mathrm{unexplained}}$	${0.135}_{-0.009}^{+0.009}$	${0.131}_{-0.011}^{+0.011}$	${0.141}_{-0.015}^{+0.015}$	⋯	${0.125}_{-0.010}^{+0.011}$	${0.121}_{-0.012}^{+0.011}$	${0.10}_{-0.03}^{+0.02}$
β	${3.02}_{-0.10}^{+0.10}$	${2.84}_{-0.10}^{+0.10}$	${2.96}_{-0.14}^{+0.14}$	1.08 ± 0.04	${1.03}_{-0.04}^{+0.05}$	${1.01}_{-0.05}^{+0.05}$	${1.01}_{-0.09}^{+0.09}$
${\alpha }_{1}$	$-{0.125}_{-0.010}^{+0.010}$	$-{0.129}_{-0.012}^{+0.012}$	$-{0.122}_{-0.017}^{+0.017}$	0.16 ± 0.03	$-{0.05}_{-0.02}^{+0.02}$	$-{0.08}_{-0.03}^{+0.03}$	$-{0.09}_{-0.11}^{+0.12}$
${\alpha }_{2}$	⋯	⋯	⋯	0.02 ± 0.03	${0.045}_{-0.017}^{+0.017}$	${0.07}_{-0.02}^{+0.02}$	${0.11}_{-0.05}^{+0.06}$
${\alpha }_{3}$	⋯	⋯	⋯	0.103 ± 0.017	$-{0.031}_{-0.013}^{+0.013}$	$-{0.026}_{-0.017}^{+0.016}$	$-{0.01}_{-0.04}^{+0.04}$
${\alpha }_{4}$	⋯	⋯	⋯	0.01 ± 0.02	$-{0.054}_{-0.012}^{+0.012}$	$-{0.069}_{-0.018}^{+0.017}$	$-{0.07}_{-0.11}^{+0.07}$
${\alpha }_{5}$	⋯	⋯	⋯	0.045 ± 0.009	$-{0.037}_{-0.009}^{+0.009}$	$-{0.036}_{-0.015}^{+0.016}$	$-{0.02}_{-0.08}^{+0.09}$
${\alpha }_{6}$	⋯	⋯	⋯	−0.041 ± 0.017	${0.011}_{-0.011}^{+0.010}$	${0.025}_{-0.016}^{+0.015}$	${0.04}_{-0.11}^{+0.09}$
γ	$-{0.01}_{-0.02}^{+0.02}$	$-{0.03}_{-0.03}^{+0.03}$	$-{0.04}_{-0.05}^{+0.05}$	⋯	$-{0.03}_{-0.02}^{+0.02}$	$-{0.04}_{-0.03}^{+0.03}$	$-{0.07}_{-0.11}^{+0.12}$
No. of outliers	4 (1.7%)	4 (2.1%)	4 (3.2%)	⋯	7 (2.9%)	3 (1.5%)	4 (3.2%)

Note. The SN Ia used in the SALT2 analysis are the ones that passed each of the SNEMO7 analyses, respectively. The "No. of outliers" is reported both as an absolute number and a percent of the data set.

Download table as:  ASCIITypeset image

Our first objective was to test the modeling methodology used by SNEMO. With only two model parameters to fit, SNEMO2 allows for a direct comparison between SALT2 and SNEMO. Figure 3 shows the resulting posterior, as inferred by UNITY1.2, for the SNEMO2 standardization equation. Numerical values are presented in Table 2.

Zoom In Zoom Out Reset image size

SNEMO2's standardization parameters are well constrained and independent of the uncertainty model. Each data set analyzed corresponds to one of the three different uncertainty models, uses our default quality cuts of ${\sigma }_{i}\leqslant 2$ and $| {c}_{i}| \lt 5$, and includes more than 800 SN Ia. There are no previous measurements for the SNEMO2 ${\alpha }_{1}$ or β, but ${\sigma }_{\mathrm{unexplained}}$ and γ can be compared to the values from SALT2. To calculate ${\sigma }_{\mathrm{unexplained}}$ you need to first remove the scatter characterized by the uncertainty model (${{\rm{\Sigma }}}_{\mathrm{mod}}$, Equation (2)). For even a modest uncertainty model, SNEMO2 and SALT2 have a comparable unexplained intrinsic scatter. In addition, the correlation with stellar mass is not statistically different. Finally, 2%–3% of the SN Ia were flagged as cosmological outliers.

Since SNEMO2 is comparable to SALT2 when standardizing SNe Ia, we claim that the modeling details in the SNEMO family of models, e.g., wavelength coverage, use of factor analysis, etc., are well-behaved. Following the idea that SNe Ia exhibit more diversity than can be captured by two parameters (Branch et al. 2006; Kim et al. 2014; Fakhouri et al. 2015; Hayden et al. 2019; Rubin 2019), we proceed to test the seven parameter SNEMO7 model.

The results of the analysis of SNEMO7 and UNITY1.2 are shown in Figure 4. This figure shows the posterior distributions when using the published version of SNEMO7 (with no uncertainty model), as well as with the addition of a 1%, and a 2% peak luminosity uncertainty model. A 3% uncertainty model was also tested, but resulted in an exaggeration of the trends already observed when increasing the uncertainty from 1% to 2%, and as such is not presented.

Zoom In Zoom Out Reset image size

There are a few things that stand out from these results. First, in Table 3, we see that the outlier percentage typically ranges from 1.5% to 3.2%. UNITY1.2 probabilistically separates these into an outlier population, where they do not affect the inlier population variables: ${M}_{B}$, ${\sigma }_{\mathrm{unexplained}}$, ${\alpha }_{i}$, β, and γ. Next, the unexplained intrinsic scatter starts at $0.125\pm 0.011\,\mathrm{mag}$, slightly smaller than that of SALT2, and decreases to $0.10\pm 0.03\,\mathrm{mag}$ if we raise the uncertainty floor to 2%. Finally, as the size of the uncertainty model increases, the sample size decreases and as expected the uncertainty in the standardization parameters increase.

3.2.1. Are the Standardization Coefficients Consistent between Data Sets?

3.2.2. How Many Standardization Coefficients Are Distinguishable from Zero?

3.2.3. What Are the Correlations between the Coefficients?

3.2.4. Does SNEMO7 Reduce Unexplained and Systematic Variations in Standardization?