THE PHOTOMETRIC CLASSIFICATION SERVER FOR Pan-STARRS1

5.1.1. SVM for PanDiSC

5.1.2. PhotoZ for PanZ

In the last decade, several efficient codes for the determination of photometric redshifts have been developed, based on either empirical methods or template fitting. In the first case, one tries to parameterize the low-dimensional surface in color–redshift space that galaxies occupy using low-order polynomials, nearest-neighbor searches, or neural networks (Csabai et al. 2003; Collister & Lahav 2004). These codes extract the information directly from the data, given an appropriate training set with spectroscopic information. Template fitting methods work instead with a set of model spectra from observed galaxies and stellar population models (Padmanabhan et al. 2005; Ilbert et al. 2006, 2009; Mobasher et al. 2007; Pello et al. 2009).

The PhotoZ code used in the PanZ component of the PCS system (see Section 5.2) belongs to this last category and is described in Bender et al. (2001). The code estimates redshifts z by comparing T, a set of discrete template SEDs, to the broadband photometry of the (redshifted) galaxies. For each SED, the full redshift posterior probability function including priors for redshift, absolute luminosity, and SED probability is computed using Bayes' theorem:

$$\begin{equation} P(z,T|F,M,\ldots) \propto p(F|z,T)p(z,T|M), \end{equation} \tag{ 1 }$$

where F is the vector of measured fluxes in different bands, M is the galaxy absolute magnitude in the B band (see below), p(z, T|M) is the prior distribution, and p(F|z, T)∝exp (− χ²/2) is the probability of obtaining a normalized χ² for the given data set, redshift, and template T. In detail, we compute χ² as

$$\begin{equation} \chi ^2=\sum _i\frac{(\alpha f_{i,\rm{mod}}(z)-f_{i,\rm{dat}})^2}{(w_i\alpha f_{i,\rm{mod}})^2+\Delta f_{i,\rm{dat}}^2}, \end{equation} \tag{ 2 }$$

where f_{i, mod}(z) and f_{i, dat} are the fluxes of the templates (at the redshift z) and of the data in a filter band i, and Δf_{i, dat} are the errors on the data. The model weights w_i quantify the intrinsic uncertainties of the SEDs for the specific filter i; presently they are all set to w_i = 0.1. The normalization parameter α is computed by minimizing χ² at each choice of parameters.

The priors are (products of) parameterized functions of the type

$$\begin{equation} p(y)\propto y^n\exp \left[-\ln (2)\left(\frac{y-\hat{y}}{\sigma _y}\right)^p\right], \end{equation} \tag{ 3 }$$

where the variable y stands for redshift or absolute magnitudes. Typically, we use n = 0, p = 6 or 8, and $\hat{y}$ and σ_y with appropriate values for mean redshifts and ranges, or mean absolute B magnitudes and ranges, which depend on the SED type. The absolute magnitudes of the objects are computed on the fly for the considered rest-frame SED, normalization α, and redshift, using the standard cosmological parameters (Ω_m = 0.3, Λ = 0.7, H₀ = 70 Mpc (km s⁻¹)⁻¹). The use of fluxes f_{i, dat} instead of magnitudes allows us to take into account negative data points and upper limits.

The set of galaxy templates is semi-empirical and can be optimized through an interactive comparison with a spectroscopic data set. The original set (Bender et al. 2001; Gabasch et al. 2004) includes 29 SEDs describing a broad range of galaxy spectral types, from early to late to star-bursting objects. Recently, we added a set of SEDs tailored to fit luminous red galaxies (LRGs; Eisenstein et al. 2001; see N. Greisel et al., in preparation), and one SED to represent an average QSO spectrum. This was obtained by averaging the low-redshift HST composite of Telfer et al. (2002) and the SDSS median quasar composite of Vanden Berk et al. (2001). Furthermore, the method also fits a set of stellar templates, allowing a star/galaxy classification and an estimate of the line-of-sight extinction for stellar objects. The templates cover typically the wavelength range λ = 900–25000 Å (with the QSO template covering instead 300–8000 Å) and are sampled with a step typically 10 Å wide (varying from 5 to 20 Å; the QSO SED has Δλ = 1 Å).

The method has been extensively tested and applied to several photometric catalogs with spectroscopic follow-up (Gabasch et al. 2004, 2008; Feulner et al. 2005; F. Brimioulle et al., in preparation). Given a (deep) photometric data set covering the wavelength range from the U to the K band, excellent photometric redshifts with (z_phot − z_spec)/(1 + z) ∼ 0.03 up to z ≈ 5 with at most a few percent catastrophic failures can be derived for every SED type. With the help of appropriate priors, photometric redshifts accurate to 2% (in (z_phot − z_spec)/(1 + z_spec)) with just 1% outliers (see Section 6 for definitions) are obtained for LRGs using the ugriz SDSS photometric data set (N. Greisel et al., in preparation). This is the first time we attempt to determine the photometric redshifts of QSOs. We couple the available SED to a strong prior in luminosity that dampens its probability as soon as the predicted B-band absolute magnitude is fainter than −24.

The code is in C++. A Fortran version is available as implemented under Astro-WISE (Valentijn et al. 2006; Saglia et al. 2012).

Figure 1 describes schematically the components and data flow of PCS. Each component, or module, is a separate unit of the package with a well-defined operational goal. Following the usual convention, it is indicated by a box with bars (for interfaces and/or processes, from A to F, plus PanZ and PanDiSC) or a cylinder (for a database) in Figure 1. There are four databases: two reside in Hawaii (PSPS, where the primary Pan-STARRS1 catalogs are stored and a subset of the output produced by PCS is copied, and MYDB, where the whole of the PCS output goes). The other two are in Garching: the Master, where configuration files and templates are stored, and the local PS1 DATA, where PCS input and output are stored. Light-blue boxes indicate interfaces to the users and the yellow box to the upper left indicates the IPP system. The arrows in the figure represent links between the components. Their colors code the type of link (red for input, blue for output/results, gray for configuration data, cyan for a trigger). For clarity, the lines joining to the C', D', and PanDiSC components are dotted. The paper-like symbols indicate generated data files. The two yellow boxes, "Photometric catalog" and "Photometric Classification Data," refer to the manual mode of operations (see below).

Zoom In Zoom Out Reset image size

In the normal batch mode of activities, the interfaces/processes A to F permanently run in the background and react to changes in the PSPS database (A) or the Master database in stand-alone mode. However, parallel manual sessions can be activated where the user is free to use parts or all of the pipeline, adding further photometric or spectroscopic data sets to the local database, changing setups, testing new SEDs and recipes, or defining and using new training sets. Below we first describe the automatic mode of operations and then summarize the manual options.

The starting point is the PSPS database in Hawaii, which is filled with catalog data produced by IPP. The module A periodically checks when new sets of data with all five band fluxes measured are available in PSPS. It copies tables of input data according to selectable input parameters to Garching. The module B detects the output of process A, ingests these data in the local database, and computes for each object the galactic absorption corrections according to the Schlegel et al. (1998) maps. They are applied to the photometry only when computing photometric redshifts. The bare photometry is considered when classifying the objects (in PanDiSC) or when testing the stellar templates (in PanZ). Once B has finished, the module C and C' start to prepare PanZ and PanDiSC jobs, respectively, to analyze the newly available data, according to the preset configuration files. The input catalog is split into many suitable chunks to allow the triggered submission of multiple jobs on the parallel queue of the computer cluster (see Section 5.3). Once all the jobs are finished, the modules D and D' become active. They copy the results of PanZ and PanDiSC, respectively, into the local database, except for the full redshift probability distributions for each object, which are too large to be written in the database directly. They are saved as separate compressed files. At this point, the procedure E is activated. This prepares the data tables for delivery to Hawaii. Finally, the module F signals PSPS in Hawaii, which then uploads them into the PSC MYDB part of the PSPS database. They can be accessed by the whole Pan-STARRS1 community through the Pan-STARRS1 Science Interface (the light-blue box at the lower left of Figure 1).

As a second mode of operation, the system can work in "manual" mode, where specific data sets can be (re-)analyzed independently of the automatic flow. This is useful in the testing phase, or while considering external data sets, such as the SDSS, or mock catalogs of galaxies (indicated by the yellow box "Photometric catalog" of Figure 1). The output of such manual runs is indicated by the yellow box "Photometric Classification Data" of Figure 1. Many of the tests discussed in this paper have been obtained in "manual" mode. Extensive spectroscopic data sets are available in the local database to allow an efficient cross-correlation with the Pan-STARRS1 objects.

The configuration files, as well as the different modes of operations, can be manipulated through a user-friendly Web interface (the light-blue box at the lower right of Figure 1). This feature, initially developed to ease the life of the current small number of PCS members, makes the system interesting for a possible future public release.

The PCS is implemented on the PanSTARRS cluster, a 175 node (each with 2.6 GHz, 4 CPUs, and 6 GB memory, for a total of 700 CPUs) Beowulf machine with 180 TB disk space, attached to a PB robotic storing device, mounted at the Max-Planck Rechenzentrum in Garching. The modules described in the previous section are a series of shell scripts, or html/php files, executing php code or SQL commands, or running compiled C++ code. In particular, the input/output interfaces A and F to the PSPS database make use of SOAP/http calls. The Schlegel maps are queried using the routines available from the Web. The local database is based on MySQL. A set of Python scripts allows the user to automatically generate plots and statistics similar to Figures 4 and 5. Presently, new available photometry is downloaded from the PSPS database in chunks of four million objects. They are split in blocks of 25,000, each of which is sent as a single job to the parallel queue. These numbers are subject to further optimization. The system allows parallel running of jobs operating on the same data set, but with different recipes.

The performance (in terms of processed objects per second) of the complete system is summarized in Table 3, where the case of a catalog of 1.8 million objects is presented.

The modular structure of the PCS allows implementation of further components, such as alternative photometric redshift codes and the modules to constrain stellar parameters that are under development.

We trained and tested the PanDiSC SVM using a ten-fold cross-validation approach. We divided the sample described in Table 4 with Pan-STARRS1 photometry in 10 partitions, each with 45 stars, 475 galaxies, and 55 QSOs. Note that they span the magnitude ranges shown in Figure 2. We constructed 10 subsets by randomly selecting 55 out of the 475 galaxies. We created 10 training sets by concatenating 9 of the subsets, and tested the PanDiSC SVM on each of the 10 partitions not used in the 10 training sets.

Table 5 shows the results: galaxies are classified correctly in almost 97% of cases, with very small variations from field to field. Stars (one of which having equal probability of being a star, a galaxy, or a QSO, and therefore not considered) and QSOs are recovered on average in ≈84% of cases. Successful star classification goes up to 95% for MDF5, and is as low as 75% for MDF10. Successful QSO classification is at the 87% level in MDF4 and drops to 69% for MDF9. We will investigate the statistical significance and possible cause of these variations with future larger Pan-STARRS1 catalogs.

The relatively poor performance of the stellar classification is driven by early-type stars alone (correctly classified in 80% of cases) and the Pan-STARRS1 filter set. The classification of late-type stars is better, with just 4% of the late stars being wrongly classified as galaxies.

We now look at the fraction of false positives. Only 21 stars and 44 QSOs are classified as galaxies; therefore the galaxy sample defined by PanDiSC is contaminated at the level of 1%. This is not unexpected because the extragalactic MDF number counts are dominated by galaxies. The purity of the star and QSO samples are much worse. There are 38 galaxies and 47 QSOs classified as stars, which results in a 19% contamination of the star catalog defined by PanDiSC. Without the contribution of galaxies, which could be flagged by adding morphological information (i.e., whether the objects are point-like or extended), the contamination (by QSOs) reduces to 10%. Preliminary tests (Klement 2009) show that this development is indeed very promising. There are 47 stars and 107 galaxies classified as QSOs. This means that the QSO catalog defined by PanDiSC is contaminated at the 28% level, without galaxies just 8.5%. Clearly, the situation will be different at lower galactic latitudes, where stars dominate the number counts at these magnitude limits. Overall, the results are not much worse than those reported by Elting et al. (2008). They can be improved when larger Pan-STARRS1 catalogs will be available, by optimizing the probability thresholds for making a classification decision since they depend on the relative distribution of stars, galaxies, and quasars in the training/test data sets used to make the assessment.

Figure 3 presents the Pan-STARRS1 color–color plots with the distributions of stars, galaxies, and quasars of the spectroscopic SDSS data set. Objects that are stars according to the spectroscopic SDSS classification are shown in the left diagrams, objects that are galaxies in the central diagrams, and objects that are QSOs in the right diagrams. Objects classified correctly by PanDiSC are shown in black. Objects wrongly classified as stars by PanDiSC are shown in green, wrongly classified as galaxies in red, and wrongly classified as QSOs in blue. So, a red dot on the left column is a star that PanDiSC classified as a galaxy.

Zoom In Zoom Out Reset image size

Clearly, misclassifications happen in regions of color space where the different types overlap, where only an additional filter (for example, the u band) would help discriminate between the populations. In contrast, late-type stars are seldom misclassified thanks to their red colors that divide them well from galaxies and QSOs. Note that the increased scatter in the stellar g_P1 − r_P1 colors at r_P1 − i_P1 > 1.8 is due to stars near or below the g_P1 and r_P1 magnitude limits of Table 2.

We repeated the same tests based on the ten-fold cross-validation technique using the SDSS Petrosian ugriz photometry: the results are presented in Table 6. While the presence of the u band boosts the success rate for QSOs (up to 93%) and stars (up to 92%, with early-type stars correctly classified in 90% of the cases), the absence of the y band and the lower quality of the z band marginally penalize the galaxy classification (down to 95%) and the stellar classification of late star types (classified correctly in 95% of cases). The star false positives are up to 28% (of the total true stars), due to the higher number of misclassified galaxies, but would drop to just the 6% of misclassified QSOs if information about size were added. The QSO false positives, slightly better at 27%, drop to just 3% without galaxies. The galaxy false positives stay at the 1% level.

The Pan-STARRS1 photometric data set misses the u band on the blue side of the SED, and the NIR colors redder than the y band. Therefore, one expects any photometric redshift program to perform best for red galaxies at moderate redshifts (e.g., the LRGs), and to fail especially when blue galaxies at low redshifts are considered. Similarly, late stars are expected to be better recognized than earlier ones. Finally, since at present the PhotoZ program misses SEDs optimized for QSOs, we expect poor performances in these cases.

Figure 4 shows the comparison between spectroscopic and photometric redshifts obtained for the SDSS sample of LRGs, field by field. Figure 5 shows the results for the whole sample. As usual, we define the percentage of catastrophic failures η as the fraction of objects (outliers) for which |z_phot − z_spec| > 0.15 × (1 + z_spec), the residual bias as the mean of (z_phot − z_spec)/(1 + z_spec) without the outliers, and the robust error as σ_z = 1.48 × Median|z_phot − z_spec|/(1 + z_spec) without the outliers. For LRGs, the Pan-STARRS1 photometry allows the determination of photometric redshifts accurate to 2.4% in σ_z, with bias smaller than 0.5%, no strong trend with redshift and 0.4% catastrophic outliers (when no QSO SED is allowed). No field-to-field dependences are present.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

As expected, the situation is not as satisfactory for non-LRGs. Figure 6 shows that especially at z_spec < 0.2 the residuals are biased in a systematic way and more than 1% catastrophic outliers are present. Nevertheless, the robust estimate of the scatter remains below 5%.

Zoom In Zoom Out Reset image size

If we now use for the same galaxies the Petrosian ugriz SDSS photometry, we find the following. The photometric redshifts for LRGs are similarly good (2.6%), but with a higher percentage of outliers. In contrast, the precision (3.7%) and percentage of outliers (1%) are better for blue galaxies, where the presence of the u band helps.

As described in Section 5.1, PanZ also computes the goodness of fits for a number of stellar templates. Therefore, the difference χ²_star − χ²_galaxy between the χ² of the best-fitting stellar SED χ²_star and the best-fitting galaxy SED χ²_galaxy provides a crude galaxy/star classification: if χ²_star − χ²_galaxy < 0, the stellar template is providing a better fit than the galaxy ones and we classify the object as a star. In Figure 7, left (where just for plotting convenience we give χ²_star/χ²_galaxy), we show that requiring χ²_star − χ²_galaxy < 0 (i.e., χ²_star/χ²_galaxy < 1 in the plot) allows us to correctly classify spectroscopically confirmed SDSS stars in 73% of the cases. The percentage of successful classifications grows to 89% if only late-type SDSS stars are considered. The percentage of success is 98% if spectroscopically confirmed SDSS galaxies are considered (Figure 7, right).

Zoom In Zoom Out Reset image size

Finally, we consider the class of QSOs. Figure 8, left, shows that spectroscopically confirmed SDSS QSOs are classified as QSOs (i.e., are best fit by the QSO SED) in 22% and as galaxies in 50% of the cases. As expected, the photometric redshifts are very poor (Figure 8, right). The QSO SED in the sample is selected as giving the best fit in 22% of the cases, giving the right redshift in 20% of the cases. For an additional 28% where catastrophically wrong redshifts are derived, the QSO SED gives the second best fit and a reasonable redshift. Still, if we allow only the QSO SED to be used, we get a good redshift (≈5% in σ_z) for only 61% of the objects. We are in the process of adding some more QSO SEDs to model better the redshift dependence of QSO evolution. First tests show that only modest improvements can be achieved since we are hitting the intrinsic limitations of the Pan-STARRS1 filter photometry, combined with the well-known difficulties of deriving photometric redshifts for the power-law-like, feature-weak shape of QSO SEDs (Budavari et al. 2001; Salvato et al. 2011). The addition of the u band certainly improves the results a lot. When we derive photometric redshifts using the SDSS ugriz Petrosian magnitudes, we get a best fit with the QSO SED in 51% of the cases (with a photometric redshift good to 5% in 43% of the cases), and for an additional 19% the QSO SED gives the second best solution with the correct redshift. If we allow only the QSO SED to be used, we get a good redshift (≈5% in σ_z) for 70% of the objects.

Zoom In Zoom Out Reset image size

Table 7 shows the confusion matrix for PanZ as a star/QSO/galaxy photometric classifier. PanZ performs as well as PanDiSC when classifying galaxies, but is poorer when it comes to stars and QSOs, probably due to a lack of appropriate SED templates. As a consequence, the false positive contamination is higher for stars (53%) and galaxies (8%) classes, but lower (4%) for QSOs, compared to PanDiSC. Finally, it is interesting to note that PanZ biases the classification in a different way than PanDiSC: there are 29 stars and 3 QSOs recognized as such by PanZ but not by PanDiSC.

Finally, Table 8 shows the confusion matrix for PanZ as a star/QSO/galaxy photometric classifier when the SDSS Petrosian ugriz photometry is used. The percentage of correctly classified QSOs doubles (but is still not competitive with the results of PanDiSC) to reach 44%, the star classification is slightly improved to 80%, and the success in the galaxy classification is slightly worse (85%). Therefore, the presence of the u band helps in the classification of (blue) stars and quasars, but does not compensate for the absence of the y-band data and of good z-band data for galaxies.

As discussed in Section 6.1, the final assessment of the relative performances of PanDiSC and PanZ as morphological classifiers will be made when larger Pan-STARRS1 catalogs will allow the derivation of optimal probability thresholds.