Alert Classification for the ALeRCE Broker System: The Light Curve Classifier

ALeRCE has been processing the public ZTF alert stream since 2019 May, which includes g and r photometry. The ALeRCE pipeline is described in detail by Förster et al. (2020); for clarity, we provide a brief description of the light curve construction process.

The ALeRCE pipeline processes the ZTF Avro alert files. ¹⁹ These files contain metadata and contextual information for a single event, which is defined as a flux-transient, a reoccurring flux-variable, or a moving object (Masci et al. 2019). To construct light curves, the ALeRCE pipeline uses the photometry of the difference image and reference image (detections), possible non-detections associated with the target during the previous 30 days of the event (5σ magnitude limit in the difference image based on PSF-fit photometry, called diffmaglimby ZTF), the real-bogus quality score reported by ZTF (rb, which ranges from 0 to 1, with values closer to 1 implying more reliable detections), and the morphological classification of the closest object obtained from PanSTARRS1 (Tachibana & Miller 2018). An overview of the pipeline is presented in Figure 1. In summary, the different stages of the pipeline are:

• (1)  
  Ingestion: the ZTF public stream is ingested using Kafka;
• (2)  
  S3 upload: the alert Avro packets are stored in AWS S3 for later access;
• (3)  
  Crossmatch: the position of the alert is used to query external catalogs;
• (4)  
  Stamp classifier: alerts from new objects are classified using their image cutouts (stamps);
• (5)  
  Preprocessing: the photometry associated with a given alert is corrected to take into account the use of difference image fluxes (see details below), and simple statistics associated with the aggregated light curve are computed;
• (6)  
  Light curve features: advanced light curve statistics (features) are computed when there are at least six detections in a given band;
• (7)  
  Light curve classifier: the light curve classifier described in this work is applied;
• (8)  
  Outliers: an outlier detection algorithm is applied; and
• (9)  
  ALeRCE stream: the aggregated, annotated, and classified light curves are reported in a Kafka stream.

Zoom In Zoom Out Reset image size

In step (3) we are experimenting with several catalogs, but for this work we use the AllWISE ²⁰ public Source Catalog (Wright et al. 2010; Mainzer et al. 2011), invoking a match radius of 2'', to obtain W1, W2, and W3 photometry (using magnitudes measured with profile-fitting photometry, e.g., w1mpro).

The preprocessing procedure (step 5) is described in detail in Förster et al. (2020; see their Appendix A). In particular, for the light curve classifier we use the corrected light curves (lc_corr; ${\hat{m}}_{\mathrm{sci}}$ in Förster et al. 2020) for sources whose closest source in the reference image coincides with the location of the alert (in a radius of 14). It is important to use the corrected light curves for variable sources in order to take into account changes in the sign of the difference between the reference and the science images, or possible changes of the reference image. For the rest of the sources, the correction is not possible to perform, and thus we use the light curves obtained using the difference images (lc_diff; m_diff in Förster et al. 2020), which correspond in general to transient sources. Note that these criteria do not require prior knowledge about the class of the source. Therefore, in this work we use lc_corr for sources with available corrected light curves, otherwise we use lc_diff, except for the supernova parametric model features and some optical colors, for which we use lc_diff for all the sources (for more details, see Section 3.1 and Appendix A).

The first version of the ALeRCE light curve classifier considers 15 subclasses of variable and transient objects, presented as a taxonomy tree defined by the ALeRCE collaboration in Figure 2. The taxonomy is subdivided in a hierarchical fashion according to both the physical properties of each class and the empirical variability properties of the light curves as follows (in parentheses we indicate the class name used by the classifier):

• 1.  
  Transient: SN Ia, SN Ibc, SN II, and super luminous supernova (SLSN);
• 2.  
  Stochastic: type 1 Seyfert galaxy (AGN; i.e., host-dominated active galactic nuclei), type 1 quasar (QSO; i.e., core-dominated active galactic nuclei), blazar (Blazar; i.e., beamed jet-dominated active galactic nuclei), young stellar object (YSO), and cataclysmic variable/nova (CV/Nova);
• 3.  
  Periodic: long-period variable (LPV; includes regular, semi-regular, and irregular variable stars), RR Lyrae (RRL), Cepheid (CEP), eclipsing binary (E), δ Scuti (DSCT), and other periodic variable stars (Periodic-Other; this includes classes of variable stars that are not well represented in the labeled set, e.g., sources classified as miscellaneous, rotational or RS Canum Venaticorum-type systems in CRTS).

Zoom In Zoom Out Reset image size

Figure 3 shows examples of light curves of the different classes considered by the light curve classifier, obtained using ZTF data.

Zoom In Zoom Out Reset image size

It is important to note that there are a number of less common classes which have not been separated out yet in the ALeRCE taxonomy tree because the number of crossmatched objects in these classes is too low to train a good classification model (e.g., SNe IIb, TDEs, KNe, among others). There is a catch-all "Periodic-Other" class for periodic classes excluded in the taxonomy tree, but not for transient or stochastic classes, and thus, for the moment, these missing classes are being grouped into one or more of the existing ones.

2.2.1. Labeled Set

The labeled set (i.e., the set of sources used to define the training and testing sets) for the light curve classifier was built using sources observed by ZTF (i.e., with ZTF light curves), with known labels obtained via spectroscopic and/or photometric analysis by previous works. Further description of the labeled set construction strategy can be found in Förster et al. (2020). We obtained labels from the following catalogs: the ASAS-SN catalog of variable stars (ASASSN; Jayasinghe et al. 2018, 2019a, 2019b, 2020), the Catalina Surveys Variable Star Catalogs (CRTS; Drake et al. 2014, 2017), the LINEAR catalog of periodic light curves (LINEAR; Palaversa et al. 2013), Gaia Data Release 2 (Gaia DR2; Mowlavi et al. 2018; Rimoldini et al. 2019), the Transient Name Server database (TNS), ²¹ the Roma-BZCAT Multi-Frequency Catalog of Blazars (ROMABZCAT; Massaro et al. 2015), the Million Quasars Catalog (MILLIQUAS, version 6.4c, 2019 December; Flesch 2015, 2019), the New Catalog of Type 1 AGNs (Oh2015; Oh et al. 2015), and the SIMBAD database (Wenger et al. 2000). Some additional CV labels were obtained from different catalogs (including Ritter & Kolb 2003), compiled by Abril et al. (2020; JAbril). It is worth mentioning that we only use the labels provided by these catalogs to build our data sets and not any other information, such as periods, colors, or redshifts. A catalog containing the labeled set can be downloaded at Zenodo:10.5281/zenodo.4279623.

Table 1 lists the number of sources in the labeled set belonging to each class (with their corresponding percentages according to their hierarchical group) and the catalogs from which the classifications were obtained. Only sources with ≥6 detections in g or ≥6 detections in r were included (considering data obtained until 2020 June 9). It is clear from the table that there is a high imbalance in the labeled set, with some classes representing less than 5% of their respective hierarchical group. Figure 4 shows the (ordered) number of sources per class for the labeled set and Figure 5 shows the fraction of sources in each class with photometry only in the g band, only in the r band, or in both bands.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Table 1. Labeled Set Definition

Hierarchical Class	Class	# of Sources a	Source Catalogs
	SN Ia	1272 (74.0%)	TNS
	SN Ibc	94 (5.5%)	TNS
Transient	SN II	328 (19.1%)	TNS
	SLSN	24 (1.4%)	TNS
	Total	1718
	QSO	26168 (75.4%)	MILLIQUAS (sources with class "Q")
	AGN	4667 (13.4%)	Oh2015, MILLIQUAS (sources with class "A")
Stochastic	Blazar	1267 (3.6%)	ROMABZCAT, MILLIQUAS (sources with class "B")
	YSO	1740 (5.0%)	SIMBAD
	CV/Nova	871 (2.5%)	TNS, ASASSN, JAbril
	Total	34713
	LPV	14076 (16.2%)	CRTS, ASASSN, Gaia DR2
	E	37901 (43.5%)	CRTS, ASASSN, LINEAR
	DSCT	732 (0.8%)	CRTS, ASASSN, LINEAR, Gaia DR2
Periodic	RRL	32482 (37.3%)	CRTS, ASASSN, LINEAR, Gaia DR2
	CEP	618 (0.7%)	CRTS, ASASSN
	Periodic-Other	1256 (1.4%)	CRTS, LINEAR
	Total	87065

Note.

^a Values in parentheses correspond to the fraction of sources of a given class (second column) within its corresponding hierarchical class (first column).

Download table as:  ASCIITypeset image

Most of the features used by the light curve classifier are computed using the observed magnitudes in the g and r bands (i.e., the detections). There are 56 different features computed for each band, and 12 features computed using a combination of both bands, yielding a total of 124 detection features. The definition of all these features can be found in Table 2. We split the table into three blocks. The first block contains new features defined by this work (i.e., novel features). Some of these features are further described in Section 3.1.1. The second block contains features that correspond to new variants of descriptors included in other works. Some of them are further described in Appendix A. Finally, the third block includes 22 features that come from the Feature Analysis for Time Series (FATS; Nun et al. 2015) Python package. Hereafter, features ending with "_1" are computed for the g band, and features ending with "_2" are computed for the r band, following the notation used in the ZTF Avro files.

3.1.1. Description of a Relevant Set of Detection Features

For each detection, the ZTF alert stream includes 5σ magnitude limits (diffmaglim), which are computed from the g and r difference images of the same area of the sky obtained in the previous 30 days, where the target associated with the alert was not detected (non-detections). These non-detections are very informative, since they can, for instance, inform us whether a transient has not been detected before, whether a non-variable source has begun to exhibit a variable behavior, or which range of observed magnitudes we should expect to measure when there are not significant differences between the science and template images, and an alert is not generated. The light curve classifier uses nine different features defined using all the non-detections associated with a given source, computed for both g and r bands, yielding a total of 18 non-detection features. Note that all these features are new and have not been used before for classification. Table 3 lists the non-detection features used by the light curve classifier. As before, non-detection features ending with "_1" are computed for the g band, and features ending with "_2" are computed for the r band.

A Decision Tree (Rokach and Maimon 2008) is a predictive algorithm that uses a tree structure to perform successive partitions on the data according to a certain criterion (e.g., a cut-off value in one of the descriptors or features) and produces possible decision paths, providing a final outcome for each path (the leaves of the tree). Decision trees are commonly used for classification, where each final leaf is associated with a given class. RFs (Breiman 2001) are algorithms that build multiple decision trees, where each tree is trained using a random sub-sample of elements from a given training set, selected allowing repetition (bootstrap sample of the training set), and using a random selection of features. The final classification is obtained by averaging the classifications provided by each single tree. This average score can be interpreted as the probability (P_RF) that the input element belongs to a given class. One of the main advantages of RF is that it naturally provides a ranking of features for the classification, by counting the number of times each feature is selected to split the data.

Chen et al. (2004) proposed a modified RF that can deal with the imbalanced data classification. In their model each individual tree is trained using a sub-sample of the training set that is defined by generating a bootstrap sample from the minority class and then randomly selecting the same number of cases, with replacement, from the majority classes. Imbalanced-learn implements the balanced RF classifier proposed by Chen et al. (2004). For the ALeRCE light curve classifier we use their BalancedRandomForestClassifier method, selecting the hyper-parameters (number of trees, maximum number of features per tree, and maximum depth of each tree) with a K-fold cross-validation procedure available in scikit-learn, with k = 5 folds and using the "macro-recall" as target score (see its definition in Section 5.1).

4.1.1. The Two-level Classifier Approach

Considering the hierarchical structure of the taxonomy (see Section 2.2), we decided to construct a balanced RF (BRF) classifier with two-level scheme. The first level consists of a single classifier that separates the sources into three broad classes. The second level consists of three distinct classifiers, which further resolve each class in the first level into subclasses. We then use the probabilities obtained for each independent classifier to obtain the final classification.

In more detail, the first level (top level hereafter) consists of a single classifier which classifies every source as periodic, stochastic, or transient. The second level (bottom level hereafter) consists of three distinct classifiers: transient, stochastic, and periodic. The classes considered by each of these three classifiers are the ones shown in Table 1 and Figure 2. Each classifier in the bottom level is trained using a training subset having only those classes included in the primary top class (for instance, the transient classifier only includes sources classified as SN Ia, SN Ibc, SN II, and SLSN). It is important to note that these four classifiers are independent and process the same input features set described in Section 3. The final classification is constructed by multiplying the probabilities obtained for each class of the top level [P_top(transient), P_top(stochastic), and P_top(periodic)] with the individual probabilities obtained by their correspondent classifier in the bottom level. Namely, the probabilities of the transient classifier (P_T ) are multiplied by P_top(transient), the probabilities of the stochastic classifier (P_S ) are multiplied by P_top(stochastic), and the probabilities of the periodic classifier (P_S ) are multiplied by P_top(periodic). We denote the product of these probabilities as P. For instance, the probability of a given source being an RRL corresponds to the product of its probability of being periodic (according to the top level) and its probability of being an RRL (according to the periodic classifier):

$$\begin{eqnarray}&&P(\mathrm{RRL})={P}_{\mathrm{top}}(\mathrm{periodic})\times {P}_{P}(\mathrm{RRL}),\end{eqnarray} \tag{ 4 }$$

while the probability of being a Blazar is computed as:

$$\begin{eqnarray}&&P(\mathrm{Blazar})={P}_{\mathrm{top}}(\mathrm{stochastic})\times {P}_{S}(\mathrm{Blazar}).\end{eqnarray} \tag{ 5 }$$

Following this, the sum of the probabilities of the 15 classes for a given source adds up to one. Finally, the class of a given object is determined by selecting the class with the maximum P. Hereafter, we refer to the results presented for the bottom level of the classifier as the final predictions.

The best cross-validation performance was obtained with the following hyper-parameter setting: 500 trees in each classifier, maximum depth trees (the nodes are expanded until all leaves are pure), and a maximum number of features equal to the square root of the total number of features, except for the stochastic classifier, where we used 20% of the features. In Section 5 we present the results obtained when applying the BRF classifier to the ZTF data.

In addition to RF, we also tested two other supervised classification algorithms: gradient boosting and multilayer perceptron. These tests were done as a complementary analysis with the purpose of guiding future efforts in improving the light curve classifier.

None of these methods has a Python implementation particularly designed to handle imbalanced data sets; however, using imbalanced-learn we can generate balanced training sets. We present the results obtained using both classifiers in Section 5.

4.2.1. Gradient Boosting

4.2.2. Multilayer Perceptron

In order to test the performance of our BRF classifier we generated 20 different training and testing sets using the ShuffleSplit iterator provided by scikit-learn, which uses random permutations to split the data, using each time 80% of the labeled set as training set and 20% as testing set, preserving the percentage of samples for each class in the original labeled set. Then, we trained 20 different BRF models using each training set, and tested their performance using the corresponding testing sets. We emphasize that the testing sets are never used in training their respective models.

Table 4 lists three different scores: precision, recall, and F1-score. For an individual class these scores are defined as:

$$\begin{eqnarray}&&{\mathrm{Precision}}_{i}=\displaystyle \frac{{\mathrm{TP}}_{i}}{{\mathrm{TP}}_{i}+{\mathrm{FP}}_{i}},\end{eqnarray} \tag{ 6 }$$

$$\begin{eqnarray}&&{\mathrm{Recall}}_{i}=\displaystyle \frac{{\mathrm{TP}}_{i}}{{\mathrm{TP}}_{i}+{\mathrm{FN}}_{i}},\end{eqnarray} \tag{ 7 }$$

$$\begin{eqnarray}&&{\rm{F}}1 \mbox{-} {\mathrm{score}}_{i}=2\times \displaystyle \frac{{\mathrm{Precision}}_{i}\times {\mathrm{Recall}}_{i}}{{\mathrm{Precision}}_{i}+{\mathrm{Recall}}_{i}},\end{eqnarray} \tag{ 8 }$$

where n_cl is the total number of classes, TP_i is the number of true positives, FP_i is the number of false positives, and FN_i is the number of false negatives, for a given class i. Despite the high imbalance present in the labeled set, all classes are equally important, and thus we compute macro-averaged scores:

$$\begin{eqnarray}&&{\mathrm{Precision}}_{\mathrm{macro}}=\displaystyle \frac{1}{{n}_{\mathrm{cl}}}\displaystyle \sum _{i=1}^{{n}_{\mathrm{cl}}}{\mathrm{Precision}}_{i},\end{eqnarray} \tag{ 9 }$$

$$\begin{eqnarray}&&{\mathrm{Recall}}_{\mathrm{macro}}=\displaystyle \frac{1}{{n}_{\mathrm{cl}}}\displaystyle \sum _{i=1}^{{n}_{\mathrm{cl}}}{\mathrm{Recall}}_{i},\end{eqnarray} \tag{ 10 }$$

$$\begin{eqnarray}&&{\rm{F}}1 \mbox{-} {\mathrm{score}}_{\mathrm{macro}}=\displaystyle \frac{1}{{n}_{\mathrm{cl}}}\displaystyle \sum _{i=1}^{{n}_{\mathrm{cl}}}{\rm{F}}1 \mbox{-} {\mathrm{score}}_{i}.\end{eqnarray} \tag{ 11 }$$

For the particular case of the BRF classifier, Table 4 reports the mean and the standard deviation of the macro-averaged scores obtained by the 20 models when applying them to their respective testing sets.

In addition, Figures 6 and 7 show the confusion matrices obtained for the top and bottom levels, respectively. To generate these confusion matrices we used the results obtained when applying each of the 20 BRF models to their corresponding testing sets, providing for each level the median and 5th and 95th percentiles of the confusion matrices obtained by each of the 20 models.

Zoom In Zoom Out Reset image size

The confusion matrix of the top level shows that the classifier can recover more than 97% of the true labels, and that the contamination between classes is below 3%. The scores obtained reflect the good performance of the top level classifier.

For the case of the bottom level we obtained an F1-score of 0.59, implying significant confusion between classes. From Figure 7 we can see that the fraction of true positives in the confusion matrix of the bottom level has values between 50% and 100%, with mean, median, and standard deviation of 78%, 76%, and 14%, respectively. In addition, from the figure it can be observed that the confusion is most often observed among classes with similar characteristics, such as among the SN classes (particularly among SN Ia versus SN Ibc and SN II versus SLSN); among the blazar, AGN, and QSO classes; and among various periodic classes. The highest standard deviation of the predictions is observed for the case of SLSN. This is a result of the low number of SLSN in the labeled set.

Zoom In Zoom Out Reset image size

To complement our analysis, in Appendix B we provide the results obtained by a one-level multi-class RF model. From its results we can conclude that the light curve classifier improves considerably when a two-level strategy is followed.

5.1.1. Comparison with the GBoost and MLP Classifiers

In this work we tested two other classifiers: GBoost and MLP. For these models we present the results obtained by using 80% of the labeled set as a training set and the remaining 20% as a testing set, preserving the percentage of samples for each class in the original labeled set.

For the case of the GBoost classifier, the macro-averaged precision, recall, and F1-score of the top level have a value of 0.99. For the bottom level, the macro-averaged precision, recall, and F1-score are, respectively, 0.72, 0.72, and 0.71. On the other hand, for the MLP classifier the macro-averaged precision, recall, and F1-score of the top level are 0.94, 0.99, and 0.96, respectively. For the bottom level the macro-averaged precision, recall, and F1-score are, respectively, 0.54, 0.69, and 0.58. The confusion matrices obtained for the bottom level of the GBoost and MLP classifiers in the testing set are presented on the left and right sides of Figure 8, respectively.

Zoom In Zoom Out Reset image size

The precision and F1-score obtained by GBoost are in general better than the ones obtained by the BRF classifier, with the exception of the recall score of the bottom level. However, the fraction of true positives in the confusion matrix of the bottom level range between 5% and 100%, with a mean, median, and standard deviation of 72%, 83%, and 29%, respectively, which explain the lower recall obtained by GBoost, compared to the BRF classifier. In addition, the classes with the largest fraction of true positives in the confusion matrix of GBoost correspond to the most populated classes in the labeled set, like QSOs, which represent 75.4% of the stochastic sources; SN Ia, which represent 74.0% of the transients; or LPV, E, and RRL, which represent 16.2%, 43.5%, and 37.3% of the periodic sources, respectively. This is not observed in the results obtained by BRF, where there is no evidence of correlation between the representativity of a given class and its fraction of true positives in the confusion matrix shown in Figure 7.

The results obtained using GBoost are promising. We obtained good scores although the current versions of GBoost available in the literature have not been designed to deal with high imbalance for the case of multi-class classification. Therefore, further efforts should be done to implement GBoost in future versions of the light curve classifier. In particular, new implementations of GBoost for multi-class classification that follow similar approaches to the ones proposed by Chen et al. (2004) or Wang et al. (2019) should be tested, as should combinations of GBoost with data augmentation techniques (i.e., generating synthetic light curves of less populated classes using physical and/or statistical models).

For the case of the MLP classifier the scores obtained are in general lower compared to BRF and GBoost. Its confusion matrix for the bottom level (see Figure 8) presents the same issues already discussed for the case of GBoost. The fraction of true positives in the confusion matrix ranges between 11% and 98%, with a mean, median, and standard deviation of 69%, 74%, and 22%, respectively. Therefore, we conclude that more work should be done in order to obtain better results with MLP.

From these tests we can conclude that the BRF is the model that currently achieves results that are less biased toward the most populated classes in the labeled set, i.e., it is able to predict all subclasses, and the fraction of true positives does not correlate with how representative a given class in the labeled set is, compared to GBoost and MLP. Thus, we decided to use BRF as the final model for the first version of the ALeRCE light curve classifier. Future work will further exploit the potential of the GBoost and MLP classifiers. For the rest of this paper, presented results correspond to the BRF classifier.

5.1.2. Results for the BRF Classifier Excluding AllWISE Data

We tested a version of the BRF classifier that excludes features computed using AllWISE data. The macro-averaged precision, recall, and F1-score of the top level are 0.93, 0.97, and 0.95, respectively. For the bottom level the macro-averaged precision, recall, and F1-score are, respectively, 0.53, 0.72, and 0.55. These scores are slightly smaller than the ones obtained using the original version of the BRF classifier. As can be observed in the confusion matrix shown in Figure 9, the stochastic classes are the most affected by the lack of AllWISE data, particularly YSOs and Blazars, whose fraction of true positives decreased 10% and 16%, respectively. This happens because of the similarities observed between the light curves of these and other classes, which are not easily separated using variability features alone. However, the results obtained by this version of the classifier are still good enough to be used in the case that AllWISE data are not available, as occurs, for instance, for faint objects.

Zoom In Zoom Out Reset image size

As can be seen in Figure 5 for some classes, like YSOs, Cepheids, and LPVs, a non-negligible fraction of sources in the labeled set have photometry available only in one band. It is therefore important to know how well the classifier behaves when a single band is available for a given source.

To evaluate this, we created 20 new testing sets defined considering only those sources with ≥6 detections in both g and r bands, from the 20 testing sets previously generated using the ShuffleSplit iterator (see Section 5.1). We then classified each new testing set with its respective model, considering: (a) the features available for the g and r bands, (b) the features available only for the g band (i.e., we hide the features measured using the r band), and (c) the features available only for the r band (i.e., we hide the features measured using the g band). Figures 10 and 11 illustrate the results of this analysis. Figure 10 reports the recall values as a function of the average magnitudes per class (i.e., the recall values are computed by comparing the true and predicted labels for the objects within each magnitude bin), with the SN subtypes grouped in the unique class SN. Figure 11 reports the recall values as a function of the number of detections (i.e., the recall values are computed for each bin of the ZTF number of detections). From both figures we can infer that in general the best results are obtained when photometry from both g and r are available, with the exceptions of QSO, CEP, and periodic-other classes.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

From Figure 10 we can also conclude that the reliability of the classification versus the average magnitude is different for each class. These distributions in general follow the magnitude distribution in the labeled set of each class considered in this model (with the exception of RRL). For instance, for the labeled set, the CEP class corresponds to one of the brightest classes, having in general r < 16, while the LPV class covers a broader range of magnitudes. On the other hand, from Figure 11, we can infer that in general the classification improves when more detections are available in both bands, with the exceptions of the QSO and periodic-other classes.

The results obtained for periodic-other are not surprising since this class includes all the periodic classes not considered in the classifier (including several different types of pulsating stars as well as the rotational variables). The results observed for the CEP class are probably due to the large fraction of Cepheids in the labeled set with photometry only in the g band, which is produced by the saturation limit of the ZTF survey (12.5–13.2 mag), and the fact that Cepheids tend to be very bright, particularly in the r band, thus for bright Cepheids, the r band light curve is not available. The results obtained for AGNs and QSOs are likely related with incorrect labels, which we discuss further in Appendix C.1.

Peculiar results of the recall values of some classes are reported in Figures 10 and 11 when only one band is available. For SNe, there is a decrease in the recall curve in the g band, presumably due to the fact that, in general, the g-band light curves of SNe tend to decay faster than the r-band light curves, producing shorter (and thus fewer detected) g-band light curves. This trend can be seen in the SN shown in Figure 3, as well as more generally in the light curve statistics for SNe. The average number of detections of SNe light curves is 12 and 16 in the g and r bands, respectively, and the total time length of SNe light curves corresponds to 53 and 64 days in the g and r bands, respectively. The low recall obtained for bright RRL when only the g band is available may be produced by differences in the variability features measured for different RRL subtypes. This issue is further discussed in Appendix C.2. The zero value recall curves obtained for the CV/nova class when only the g band is available is produced by the similarities in the AllWISE+ZTF colors of CV/novae and some periodic classes. This is discussed in Appendix C.3. Despite this, most of the sources in the labeled set have photometry in both g and r bands (see Figure 5), and thus this low performance obtained for some classes when only one band is available does not highly affect the presented results.

In order to use the BRF classifier to classify the ZTF alert stream we need to train a single BRF model. We call this model "the deployed BRF classifier." This "deployed" model corresponds to a totally independent classifier (i.e., it is different from the previously trained 20 models), and uses the same 152 features presented in Section 3. As in the previous sections, we trained the deployed BRF model using 80% of the labeled set as the training set and the remaining 20% as the testing set. The macro-averaged precision, recall, and F1-score obtained for the top level are 0.96, 0.98, and 0.97, respectively, while for the bottom level they are 0.58, 0.79, and 0.61. The reason for these large differences in the macro-averaged metrics between the top and bottom levels can be understood from Figure 12, which shows the confusion matrix of the bottom level of the deployed BRF model. This confusion matrix is in agreement with the results presented in Figure 7, and shows that some classes, such as the the SN Ibc, CV/nova, YSO, CEP, and periodic-other classes, require further work in order to improve the results obtained by the classifier; this may involve data augmentation procedures, and better period estimations, as discussed in the following sections.

Zoom In Zoom Out Reset image size

5.3.1. Feature Ranking of the Deployed BRF Classifier

5.3.2. Results for the Unlabeled ZTF Set

We now turn to discuss the results obtained when applying the final BRF classifier to all the sources in ZTF with ≥6 detections in g or ≥6 detections in r. Considering the data obtained by ZTF until 2020 June 9, there are 868,371 sources that satisfy this condition, hereafter defined as the "unlabeled ZTF set." We define the class of a given object in the unlabeled ZTF set by selecting the class with the maximum probability obtained by the deployed BRF classifier. However, users of this classifier can use the obtained probabilities to make their own probability cuts and select samples for their science. The features, classifications, and probabilities obtained for the unlabeled ZTF set with data until 2020 June 9 can be downloaded at Zenodo:10.5281/zenodo.4279623.

It is important to note that the classifications obtained by the light curve classifier are updated every day, as new alerts are received. Whenever a new alert is received for a given source, the ALeRCE pipeline recomputes its variability features and provides an updated classification. These updated classifications can be found at the ALeRCE Explorer website ²² using the "light curve classifier" tab and specifying the desired class. Considering the results shown in Figure 11, we expect that the quality of the classification for a given source will improve as more detections are added to the light curve. In addition, with new alerts, more objects will satisfy the condition of having ≥6 detections in g or ≥6 detections in r, and thus, the size of the unlabeled ZTF set increases every day. Moreover, with new detections the size of the labeled set will increase, allowing the training of new BRF models. Updates regarding any possible modification to the light curve classifier (e.g., labeled set and models) will be published on the ALeRCE Science website. ²³

Figure 13 shows the number of candidates per class obtained for the 868,371 sources with enough alerts until 2020 June 9. Compared to Figure 4, it can be noticed that there is no correlation between the number of sources per class for the unlabeled set and the number of sources per class for the labeled set. The periodic-other, E, and LPV classes have the highest number of candidates, while the SN classes have the lowest. The distribution of candidates per class is consistent with the astrophysical number densities (i.e., we are likely not misclassifying large numbers of sources). For instance, blazars are relativistically beamed (and thus seen to have farther distances), but only over very small viewing angles, and hence are expected to be less common than QSOs and AGNs. In the case of SNe, not factoring in the amount of time a particular SN is above the magnitude limits of the search (the "control time"), we find ratios of SNe II/SNe Ia and SNe Ibc/SN Ia of 0.21 and 0.41, respectively. Computing these ratios using the number of such classes reported from ASAS-SN discoveries in Holoien et al. (2019) yields 0.36 and 0.09, respectively. The significant differences between the SNe Ibc/SN Ia ratios implies that we are strongly overestimating the numbers of SN Ibc; given the similarities between SN Ibc and SN Ia light curves, we are likely classifying a non-negligible fraction of SN Ia as SN Ibc. This highlights the importance of including distance estimations to improve the classification of transients. We are currently working to include distance-based features in future versions of the classifier.

Zoom In Zoom Out Reset image size

To investigate the quality of the predictions, we plotted the probability distributions of the top and bottom levels on the left and right sides of Figure 14. The red lines denote the position of the median probability for each class and the green lines denote the 5th and 95th percentiles. It is clear from the figure that the distribution of probabilities for the top level are higher compared to the bottom level. For the top level, the classes with the lowest probabilities are CV/nova and YSO. For the bottom level, the classes with the highest probabilities are LPV, QSO, and AGN, and the lowest probabilities are for the different classes of SNe, YSO, CV/nova, and some periodic variables.

Zoom In Zoom Out Reset image size

The low probabilities obtained for some classes are related to the confusion between classes observed in Figure 7. For instance, in Figure 7 we can see that the SN classes present a high confusion among them. On the other hand, the SN Ia, SN Ibc, SN II, and SLSN median probabilities of sources classified as SN II are 0.16, 0.19, 0.28, and 0.19, respectively. The high confusion among the SN classes may be due to the low number of sources in the labeled set, but also to the intrinsic similarities among these classes. For example, the physical mechanism responsible for the main peak of the light curve of SNe Ia and SNe Ib/c is the same, the diffusion of energy deposited by radioactive ⁵⁶Ni (Arnett 2008). Indeed, Villar et al. (2019) report that 15% of their Type Ibc SNe are classified as Ia. This might be improved by performing data augmentation using, for example, Gaussian process modeling (e.g., Boone 2019). On the other hand, the low probabilities observed for CV/novae and YSOs can be produced by the similarities between their colors and the colors of some periodic sources and the fact that some CV/novae and YSOs present very rapid variability compared to the ZTF cadence, which produces light curves with low autocorrelation, and thus low values of the IAR_phi parameter, which is normally observed for periodic sources (excluding LPVs). These similarities can be seen in Figure 15, where we show the distributions of IAR_phi_1 and g−W3 for YSOs and CV/novae (grouped), the rest of the stochastic classes, and periodic sources from the labeled set.

Zoom In Zoom Out Reset image size

Figure 16 shows the normalized r-band magnitude distribution of the different classes considering sources present in the labeled set and candidates from the unlabeled ZTF set. In general, the distributions of magnitudes of the candidates are similar to or fainter than found among sources from the labeled set. For instance, the SNe classes have candidates that are ∼0.5 mag fainter than the labeled set, and the YSO, CV/nova and E classes have candidates that are ∼1 mag fainter than the labeled set. These results show that the classifier is able to detect faint and bright candidates regardless of the luminosity biases present in the labeled set, which can be dominated by the brightest tail of the true magnitude distribution of each class. This will be particularly relevant for surveys like LSST, since, in general, the available training set will be ∼2–3 mag brighter than the limiting magnitudes of the single images, thus we would expect that currently existent bright samples will allow us to detect fainter candidates.

Zoom In Zoom Out Reset image size

A simple way to test the performance of the BRF classifier is to verify whether the results obtained when the model is applied to the unlabeled ZTF set are in agreement with what is astrophysically expected from previous works. For instance, younger Galactic targets like YSOs, classical Cepheids, and LPVs should reside near the Galactic Plane (e.g., Catelan & Smith 2015; Mowlavi et al. 2018), while extragalactic sources like AGNs, QSOs, blazars, and SNe should have roughly isotropic distributions, perhaps with fewer sources near the Galactic Plane due to attenuation/reddening from gas and dust (e.g., Calzetti et al. 2000; Padovani et al. 2017). On the left side of Figure 17 the sky distribution, in Galactic coordinates, of LPV, CEP, and YSO candidates is shown. It is clear from the figure that most of them are located in the Galactic Plane and that sources located outside the plane have a low BRF probability. This is consistent with the results obtained by previous works (e.g., Mowlavi et al. 2018; Rimoldini et al. 2019). The right panel of Figure 17 shows the Galactic latitude versus the g−r color obtained using the mean magnitude of the light curves in each band, for extragalactic candidates (QSO, AGN, blazar, SN Ia, SN Ibc, SN II, and SLSN). From the figure we can see that the fraction of extragalactic candidates observed around the Galactic Plane is low and that most of the candidates located in the plane have low probabilities. Moreover, the g−r colors of the extragalactic candidates are consistent with what is expected for these classes, with clear evidence of reddening for the candidates located around the Galactic Plane. The sky distribution, in Galactic coordinates, of the extragalactic candidates can be found Appendix D.

Zoom In Zoom Out Reset image size

In this paper we presented the first version of the ALeRCE light curve classifier. This classifier uses a total of 152 features, including variability features computed from ZTF light curves with ≥6 epochs in g or r bands, and colors computed using ZTF and AllWISE photometry (see Section 3), to classify each source into 15 subclasses, including periodic, transient, and stochastic variable sources (see Section 2.2). The light curve classifier uses a balanced RF classifier (see Section 4.1), constructed with a two-level scheme. The first level (top level) classifies each source as periodic, stochastic, or transient. The second level (bottom level) consists of three classifiers that further resolve each hierarchical class into subclasses.

We trained and tested the BRF classifier using a labeled set obtained by cross-matching the ZTF database with different catalogs of transients, stochastic, and periodic sources (see Section 2.2.1). For the top level we obtained macro-averaged precision, recall, and F1-score values of 0.96, 0.99, and 0.97, respectively, while for the bottom level we obtained macro-averaged precision, recall, and F1-score values of 0.57, 0.76, and 0.59, respectively.

We used the BRF classifier to classify 868,371 sources from ZTF (unlabeled ZTF set), obtaining results that are in agreement with what we expect astrophysically. For instance, most of the high probability extragalactic candidates are located outside the Galactic Plane and most of the high probability YSO, LPV, and CEP candidates are located in the Galactic Plane.

The condition of ≥6 detections in g or r normally equates to a timespan between 3 and 30 days since the first detection. Whenever a new detection is received for an object, the ALeRCE pipeline processes it and provides an updated classification in ∼1 s. The light curve classifier provides updated classifications for objects with new ZTF alerts every day. These updated classifications can be found on the ALeRCE Explorer website, ²⁴ selecting the "light curve classifier" option, and specifying the desired class. Catalogs containing the labeled set, and the features and RF probabilities obtained by the top and bottom levels for the unlabeled ZTF set (up to 2020 June 9) can be downloaded at Zenodo:10.5281/zenodo.4279623. In addition, more examples and instructions on how to use the ALeRCE database and classifications can be found on the ALeRCE Science website ²⁵ and in Förster et al. (2020), and a detailed description of the ALeRCE database can be found in the database schema. ²⁶ Finally, the code used to train the deployed BRF classifier can be found on the "light_curve_classifier_SanchezSaez_2020" GitHub repository, ²⁷ and the implementation of this classifier in the ALeRCE pipeline can be found on the "lc_classifier" GitHub repository, ²⁸ which version 1.0.1 is archived in Zenodo (Sánchez-Sáez et al. 2020).

One of the main challenges found during the development of the ALeRCE light curve classifier was the high imbalance present in the labeled set. For instance, the transient sources represent 1.4% of the labeled set, while the periodic sources represent 70.5%. Each hierarchical class also suffers from high imbalance among its subclasses; for example, in the case of the transient class, SN Ia comprise 74.0% of the sample, while SLSN correspond to only 1.4%. We addressed this problem by using the balanced RF implementation of the imbalanced-learn Python package, which follows the procedure proposed by Chen et al. (2004). This method uses a downsampling majority class technique to train each tree with a balanced sub-sample. We also tested two other algorithms, GBoost and MLP, but concluded that more work is needed if we want to obtain better results from those.

Another challenge was to find features useful to separate the different classes. Previous works have normally used features similar to those available in the FATS Python package (e.g., Kim et al. 2014; Martínez-Palomera et al. 2018), however, our first tests demonstrated that these features were not informative enough to separate the 15 classes considered by the light curve classifier, in particular, the stochastic and transient classes. Thus, novel features were designed and implemented for this work, like the IAR_phi parameter, the MHPS features, and the non-detection features. In addition, during the development of the light curve classifier we realized that some stochastic classes are hard to separate using just variability features. In particular, the separation of YSOs from the other stochastic classes improved significantly once we included AllWISE colors in the set of features.

Furthermore, the computation of reliable periods was quite challenging, particularly considering that the ALeRCE pipeline requires fast computation of features. Huijse et al. (2018) demonstrated that very good results can be achieved by quadratic mutual information (QMI) estimators, however, these techniques are computationally expensive [O(n²)]. We solve this issue by using the MHAOV periodogram, which provides less reliable periods, but is much faster to compute [O(n)]. Periods become increasingly unreliable as the number of data points decreases, but the classification of periodic variables can still be accurate, as other features can compensate for the decreasing quality of the periodogram (e.g., features related with the amplitude and the timescale of the variability). ALeRCE is currently working to implement methods for period estimation that are both accurate and fast. Computing the periodogram is expensive for sources with a large number of detections. We are currently exploring so-called "online" periodograms, which are updated as new samples arrive, at a fraction of the computational cost of recomputing the period each time from scratch (Zorich et al. 2020), as well as other techniques that might work better with eclipsing binary light curves (e.g., Kovács et al. 2002; Mighell & Plavchan 2013).

Moreover, the classification of the different SN classes was particularly challenging. First, the number of SNe in the labeled set was very small compared to other classes, and second, the light curves of SN classes can present similarities, which makes their separation difficult, as discussed in Section 5.3.2. We solved this issue by using the BalancedRandomForestClassifier method from imbalanced-learn, and by including the SPM features, whose definition is a modification of the work of Villar et al. (2019). In the future we plan to test other techniques to improve the separation of SN classes. Previous works have performed Gaussian process regression to model SNe light curves and generate new light curves with different cadences therefrom (e.g., Boone 2019). Moreover, better results can be obtained if we use information regarding the SN host galaxy (e.g., Foley & Mandel 2013; Baldeschi et al. 2020).

In the future we also plan to perform data augmentation to improve the classification of variable objects. For the case of variable stars, light curves can be modeled with Gaussian process or with a combination of harmonics, and then basic transformations can be applied to these models to obtain light curves with different periods and amplitudes (e.g., Elorrieta et al. 2016; Castro et al. 2018; Martínez-Palomera et al. 2018; Aguirre et al. 2019; Hosenie et al. 2020). To the best of our knowledge for the case of AGNs, QSOs, and blazars, no previous attempts to perform data augmentation have been made. A promising option is to use synthetic light curve generators that consider the physical processes behind the variability (e.g., Sartori et al. 2019).

Most of the features used by this classifier can be implemented and used to classify light curves from other data sets. In particular, for the case of LSST, the non-detection features can be adapted to work with the forced photometry that will be provided for each alert (DIAForcedSources in the Data Products Definition Document; Jurić et al. 2019). LSST will also benefit from the multiband ugrizy light curves. As we demonstrated in this work, in general the light curve classification improves when both ZTF g and r data are available. For the case of LSST this would be the same, and probably we should even be able to further resolve some of the subclasses presented in this work. For instance, using the zy light curves, we should be able to separate local type 1 and type 2 active galactic nuclei since for low-redshift sources, we can detect variability from the dusty torus at these wavelengths (see Sánchez et al. 2017 and references therein) or identify high-redshift QSOs, whose emission is expected to be absorbed in the bluer bands. We encourage researches interested in classifying stochastic and transient sources in particular to use the novel (or modified) features presented in this work, like the IAR_phi parameter, the MHPS features, the SPM features, and the non-detection features.

It is worth noting that the ALeRCE light curve classifier is being constantly improved and this work describes its Version 1.0. Future versions of this classifier may include new classes of variable and transient objects, as well as subclasses of sources already present in the taxonomy (e.g., RRL types ab and c; classical and type II Cepheids; contact, detached, and semi-detached eclipsing binaries; among others). We are also working to find new features and techniques that can improve the performance of the classifier. Future work will report any changes included in the classifier model, like the inclusion of data augmentation or the use of other classification strategies (e.g., semi-supervised training). We recommend that the users of this classifier check the ALeRCE Science website ²⁹ to get updates related to the different classifiers and the data processing. We are exploring different classification algorithms that are not based on manually designed features, but on automatically derived, recurrent, implicitly extracted features, via deep learning (e.g., Naul et al. 2018; Muthukrishna et al. 2019; Becker et al. 2020). However, up to this point we have found that the former produce better results when applied to real data. Most likely, a combination of simulated and real data will be required to train reliable deep learning classification models in the future, as found by Carrasco-Davis et al. (2019).

In this work we present the first attempt to separate different types of active galactic nuclei according to their variability properties. As we mentioned in Section 2.2, we separate active galactic nuclei in the following way:

• 1.  
  AGN: type 1 Seyfert galaxies (i.e., active galactic nuclei whose emission is dominated by the host galaxy), selected from MILLIQUAS (broad type "A"), and from Oh et al. (2015).
• 2.  
  QSO: type 1 core-dominated active galactic nuclei (i.e., active galactic nuclei whose emission is dominated by their active nuclei), selected from MILLIQUAS (broad type "Q").
• 3.  
  Blazar: BL Lac objects and flat spectrum radio quasars (FSRQ), selected from ROMABZCAT and MILLIQUAS.

In Figure 11 we showed the recall curves for each class as a function of the number of detections and we could observe that these curves decrease for QSOs and AGNs when more detections are available, particularly in the r band. These results are puzzling, considering that we would normally expect to improve the classification of a given class as more detections are included in the light curves. In order to better understand the origin of these results, we show in Figure 19 the recall curves as a function of average magnitude and number of detections for AGNs, QSOs, and blazars grouped as a single class. In this case, the recall curves are around 0.8 and 1.0 for every bin of magnitude (specially for g > 16 or r > 16) and number of detections. From this we can infer that the light curve classifier has a very good performance selecting active galactic nuclei as a single class, but some issues still remain regarding the separation of AGNs, QSOs, and blazars. There are two main explanations for these results: (a) the method cannot properly separate QSOs from AGNs and blazars, or (b) there are sources in the labeled set with incorrect labels.

Zoom In Zoom Out Reset image size

A possible way to explore how well the light curve classifier can discriminate among AGNs, QSOs, and blazars, is to check whether the features available in the light curve classifier can separate these three populations. Figure 20 shows six different features used by the classifier, (g−r)_mean_corr, Meanvariance_1, ExcessVar_1, sgscore1, Meanvariance_2, and ExcessVar_2, for QSOs, AGNs, and blazars from the labeled set (most of these features are in the top 30-ranked features shown in Table 5). From the figure we can see that these three classes have different color distributions, different morphologies, and also different variability properties. AGNs and blazars tend to be redder than QSOs (see (g−r)_mean_corr), blazars and AGNs tend to have larger amplitudes (see Meanvariance_1 and ExcessVar_1), and AGNs tend to have more extended morphologies compared to QSOs and blazars. These are just some examples of features that can be used to separate the three classes mentioned above. After a visual inspection of the feature distribution of AGNs, QSOs, and blazars, we found that more than 30 features can be used to separate them, including, for instance, PercentileAmplitude, Q31, GP_DRW_sigma, GP_DRW_tau, MHPS_low, MHPS_high, SF_ML_amplitude, among others. From this we can infer that the light curve classifier should be able to separate these three populations.

Zoom In Zoom Out Reset image size

In addition, Figure 20 highlights that the g-band features seem to separate the AGN and QSO classes better compared to the same features in the r band. This behavior is also seen in other features, like PercentileAmplitude, GP_DRW_sigma, Std, among others related to the amplitude of the variability. These differences might be produced by the combined effect of having a higher contamination from the host in the r band and intrinsically lower amplitude of the variability in the r band, due to the well-known anti-correlation between the amplitude of the variability and the wavelength of emission (see Sánchez et al. 2017 and references therein). From this, we can understand the differences observed in the g and r band recall curves of AGNs and QSOs shown in Figure 11, which should be produced by these differences in the features' distributions.

On the other hand, one key source of confusion between AGNs and QSOs is the definition of this division criteria in the labeled set. MILLIQUAS uses a luminosity-based division to separate AGNs and QSOs (see Section 5 of Flesch 2015), while the variability amplitude features are also affected by the ratio between core and host luminosities. Our variability-based criteria appear to separate host-dominated from core-dominated sources, in rough correspondence with low and high core luminosities, respectively. In addition, it is hard to make strict cuts to separate the AGN, QSO, and blazar classes, considering their observational properties (e.g., luminosity, redshift, orientation). In particular, the dividing line between AGNs and QSOs has never been defined properly (e.g., Netzer 2013; Padovani et al. 2017), and thus different catalogs will provide different classifications for the same object. Note that since the redshift of the sources in the unlabeled set is generally unknown, luminosity cannot be used as a feature by the classifier. Our aim is to separate host-dominated from core-dominated sources to favor the identification of different populations of active galactic nuclei by the light curve classifier, and use the AGN and QSO classifications as representative of these classes in the extremes of the luminosity distribution, but expect that there will be strong mixing of these classes close to the dividing luminosity used in the labeled set.

In order to see whether there are sources in the labeled set with incorrect QSO or AGN labels, we crossmatched our labeled set with the SIMBAD database. There are 26,168 QSOs in the labeled set (all obtained from MILLIQUAS), and 1590 of them are classified as Seyfert or AGN by SIMBAD (6%), with 1580 having a reported redshift <1. On the other hand, there are 4667 AGNs in the labeled set, and 830 (17%) of them are classified as QSO in SIMBAD. Therefore, there are 2420 sources in the labeled set with inconsistent classification in MILLIQUAS and SIMBAD. The light curve classifier classifies 920 of these sources as QSOs and 1319 as AGNs.

To understand better this discrepancy in the classification of some sources, we crossmatched our labeled set with the catalog of spectral properties of quasars from SDSS DR14 provided by Rakshit et al. (2020), as well as with the catalog of Oh et al. (2015) in order to obtain bolometric luminosities (L_bol), BH masses (BH_mass), Eddington ratios (L/L_Edd), and redshifts, for our sample of QSOs and AGNs. We excluded blazars from this analysis since they are not properly identified and characterized in these catalogs.

We show in Figure 21 the distribution of these spectral properties for AGNs and QSOs from the labeled set and for misclassified sources (i.e., AGNs from the labeled set classified as QSO, and vice versa). It is clear from the figure that AGNs and QSOs from the labeled set have spectral property distributions that are not strictly separated. From this we can conclude that the luminosity-based division performed by MILLIQUAS is not physically so meaningful. One issue, for instance, is to what extent the host contributes to the total magnitude used to classify the source as AGN or QSO. In addition, Figure 21 shows that the misclassified sources have similar distributions of redshift and luminosity. Moreover, these distributions lie exactly in the range of values where the properties of AGNs and QSOs from the labeled set overlap (redshift ∼0.5 and L_bol ∼ 10^45.5 [erg s⁻¹]).

Zoom In Zoom Out Reset image size

From these results, we can infer that the decrease in the recall as a function of the number of detections observed for QSOs and AGNs is produced by the discrepancy in the QSO/AGN classification obtained from different catalogs, which is produced by the difficulty of making strict cuts to separate AGNs and QSOs. When more epochs are available, it is easier for the light curve classifier to perform a correct variability-based classification, and therefore, identify an original inconsistent classification provided by a given catalog. We propose that by using the variability properties of active galactic nuclei we can more easily separate them as core-dominated or host-dominated, or in other words, as bright QSOs or low-luminosity AGNs.

In Section 5.2 we claimed that the low recall values obtained when only the g-band photometry is used for bright RRL can be explained by the differences in the variability features of the RRL subtypes. Figure 22 shows the Multiband_period versus the Meanvariance measured in the g and r bands for bright RRLs (mean g < 16) split into their subclasses "ab" and "c," and for E, CEP, and DSCT (grouped as a single class). From the figure we can notice that ab-type RRLs tend to have larger Meanvariance in the g band, which helps to distinguish them from E/CEP/DSCT, while c-type RRLs have values of Meanvariance in both bands similar to those of E/CEP/DSCT, which makes it difficult to tell them apart. The RRL class in our labeled set is dominated by ab-type RRLs (∼80%), and thus the light curve classifier identifies those more easily. However, for sources with g ≤ 15, the fraction of ab-type RRLs decreases to 64%, which explains the low recall values obtained for this regime of brightness.

Zoom In Zoom Out Reset image size

In spite of all this, the fraction of RRLs with only g-band photometry is low (see Figure 5), and thus, the low performance obtained when we only consider the g-band features for bright RRLs does not substantially affect the presented results. As a comparison, when both bands are available for bright RRLs, we classify 87.5% of them as RRLs, but when we hide the r-band features, leaving only the g band features, we recover only 17.2% of the bright RRLs.

In Section 5.2 we show that the recall curves of the CV/nova class are close to zero when only the g band is available. We noticed that in this case most of the CV/novae are misclassified as periodic by the top level of the classifier (i.e., the class with the highest probability is periodic), and that most of them are misclassified as CEP or RRL by the bottom level. For these sources, the second or third classes with the highest probabilities in the bottom level is CV/nova, and the probability of the CV/nova class returned by the stochastic classifier [P_S (CV/nova)] is larger than the probability of being a CEP or RRL returned by the periodic classifier [P_P (CEP) or P_P (RRL)]. This confusion of the CV/nova class with the periodic classes can also be seen in Figures 7 and 12. Therefore, the problem is produced in the first level of the BRF model.

We inspected the feature distributions in both bands for the CV/nova, CEP, and RRL classes, in order to understand why the results obtained when only the g band is available are so different from the results obtained when only the r band is available. We did not find large differences among the g and r feature distributions, except for the g−W3 and r−W3 colors, where the confusion of CV/nova with CEP and RRL is larger for the case of g−W3, as can be seen in Figure 23. However, these differences are not large enough to totally explain the low recall curves obtained for the CV/nova class. Another possible explanation is that there is a large fraction of Cepheids with photometry only in the g band (see Figure 5). This may be playing a role in the obtained results, since for the BRF model a lack of features in the r band and AllWISE+ZTF colors similar to those of the periodic classes could imply that the source is periodic.

Zoom In Zoom Out Reset image size

Despite all this, more than 96% of the CV/novae have photometry available in both bands, thus the low recall obtained for the g band is not a relevant issue. We plan to explore in future work better ways to classify CV/Novae. In particular, we will focus our efforts on classifying subclasses of CVs and Novae.