Planetary Candidates Observed by Kepler. VIII. A Fully Automated Catalog with Measured Completeness and Reliability Based on Data Release 25

The DR25 KOI catalog is designed to support rigorous occurrence rate studies. To do that well, it was critical that we not only identify the exoplanet transit signals in the data but also measure the catalog reliability (the fraction of transiting candidates that are not caused by noise) and the completeness of the catalog (the fraction of true transiting planets detected).

The measurement of the catalog completeness has been split into two parts: the completeness of the TCE list (the transit search performed by the Kepler Pipeline) and the completeness of the KOI catalog (the vetting of the TCEs). The completeness of the Kepler Pipeline and its search for transits have been studied by injecting transit signals into the pixels and examining what fraction are found by the Kepler Pipeline (Christiansen et al. 2013, 2015; Christiansen 2017). Burke et al. (2015) applied the appropriate detection efficiency contours (Christiansen 2015) to the 50- to 300-day-period planet candidates in the Q1–Q16 KOI catalog (Mullally et al. 2015) in order to measure the occurrence rates of small planets. However, that study was not able to account for those transit signals correctly identified by the Kepler Pipeline but thrown out by the vetting process. Along with the DR25 KOI catalog, we provide a measure of the completeness of the DR25 vetting process.

Kepler light curves contain variability that is not due to planet transits or eclipsing binaries. While the reliability of Kepler catalogs against astrophysical false positives is mostly understood (see, e.g., Morton et al. 2016), the reliability against false alarms (a term used in this paper to indicate TCEs caused by intrinsic stellar variability, overcontact binaries, or instrumental noise, i.e., anything that does not look transit-like) has not previously been measured. Instrumental noise, statistical fluctuations, poor detrending, and/or stellar variability can conspire to produce a signal that looks similar to a planet transit. When examining the smallest exoplanets in the longest orbital periods, Burke et al. (2015) demonstrated the importance of understanding the reliability of the catalog, showing that the occurrence of small, Earth-like-period planets around G-dwarf stars changed by a factor of ≈10 depending on the reliability of a few planet candidates. In this catalog we measure the reliability of the reported planet candidates against this instrumental and stellar noise.

The completeness of the vetting process is measured by vetting thousands of injected transits found by the Kepler Pipeline. Catalog reliability is measured by vetting signals found in scrambled and inverted Kepler light curves and counting the fraction of simulated false alarms that are dispositioned as planet candidates. This desire to vet both the real and simulated TCEs in a reproducible and consistent manner demands an entirely automated method for vetting the TCEs.

Automated vetting was introduced in the Q1–Q16 KOI catalog (Mullally et al. 2015) with the Centroid Robovetter and was then extended to all aspects of the vetting process for the DR24 KOI catalog (Coughlin et al. 2016). Because of this automation, the DR24 catalog was the first with a measure of completeness that extended to all parts of the search, from pixels to planet candidates. Now, with the DR25 KOI catalog and simulated false alarms, we also provide a measure of how effective the vetting techniques are at identifying noise signals and translate that into a measure of the catalog reliability. As a result, the DR25 KOI catalog is the first to explicitly balance the gains in completeness against the loss of reliability, instead of always erring on the side of high completeness.

We try to avoid unnecessary acronyms and abbreviations, but a few are required to efficiently discuss this catalog. Here we itemize those terms and abbreviations that are specific to this paper and are used repeatedly. The list is short enough that we choose to group them by meaning instead of alphabetically.

• TCE: threshold crossing event. Periodic signals identified by the transiting planet search (TPS) module of the Kepler Pipeline (Jenkins 2017).
• obsTCE: observed TCEs. TCEs found by searching the observed DR25 Kepler data and reported in Twicken et al. (2016). See Section 2.1.
• injTCE: injected TCEs. TCEs found that match a known, injected transit signal (Christiansen 2017). See Section 2.3.1.
• invTCE: inverted TCEs. TCEs found when searching the inverted data set in order to simulate instrumental false alarms (Coughlin 2017b). See Section 2.3.2.
• scrTCE: scrambled TCEs. TCEs found when searching the scrambled data set in order to simulate instrumental false alarms (Coughlin 2017b). Section 2.3.2.
• TPS: transiting planet search module. This module of the Kepler Pipeline performs the search for planet candidates. Significant, periodic events are identified by TPS and turned into TCEs.
• DV: data validation. Named after the module of the Kepler Pipeline (Jenkins 2017) that characterizes the transits and outputs one of the detrended light curves used by the Robovetter metrics. DV also created two sets of transit fits: original and supplemental (Section 2.4).
• ALT: alternative. As an alternative to the DV detrending, the Kepler Pipeline implements a detrending method that uses the methods of Garcia (2010) and the out-of-transit points in the pre-search data conditioned (PDC) light curves to detrend the data. The Kepler Pipeline performs a trapezoidal fit to the folded transit on the ALT detrended light curves (Section 2.4).
• MES: multiple event statistic. A statistic that measures the combined significance of all of the observed transits in the detrended, whitened light curve assuming a linear ephemeris (Jenkins 2002).
• KOI: Kepler Object of Interest. Periodic, transit-like events that are significant enough to warrant further review. A KOI is identified with a KOI number and can be dispositioned as a planet candidate or a false positive. The DR25 KOIs are a subset of the DR25 obsTCEs. See Section 6.
• PC: planet candidate. A TCE or KOI that passes all of the Robovetter false-positive identification tests. Planet candidates should not be confused with confirmed planets where further analysis has shown that the transiting planet model is overwhelmingly the most likely astrophysical cause for the periodic dips in the Kepler light curve. See Section 3.
• FP: false positive. A TCE or KOI that fails one or more of the Robovetter tests. Notice that the term includes all types of signals found in the TCE lists that are not caused by a transiting exoplanet, including eclipsing binaries and false alarms. See Section 3.
• MCMC: Markov chain Monte Carlo. This refers to transit fits that employ an MCMC algorithm in order to provide robust errors for fitted model parameters for all KOIs (Hoffman & Rowe 2017). See Section 6.3.

The DR25 KOI catalog is a uniformly vetted list of planet candidates and false positives found by searching the DR25 Kepler light curves and includes a measure of the catalog completeness and reliability. In the brief outline that follows we highlight how the catalog was assembled, indicate how we measure the completeness and reliability, and discuss those aspects of the process that are different from the DR24 KOI catalog (Coughlin et al. 2016).

In Section 2.1 we describe the observed TCEs (obsTCEs), which are the periodic signals found in the actual Kepler light curves. For reference, we also compare them to the DR24 TCEs. To create the simulated data sets necessary to measure the vetting completeness and the catalog reliability, we ran the Kepler Pipeline on light curves that either contained injected transits, were inverted, or were scrambled. This creates injTCEs, invTCEs, and scrTCEs, respectively (see Section 2.3).

We then created and tuned a Robovetter to vet all the different sets of TCEs. Section 3 describes the metrics and the logic used to disposition TCEs into PCs and FPs. Because the DR25 obsTCE population was significantly different from the DR24 obsTCEs, we developed new metrics to separate the PCs from the FPs (see Appendix A for the details on how each metric operates). Several new metrics examine the individual transits for evidence of instrumental noise (see Appendix A.3.7). As in the DR24 KOI catalog, we group FPs into four categories (Section 4) and provide minor false-positive flags (Appendix B) to indicate why the Robovetter decided to pass or fail a TCE. New to this catalog is the addition of a disposition score (Section 3.2) that gives users a measure of the Robovetter's confidence in each disposition.

Unlike previous catalogs, for the DR25 KOI catalog the choice of planet candidate versus false positive is no longer based on the philosophy of "innocent until proven guilty." We accept certain amounts of collateral damage (i.e., exoplanets dispositioned as FP) in order to achieve a catalog that is uniformly vetted and has acceptable levels of both completeness and reliability, especially for the long-period and low-S/N PCs. In Section 5 we discuss how we tuned the Robovetter using the simulated TCEs as populations of true planet candidates and true false alarms. We provide the Robovetter source code and all the Robovetter metrics for all of the sets of TCEs (obsTCEs, injTCEs, invTCEs, and scrTCEs) to enable users to create a catalog tuned for other regions of parameter space if their scientific goals require it.

We assemble the catalog (Section 6) by federating to previously known KOIs before creating new KOIs. Then to provide planet parameters, each KOI is fit with a transit model that uses an MCMC algorithm to provide error estimates for each fitted parameter (Section 6.3). In Section 7 we summarize the catalog and discuss the performance of the vetting using the injTCE, invTCE, and scrTCE sets. We show that both decrease significantly with decreasing number of transits and decreasing S/N. We then discuss how one may use the disposition scores to identify the highest-quality candidates, especially at long periods (Section 7.3.4). We conclude that not all declared planet candidates in our catalog are actually astrophysical transits, but we can measure what fraction are caused by stellar and instrumental noise. Because of the interest in terrestrial, temperate planets, we examine the high-quality, small candidates in the habitable zone in Section 7.5. Finally, in Section 8 we give an overview of what must be considered when using this catalog to measure accurate exoplanet occurrence rates, including what information is available in other Kepler products to do this work.

As with the previous three Kepler KOI catalogs (Mullally et al. 2015; Rowe et al. 2015a; Coughlin et al. 2016), the population of events that were used to create KOIs and planet candidates are known as obsTCEs. These are periodic reductions of flux in the light curve that were found by the TPS module and evaluated by the DV module of the Kepler Pipeline (Jenkins 2017).⁴⁰ The DR25 obsTCEs were created by running the SOC 9.3 version of the Kepler Pipeline on the DR25, Q1–Q17 Kepler time series. For a thorough discussion of the DR25 TCEs and on the pipeline's search, see Twicken et al. (2016).

The DR25 obsTCEs, their ephemerides, and the metrics calculated by the pipeline are available at the NASA Exoplanet Archive (Akeson et al. 2013). In this paper we endeavor to disposition these signals into planet candidates and false positives. Because the obsTCEs act as the input to our catalog, we first describe some of their properties as a whole and reflect on how they are different from the obsTCE populations found with previous searches.

We have plotted the distribution of the 32,534 obsTCEs in terms of period in Figure 1. Notice that there are an excessive number of short- and long-period obsTCEs compared to the number of expected transiting planets. Not shown, but worth noting, is that the number of obsTCEs increases with decreasing MES.

Zoom In Zoom Out Reset image size

As with previous catalogs, the short-period (<10 days) excess is dominated by true variability of stars owing to both intrinsic stellar variability (e.g., spots or pulsations) and contact/near-contact eclipsing binaries. The long-period excess is dominated by instrumental noise. For example, a decrease in flux following a cosmic-ray hit (known as an SPSD; Van Cleve et al. 2016a) can match up with other decrements in flux to produce a TCE. Also, image artifacts known as rolling bands are very strong on some channels (see Section 6.7 of Van Cleve & Caldwell 2016), and since the spacecraft rolls approximately every 90 days, causing a star to move on/off a Kepler detector with significant rolling-band noise, these variations can easily line up to produce TCEs at Kepler's heliocentric orbital period (≈372 days, 2.57 in log-space). This is the reason for the largest spike in the obsTCE population seen in Figure 1. The narrow spike at 459 days (2.66 in log-space) in the DR24 obsTCE distribution is caused by edge effects near three equally spaced data gaps in the DR24 data processing. The short-period spikes in the distribution of both the DR25 and DR24 obsTCEs are caused by contamination by bright variable stars (see Appendix A.6 and Coughlin et al. 2014).

Generally, the excess of long-period TCEs is significantly larger than it was in the DR24 TCE catalog (Seader et al. 2015), also seen in Figure 1. Most likely, this is because DR24 implemented an aggressive veto known as the bootstrap metric (Seader et al. 2015). For DR25 this metric was calculated but was not used as a veto. Also, other vetoes were made less strict, causing more TCEs across all periods to be created.

To summarize, for DR25 the number of false signals among the obsTCEs is dramatically larger than in any previous catalog. This was done on purpose in order to increase the Pipeline completeness by allowing more transiting exoplanets to be made into obsTCEs.

The DR25 TCE table at the NASA Exoplanet Archive contains 32,534 obsTCEs and 1498 rogue TCEs⁴¹ for a total of 34,032. The rogue TCEs are three-transit TCEs that were only created because of a bug in the TPS module of the Kepler Pipeline. This bug prevented certain three-transit events from being vetoed, and as a result they were returned as TCEs. This bug was not in place when characterizing the Pipeline using flux-level transit injection (see Burke & Catanzarite 2017a, 2017b), and because the primary purpose of this catalog is to be able to accurately calculate occurrence rates, we do not use the rogue TCEs in the creation and analysis of the DR25 KOI catalog. Also note that all of the TCE populations (observed, injection, inversion, and scrambling; see the next section) had rogue TCEs that were removed prior to analysis. The creation and analysis of this KOI catalog only rely on the nonrogue TCEs. Although they are not analyzed in this study, we encourage the community to examine the designated rogue TCEs, as the list does contain some of the longest-period events detected by Kepler.

In order to measure the performance of the Robovetter and the Kepler Pipeline, we created simulated transits, simulated false positives, and simulated false alarms. The simulated transits are created by injecting transit signals into the pixels of the original data. The simulated false positives were created by injecting eclipsing binary signals and positionally off-target transit signals into the pixels of the original data (see Coughlin 2017b; Christiansen 2017, for more information). The simulated false alarms were created in two separate ways: by inverting the light curves, and by scrambling the sequence of cadences in the time series. The TCEs that resulted from these simulated data are available at the Exoplanet Archive on the Kepler simulated data page.⁴²

2.3.1. True Transits—Injection

We empirically measure the completeness of the Kepler Pipeline and the subsequent vetting by injecting a suite of simulated transiting planet signals into the calibrated pixel data and observing their recovery, as was done for previous versions of the Kepler Pipeline (Christiansen et al. 2013, 2016; Christiansen 2015). The full analysis of the DR25 injections is described in detail in Christiansen (2017). In order to understand the completeness of the Robovetter, we use the on-target injections (Group 1 in Christiansen 2017); we briefly describe their properties here. For each of the 146,294 targets, we generate a model transit signal using the Mandel & Agol (2002) formulation, with parameters drawn from the following uniform distributions: orbital periods from 0.5 to 500 days (0.5–100 days for M-dwarf targets), planet radii from 0.25 to 7 R_⊕ (0.25–4 R_⊕ for M-dwarf targets), and impact parameters from 0 to 1. After some redistribution in planet radius to ensure sufficient coverage where the Kepler Pipeline is fully incomplete (0% recovery) to fully complete (100% recovery), 50% of the injections have planet radii below 2 R_⊕ and 90% below 40 R_⊕. The signals are injected into the calibrated pixels and then processed through the remaining components of the Kepler Pipeline in an identical fashion to the original data. Any detected signals are subjected to the same scrutiny by the Pipeline and the Robovetter as the original data. By measuring the fraction of injections that were successfully recovered by the Pipeline and called a PC by the Robovetter with any given set of parameters (e.g., orbital period and planet radius), we can then correct the number of candidates found with those parameters to the number that are truly present in the data. While the observed population of true transiting planets is heavily concentrated toward short periods, we chose the 0.5- to 500-day uniform period distribution of injections because more long-period, low-S/N transits are both not recovered and not vetted correctly—injecting more of these hard-to-find, long-period planets ensures that we can measure the Pipeline and Robovetter completeness. In this paper we use the set of on-target, injected planets that were recovered by the Kepler Pipeline (the injTCEs, whose period distribution is shown in Figure 2) to measure the performance of the Robovetter. Accurate measurement of the Robovetter performance is limited to those types of transits injected and recovered.

Zoom In Zoom Out Reset image size

It is worth noting that the injections do not completely emulate all astrophysical variations produced by a planet transiting a star. For instance, the injected model includes limb darkening, but not the occultation of stellar pulsations or granulation, which has been shown to cause a small, but non-negligible, error source on measured transit depth (Chiavassa et al. 2017) for high-S/N transits.

2.3.2. False Alarms—Inverted and Scrambled

2.3.3. Cleaning Inversion and Scrambling

The creation of this KOI catalog depends on four different transit fits: (1) the original DV transit fits, (2) the trapezoidal fits performed on the ALT Garcia (2010) detrended light curves, (3) the supplemental DV transit fits, and (4) the MCMC fits (see Section 6.3). The Kepler Pipeline fits each TCE with a Mandel & Agol (2002) transit model using Claret (2000) limb-darkening parameters. After the transit searches were performed for the observed, injected, scrambled, and inverted TCEs, we discovered that the transit fit portion of DV was seeded with a high impact parameter model that caused the final fits to be biased toward large values, causing the planet radii to be systematically too large (for further information, see Christiansen 2017; Coughlin 2017b). Since a consistent set of reliable transit fits is required for all TCEs, we refit the transits. The same DV transit fitting code was corrected for the bug and seeded with the Kepler identification number, period, epoch, and MES of the original detection. These "supplemental" DV fits do not have the same impact parameter bias as the original. Sometimes the DV fitter fails to converge, and in these cases we were not able to obtain a supplemental DV transit fit, causing us to fall back on the original DV fit. Also, at times the epoch wanders too far from the original detection; in these cases we do not consider it to be a successful fit and again fall back on the original fit.

Because the bug in the transit fits was only discovered after all of the metrics for the Robovetter were run, the original DV and trapezoidal fits were used to disposition all of the sets of TCEs. These are the same fits that are available for the obsTCEs in the DR25 TCE table at the NASA Exoplanet Archive. Nearly all of the Robovetter metrics are agnostic to the parameters of the fit, and so the supplemental DV fits would only change a few of the Robovetter decisions, namely, the V-shape metric, as it relies on the radius ratio and impact parameter (see Appendix A.4.3), and the model-shift metrics, since that test utilizes the transit model fit (see Appendix A.3.4). The impact on all the model-shift tests is negligible since they only utilize the general shape of the transit fit, and not the fitted parameters themselves. Note that if the supplemental fits were used for either test, we would have chosen different thresholds for the metrics so as to obtain a very similar catalog. While the Robovetter itself runs in a few minutes, several of the metrics used by the Robovetter (see Appendix A) require weeks to compute, so we chose not to update the metrics in order to achieve a small improvement in the consistency of our products. For all sets of TCEs, the original DV fits are listed in the Robovetter input files.⁴³ The supplemental fits are used to understand the completeness and reliability of the catalog as a function of fitted parameters (such as planet radii or insolation flux). For all sets of TCEs, the supplemental DV fits are available as part of the Robovetter results tables linked from the TCE documentation page⁴⁴ for the obsTCEs and from the simulated data page (see footnote 43; see also Christiansen 2017; Coughlin 2017b) for the injected, inverted, and scrambled TCEs.

The MCMC fits are only provided for the KOI population and are available in the DR25 KOI table⁴⁵ at the NASA Exoplanet Archive. The MCMC fits have no consistent offset from the supplemental DV fits. To show this, we plot the planet radii derived from the two types of fits for the planet candidates in DR25 and show the distribution of fractional change in planet radii; see Figure 3. The median value of the fractional change is 0.7% with a standard deviation of 18%. While individual systems disagree, there is no offset in planet radii between the two populations. The supplemental DV fitted radii and MCMC fitted radii agree within 1σ of the combined error bar (i.e., the square root of the sum of the squared errors) for 78% of the KOIs and 93.4% of PCs (only 1.8% of PC's radii differ by more than 3σ). The differences are caused by discrepancies in the detrending and because the MCMC fits include a nonlinear ephemeris in its model when appropriate (i.e., to account for transit-timing variations [TTVs]).

Zoom In Zoom Out Reset image size

Some of the derived parameters from transit fits (e.g., planetary radius and insolation flux) of the TCEs and KOIs rely critically on the accuracy of the stellar properties (e.g., radii, mass, and temperature). For all transit fits used to create this catalog we use the DR25 Q1–Q17 stellar table provided by Mathur et al. (2017), which is based on conditioning published atmospheric parameters on a grid of Dartmouth isochrones (Dotter et al. 2008). The best-available observational data for each star are used to determine the stellar parameters; for example, asteroseismic or high-resolution spectroscopic data, when available, are used instead of broadband photometric measurements. Typical uncertainties in this stellar catalog are ≈27% in radius, ≈17% in mass, and ≈51% in density, which are somewhat smaller than in previous catalogs.

After completion of the DR25 catalog, an error was discovered: the metallicities of 780 KOIs were assigned a fixed erroneous value ([Fe/H] = 0.15 dex). These targets can be identified by selecting those that have the metallicity provenance column set to "SPE90." Since radii are fairly insensitive to metallicity and the average metallicity of Kepler stars is close to solar, the impact of this error on stellar radii is typically less than 10% and does not significantly change the conclusions in this paper. Corrected stellar properties for these stars will be provided in an upcoming erratum to Mathur et al. (2017). The KOI catalog vetting and fits rely exclusively on the original DR25 stellar catalog information. Because the stellar parameters will continue to be updated (with data from missions such as Gaia; Gaia Collaboration et al. 2016a, 2016b), we perform our vetting and analysis independent of stellar properties where possible and provide the fitted information relative to the stellar properties in the KOI table. A notable exception is the limb-darkening values; precise transit models require limb-darkening coefficients that depend on the stellar temperature and gravity. However, limb-darkening coefficients are fairly insensitive to the most uncertain stellar parameters in the stellar properties catalog (e.g., surface gravity; Claret 2000).

In Figure 4 we present a flowchart that outlines our robotic vetting procedure. Each TCE is subjected to a series of "yes" or "no" questions (represented by diamonds) that either disposition it into one or more of the four FP categories or else disposition it as a PC. Behind each question is a series of more specific questions, each answered by quantitative tests.

Zoom In Zoom Out Reset image size

First, if the TCE under investigation is not the first in the system, the Robovetter checks whether the TCE corresponds to a secondary eclipse associated with an already-examined TCE in that system. If not, the Robovetter then checks whether the TCE is transit-like. If it is transit-like, the Robovetter then looks for the presence of a secondary eclipse. In parallel, the Robovetter looks for evidence of a centroid offset, as well as an ephemeris match to other TCEs and variable stars in the Kepler field.

Similar to previous KOI catalogs (Mullally et al. 2015; Rowe et al. 2015a; Coughlin et al. 2016), the Robovetter assigns FP TCEs to one or more of the following false-positive categories:

• 1.  
  Not-Transit-Like (NT): a TCE whose light curve is not consistent with that of a transiting planet or eclipsing binary. These TCEs are usually caused by instrumental artifacts or noneclipsing variable stars. If the Robovetter worked perfectly, all false alarms, as we have defined them in this paper, would be marked as FPs with only this Not-Transit-Like flag set.
• 2.  
  Stellar Eclipse (SS): a TCE that is observed to have a significant secondary event, V-shaped transit profile, or out-of-eclipse variability that indicates that the transit-like event is very likely caused by an eclipsing binary. Self-luminous, hot Jupiters with a visible secondary eclipse are also in this category, but they are still given a disposition of PC. In previous KOI catalogs this flag was known as Significant Secondary.
• 3.  
  Centroid Offset (CO): a TCE whose signal is observed to originate from a source other than the target star, based on examination of the pixel-level data.
• 4.  
  Ephemeris Match Indicates Contamination (EC): a TCE that has the same period and epoch as another object and is not the true source of the signal given the relative magnitudes, locations, and signal amplitudes of the two objects. See Coughlin et al. (2014).

The specific tests that caused the TCE to fail are specified by minor flags. These flags are described in Appendix B and are available for all FPs. Table 3 gives a summary of the specific tests run by the Robovetter when evaluating a TCE. The table lists the false-positive category (NT, SS, CO, or EC) of the test and also which minor flags are set by that test. Note that there are several informative minor flags, which are listed in Appendix B but are not listed in Table 3 because they do not change the disposition of a TCE. Also, Appendix B tabulates how often each minor flag was set to help understand the frequency of each type of FP.

New to this Robovetter are several tests that look at individual transits. The tests are named after the code that calculates the relevant metric and are called Rubble, Marshall, Chases, Skye, Zuma, and Tracker. Each metric only identifies which transits can be considered "bad," or not sufficiently transit-like. The Robovetter only fails the TCE if the number of remaining good transits is less than 3, or if the recalculated MES, using only the good transits, drops below 7.1.

Another noteworthy update to the Robovetter in DR25 is the introduction of the V-shape metric, originally introduced in Batalha et al. (2013). The intent is to remove likely eclipsing binaries that do not show significant secondary eclipses by looking at the shape and depth of the transit (see Appendix A.4.3).

We introduce a new feature to this catalog called the disposition score. Essentially the disposition score is a value between 0 and 1 that indicates the confidence in a disposition provided by the Robovetter. A higher value indicates more confidence that a TCE is a PC, regardless of the disposition it was given. This feature allows one to select the highest-quality PCs by ranking KOIs via the disposition score, for both use in selecting samples for occurrence rate calculations and prioritizing individual objects for follow-up. We stress that the disposition score does not map directly to a probability that the signal is a planet. However, in Section 7.3.4 we discuss how the disposition score can be used to adjust the reliability of a sample.

The disposition score was calculated by wrapping the Robovetter in a Monte Carlo routine. In each Monte Carlo iteration, for each TCE, new values are chosen for most of the Robovetter input metrics by drawing from an asymmetric Gaussian distribution⁴⁷ centered on the nominal value. The Robovetter then dispositions each TCE given the new values for its metrics. The disposition score is simply the fraction of Monte Carlo iterations that result in a disposition of PC. (We used 10,000 iterations for the results in this catalog.) For example, if a TCE that is initially dispositioned as a PC has several metrics that are just barely on the passing side of their Robovetter thresholds, in many iterations at least one will be perturbed across the threshold. As a result, many of the iterations will produce a false positive and the TCE will be dispositioned as a PC with a low score. Similarly, if a TCE only fails as a result of a single metric that was barely on the failing side of a threshold, the score may be near 0.5, indicating that it was deemed a PC in half of the iterations. Since a TCE is deemed an FP even if only one metric fails, nearly all FPs have scores less than 0.5, with most very close to 0.0. PCs have a wider distribution of scores from 0.0 to 1.0 depending on how many of their metrics fall near to the various Robovetter thresholds.

To compute the asymmetric Gaussian distribution for each metric, we examined the metric distributions for the injected on-target planet population on FGK dwarf targets. In a 20 by 20 grid in linear period space (ranging from 0.5 to 500 days) and logarithmic MES space (ranging from 7.1 to 100), we calculated the median absolute deviation (MAD) for those values greater than the median value and separately for those values less than the median value. We chose to use MAD because it is robust to outliers. MES and period were chosen, as they are fundamental properties of a TCE that well characterize each metric's variation. The MAD values were then multiplied by a conversion factor of 1.4826 to put the variability on the same scale as a Gaussian standard deviation (Hampel 1974; Ruppert 2010). A two-dimensional power law was then fitted to the 20 by 20 grid of standard deviation values, separately for the greater-than-median and less-than-median directions. With this analytical approximation for a given metric, an asymmetric Gaussian distribution can be generated for each metric for any TCE given its MES and period.

An example is shown in Figure 5 for the Locality Preserving Projections (LPP) metric (see Appendix A.3.1) using the DV detrending. The top left panel shows the LPP values of all on-target injected planets on FGK dwarf targets as a function of period, and the top right panel shows them as a function of MES. The middle left panel shows the measured positive 1σ deviation (in the same units as the LPP metric) as a function of MES and period, and the middle right panel shows the resulting best-fit model. The bottom panels show the same thing but for the negative 1σ deviation. As can be seen, the scatter in the LPP metric has a weak period dependence but a strong MES dependence, due to the fact that it is easier to measure the overall shape of the light curve (LPP's goal) with higher MES (S/N).

Zoom In Zoom Out Reset image size

Most, but not all, of the Robovetter metrics were amenable to this approach. Specifically, the list of metrics that were perturbed in the manner above to generate the score values were LPP (both DV and ALT), all the model-shift metrics (MS₁, MS₂, MS₃, and MS Secondary, both DV and ALT), TCE Chases, max-Single Event Statistic (SES) to MES, the two odd/even metrics (both DV and ALT), Ghost Diagnostic, and the recomputed MES using only good transits left after the individual transit metrics.

To assess the catalog reliability against false alarms, we will assume that the scrTCEs and invTCEs are similar (in frequency and type) to the obsTCEs. One way to calculate the reliability of the catalog from our false-alarm sets is to first calculate how often the Robovetter correctly identifies false alarms as FPs, a value we call effectiveness (E). Then, given the number of FPs we identify in the obsTCE set, we determine the reliability of the catalog against the type of false alarms present in the simulated sets (invTCEs and scrTCEs). This method assumes that the relative frequency of the different types of false alarms is well emulated by the simulated data sets, but it does not require the total number of false alarms to be well emulated.

Robovetter effectiveness (E) is defined as the fraction of FPs correctly identified as FPs in the obsTCE data set:

$$\begin{eqnarray}&&E\equiv \displaystyle \frac{{N}_{{\mathrm{FP}}_{\mathrm{obs}}}}{{T}_{{\mathrm{FP}}_{\mathrm{obs}}}},\end{eqnarray} \tag{ 3 }$$

where ${T}_{{\mathrm{FP}}_{\mathrm{obs}}}$ is the number of identified FPs that are truly FPs and ${N}_{{\mathrm{FP}}_{\mathrm{obs}}}$ is the total number of measured FPs. Notice that we are using N to indicate the measured number and T to indicate the "True" number.

If the simulated (sim) false-alarm TCEs accurately reflect the obsTCE false alarms, E can be estimated as the number of simulated false-alarm TCEs dispositioned as FPs (${N}_{{\mathrm{FP}}_{\mathrm{sim}}}$) divided by the number of simulated TCEs (∼N):

$$\begin{eqnarray}&&E\approx \displaystyle \frac{{N}_{{\mathrm{FP}}_{\mathrm{sim}}}}{{N}_{\mathrm{sim}}}.\end{eqnarray} \tag{ 4 }$$

For our analysis of the DR25 catalog, we primarily use the union of the invTCEs and the scrTCEs as the population of simulated false alarms when calculating E; see Section 7.3.

Recall that the Robovetter makes a binary decision, and TCEs are either PCs or FPs. For this derivation we do not take into consideration the reason the Robovetter calls a TCE an FP (i.e., some false alarms fail because the Robovetter indicates that there is a stellar eclipse or centroid offset). For most of parameter space, an overwhelming fraction of FPs are false alarms in the obsTCE data set. Future studies will look into separating out the effectiveness for different types of FPs using the set of injected astrophysical FPs (see Section 2.3).

At this point we drop the obs and sim designations in subsequent equations, as the simulated false-alarm quantities are all used to calculate E. The N values shown below refer entirely to the number of PCs or FPs in the obsTCE set so that N = N_PC + N_FP = T_PC + T_FP. We rewrite the definition for reliability (Equation (2)) in terms of the number of true false alarms in obsTCE, T_FP:

$$\begin{eqnarray}&&R\equiv \displaystyle \frac{{T}_{\mathrm{PC}}}{{N}_{\mathrm{PC}}}=1+\displaystyle \frac{{T}_{\mathrm{PC}}-{N}_{\mathrm{PC}}}{{N}_{\mathrm{PC}}}=1+\displaystyle \frac{N-{T}_{\mathrm{FP}}-{N}_{\mathrm{PC}}}{{N}_{\mathrm{PC}}}.\end{eqnarray} \tag{ 5 }$$

When we substitute N_FP = N − N_PC in Equation (5), we get another useful way to think about reliability, as 1 minus the number of unidentified FPs relative to the number of candidates,

$$\begin{eqnarray}&&R=1-\displaystyle \frac{{T}_{\mathrm{FP}}-{N}_{\mathrm{FP}}}{{N}_{\mathrm{PC}}}.\end{eqnarray} \tag{ 6 }$$

However, the true number of false alarms in the obsTCE data set, T_FP, is not known. Using the effectiveness value (Equation (4)) and combining it with our definition for effectiveness (Equation (3)), we get

$$\begin{eqnarray}&&{T}_{\mathrm{FP}}=\displaystyle \frac{{N}_{\mathrm{FP}}}{E},\end{eqnarray} \tag{ 7 }$$

and substituting into Equation (6), we get

$$\begin{eqnarray}&&R=1-\displaystyle \frac{{N}_{\mathrm{FP}}}{{N}_{\mathrm{PC}}}\left(\displaystyle \frac{1-E}{E}\right),\end{eqnarray} \tag{ 8 }$$

which relies on the approximation of E from Equation (4) and is thus a measure of the catalog reliability using all measurable quantities.

This method to calculate reliability depends sensitively on the measured effectiveness, which relies on how well the set of known false alarms match the false alarms in the obsTCE data set. For example, a negative reliability can result if the measured effectiveness is lower than the true value. In these cases, it implies that there should be more PCs than exist, i.e., the number of unidentified false alarms is smaller than the number of remaining PCs to draw from.

In order to use the scrTCE and invTCE sets to determine the reliability of our catalog, we must assume that the properties of these simulated false alarms are similar to those of the false alarms in the obsTCE set. Specifically, these simulated data should mimic the not-transit-like obsTCEs, e.g., FPs created by instrumental noise and stellar spots. For instance, our assumptions break down if all of the simulated false alarms were long-duration rolling-band FPs, but only a small fraction of the observed FPs were caused by this mechanism. Stated another way, the method we use to measure reliability hinges on the assumption that for a certain parameter space the fraction of a particular type of FP TCEs is the same between the simulated and observed data sets. This is the reason we removed the TCEs caused by KOIs and eclipsing binaries in the simulated data sets (see Section 2.3.3). Inverted eclipsing binaries and transits are not the type of FP found in the obsTCE data set. Since the Robovetter is very good at eliminating inverted transits, if they were included, we would have an inflated value for the effectiveness and thus incorrectly measure a higher reliability.

Figure 2 demonstrates that the number of TCEs from inversion and scrambling individually is smaller than the number of obsTCEs. At periods less than ≈100 days this difference is dominated by the lack of planets and eclipsing binaries in the simulated false-alarm data sets. At longer periods, where the TCEs appear to be dominated by false alarms, this difference is dominated by the cleaning (Section 2.3.3). Effectively, we search a significantly smaller number of stars for instances of false alarms. The deficit is also caused by the fact that all types of false alarms are not accounted for in these simulations. For instance, the invTCE set will not reproduce false alarms caused by sudden dropouts in pixel sensitivity caused by cosmic rays (i.e., SPSDs). The scrTCE set will not reproduce the image artifacts from rolling band because the artifacts are not as likely to line up at exactly one Kepler year. However, despite these complications, the period distribution of false alarms in these simulated data sets basically resembles the same period distribution as the obsTCE FP population once the two simulated data sets are combined. And since reliability is calculated using the fraction of false alarms that are identified (effectiveness), the overabundance that results from combining the sets is not a problem.

Another way to judge how well the simulated data sets match the type of FP in the obsTCEs is to look at some of the Robovetter metrics. Each metric measures some aspect of the TCEs. For example, the LPP metric measures whether the folded and binned light curves are transit shaped, and Skye measures whether the individual transits are likely due to rolling-band noise. If the simulated TCEs can be used to measure reliability in the way described above, then the fraction of false alarms in any period bin caused by any particular metric should match between the two sets. In Figure 6 we show that this is basically true for both invTCEs and scrTCEs, especially for periods longer than 100 days or MES less than 15. Keep in mind that more than one metric can fail any particular TCE, so the sum of the fractions across all metrics will be greater than 1. The deviation between TCE sets is as large as 40% for certain period ranges, and such differences may cause systematic errors in our measurements of reliability. But since the types of FPs overlap, it is not clear how to propagate this information into a formal systematic error bar on the reliability.

Zoom In Zoom Out Reset image size

For our discussion of the reliability estimate, we are cautiously satisfied with this basic agreement. Given that neither of the two sets performs better across all regions of parameter space, and since having more simulated false alarms improves the precision on effectiveness, we have calculated the catalog reliability using a union of the scrTCE and invTCE sets after they have been cleaned as described in Section 2.3.3.

For the first step in Robovetter tuning, we perform an optimization that finds the metric thresholds that achieve a balance between maximizing the fraction of TCEs from the injTCE set that are classified as PCs (i.e., completeness) and maximizing the fraction of TCEs from the scrTCE and invTCE sets identified as FPs (i.e., effectiveness). Optimization varies the thresholds of the subset of the Robovetter metrics described in Section 3. The set of metrics chosen for the joint optimization, called "optimized metrics," are LPP (Appendix A.3.1), the model-shift uniqueness test metrics (MS₁, MS₂, and MS₃; Appendix A.3.4), Max SES to MES (Appendix A.3.5), and TCE Chases (Appendix A.3.3). Both the DV and ALT versions of these metrics, when applicable, were used in the optimization.

Metrics not used in the joint optimization are incorporated by classifying TCEs as PCs or FPs using fixed a priori thresholds prior to optimizing the other metrics. After optimization, a TCE is classified as a PC only if it passes both the nonoptimized metrics and the optimized metrics. Prior to optimization, the fixed thresholds for these nonoptimized metrics pass about 80% of the injTCE set, so the final optimized set can have at most 80% completeness. Note that the nonoptimized metric thresholds for the DR25 catalog changed after doing these optimizations. The overall effect was that the final measured completeness of the catalog increased (see Section 7), especially for the low-MES TCEs. If the optimization were redone with these new thresholds, then it would find that the nonoptimized metrics pass 90% of the injTCEs. We decided that this change was not a sufficient reason to rerun the optimization since it was only being used to inform and not set the final thresholds.

Optimization is performed by varying the selected thresholds, determining which TCEs are classified as PCs by both the optimized and nonoptimized metrics using the new optimized thresholds, and computing C and 1 − E. Our optimization seeks thresholds that minimize the objective function $\sqrt{{(1-E)}^{2}+{(C-{C}_{0})}^{2}}$, where C₀ is the target completeness, so the optimization tries to get as close as possible to 1 − E = 0 and C = C₀. We varied C₀ in an effort to increase the effectiveness. The thresholds are varied from random starting seed values, using the Nelder–Mead simplex algorithm via the MATLAB fminsearch function. This MATLAB function varies the thresholds until the objective function is minimized. There are many local minima, so the optimal thresholds depend sensitively on the random starting threshold values. The optimal thresholds we report are the smallest of 2000 iterations with different random seed values.

Our final optimal threshold used a target of C₀ = 0.8, which resulted in thresholds that yielded E = 0.9956 and C = 0.799. We experimented with smaller values of C₀, but these did not significantly increase effectiveness. We also performed an optimization that maximized reliability defined in Section 4.1 rather than maximizing effectiveness. This yielded similar results. We also explored the dependence of the optimal thresholds on the range of TCE MES and period. We found that the thresholds have a moderate dependence, while the effectiveness and completeness have significant dependence on MES and period range. Exploration of this dependence of Robovetter threshold on MES and period range is a topic for future study.

The results of this algorithmic optimization were used as a starting point for the final thresholds chosen for the DR25 catalog. We used the Confirmed Planet table and the Certified False Positive Table at the Exoplanet Archive, as well as the results of some prominent KOIs, to manually adjust the thresholds. Because most of the injTCEs, invTCEs, and scrTCEs are at long periods and low MES, the automated tuning optimized the completeness and effectiveness for this part of the catalog. However, many of Kepler's PCs have short periods and high S/N. The final catalog thresholds balanced the needs of the different parts of the catalog and endeavored to keep the completeness of the long-period candidates above 70%.

For those interested in a certain part of the KOI catalog, it may be better to retune the thresholds to optimize for higher reliability or to more aggressively remove certain types of false alarms. The Robovetter code (see footnote 47) and the Robovetter input files are provided with the tunable thresholds listed at the top of the code. As an example, we include Table 4 as a list of the easily tunable thresholds for the metrics that determine whether an object is not transit-like. The table lists the thresholds we settled on for the DR25 catalog here, but it also provides the metrics for a higher-reliability (lower-completeness) catalog and a higher-completeness (lower-reliability) catalog. (These two different sets of thresholds are also included as commented-out lines in the Robovetter code after the set of thresholds used to create the DR25 catalog.) Each metric has its own range of possible values, and some are more sensitive to small adjustments than others. Users should use caution when changing the thresholds and should endeavor to understand the different metrics, described in Section 3 and Appendix A, before doing so.

The Robovetter gives every obsTCE a disposition, a reason for the disposition, and a disposition score. However, only those that are transit-like, i.e., have some possibility of being a transiting exoplanet or eclipsing binary system, are intended to be placed in the KOI catalog. For scheduling reasons, we created the majority of KOIs before we completed the Robovetter, so some not-transit-like KOIs have been included in the KOI catalog. Using the final set of Robovetter dispositions, we made sure to include the following obsTCEs in the KOI table: (1) those that are "transit-like," i.e., are not marked with the NT flag, and (2) KOIs that are not transit-like FPs and have a score value larger than 0.1. This last group was included to ensure that obsTCEs that marginally failed one Robovetter metric were easily accessible via the KOI catalog and given full transit fits with MCMC error bars. As in previous catalogs, all DR25 obsTCEs that federate (Section 6.2) to a previously identified KOI are included in the DR25 KOI table even if the Robovetter set the disposition to a not-transit-like FP. All previous KOIs that were not found by the DR25 Kepler Pipeline (i.e., did not trigger a DR25 obsTCE) are not included in the DR25 KOI table at the Exoplanet Archive.

All obsTCEs that were included in the KOI catalog were either federated to known KOIs or given a new KOI number. Since KOIs have been identified before, federating the known KOIs to the TCE list is a necessary step to ensure that we do not create new KOIs out of events previously identified by the Kepler Pipeline. The process has not changed substantially from the DR24 KOI catalog (see Section 4.2 of Coughlin et al. 2016), so we simply summarize the criteria for a match here. If the TCE orbital period matches within 0.2% of the KOI period, then the two are considered federated if at least 25% of the transit events overlap. If the relative period mismatch was more than 0.2%, then 70% of the transit events need to overlap. Also, those that are found at double or half the period are also considered matches (244 KOIs in total). In the cases where the period is double or half, then at least 95% of the expected (every other) transit cadences need to overlap in order to federate the two.

In some cases our automated tools want to create a new KOI in a system where one of the other previously known KOIs in the system did not federate to a DR25 TCE. In these cases we inspect the new system by hand and ensure that a new KOI number is truly warranted. If it is, we create a new KOI. If not, we ban the event from the KOI list. For instance, events that are caused by video cross talk (Van Cleve & Caldwell 2016) can cause short-period transit events to appear in only one quarter each year. As a result, the Kepler Pipeline finds several 1 yr period events for an astrophysical event that is truly closer to a few days. In these cases we federate the one found that most closely matches the known KOI, and we ban any other obsTCEs from creating a new KOI around this star. In Table 5 we report the entire list of obsTCEs that were not made into KOIs despite being dispositioned as transit-like (or not transit-like with a disposition score ≥0.1) and the automated federation telling us that one was appropriate. To identify the TCEs, we specify the Kepler Input Catalog number and the planet number given by the Kepler Pipeline (Twicken et al. 2016).

It is worth pointing out that the removal of the banned TCEs is the one pseudo-manual step that is not repeated for all the simulated TCEs. These banned TCEs effectively disappear when doing statistics on the catalog (i.e., these TCEs do not count as either a PC or an FP). They are not present in the simulated data sets, nor are we likely to remove good PCs from our sample this way. Most banned TCEs either are caused by a short-period binary whose flux is contaminating our target star (at varying depths through mechanisms such as video cross talk or reflection) or are systems with strong TTVs (see Section 6.3). In both cases, the pipeline finds several TCEs at various periods, but only one astrophysical system causes the signal. By banning these obsTCEs, we are removing duplicates from the KOI catalog and improving the completeness and reliability statistics reported in Section 7.3.

Each KOI, whether from a previous catalog or new to the DR25 catalog, was fit with a transit model in a consistent manner. The model fitting was performed in a similar way to that described in Section 5 of Rowe et al. (2015a). The model fits start by detrending the DR25 Q1–Q17 PDC photometry from MAST⁴⁸ using a polynomial filter as described in Section 4 of Rowe et al. (2014). A transit model based on Mandel & Agol (2002) is fit to the photometry using a Levenberg–Marquardt routine (More et al. 1980) assuming circular orbits and using fixed quadratic limb-darkening coefficients (Claret & Bloemen 2011) calculated using the DR25 stellar parameters (Mathur et al. 2017). TTVs are included in the model fit when necessary; the calculation of TTVs follows the procedure described in Section 4.2 of Rowe et al. (2014). The 296 KOIs with TTVs and the TTV measurements of each transit are listed in Table 6. The uncertainties for the fitted parameters were calculated using an MCMC method (Ford 2005) with a single chain with a length of 2 × 10⁵ calculated for each fit. In order to calculate the posterior distribution, the first 20% of each chain was discarded. The transit model fit parameters were combined with the DR25 stellar parameters and associated errors (Mathur et al. 2017) in order to produce the reported planetary parameters and associated errors. The MCMC chains are all available at the Exoplanet Archive and are documented in Hoffman & Rowe (2017).

The listed planet parameters come from the least-squares (LS) model fits, and the associated errors come from the MCMC calculations. Note that not all KOIs could be successfully modeled, resulting in three different fit types: LS+MCMC, LS, and none. In the case of LS+MCMC the KOIs were fully modeled with a least-squares model fit and the MCMC calculations were completed to provide associated errors. In the cases where the MCMC calculations did not converge but there is a model fit, the least-square parameters are available without uncertainties (LS fit type). In the final case, where a KOI could not be modeled (e.g., cases where the transit event was not found in the detrending used by the MCMC fits), only the period, epoch, and duration of the federated TCE are reported, and the fit type is listed as none.

The final DR25 KOI catalog, available at the NASA Exoplanet Archive, contains all TCEs that pass the not-transit-like tests (Appendix A.3) and those that fail as not-transit-like with a disposition score ≥0.1. Some overall statistics of the DR25 KOI catalog are as follows:

• 1.  
  8054 KOIs
• 2.  
  4034 PCs
• 3.  
  738 new KOIs
• 4.  
  219 new PCs
• 5.  
  85.2% of injTCEs are PCs
• 6.  
  99.6% of invTCEs and scrTCEs are FPs

A plot of the planetary periods and radii is shown in Figure 7, with the color indicating the disposition score. The distributions of the periods and planetary radii of the planet candidates in this catalog are shown along the x- and y-axes. A clear excess of candidates exists with periods near 370 days; this excess disappears if we only consider those with a disposition score >0.7. While the disposition score provides an easy way to make an additional cut on the PC population at long periods, when discussing the catalog PCs below we are using the pure dispositions of the Robovetter unless otherwise stated. The slight deficit of planets with radii just below 2.0 R_⊕ is consistent with the study of Fulton et al. (2017), where they report a natural gap in the abundance of planets between super-Earths and mini-Neptunes by applying precise stellar parameters to a subset of the Kepler transiting candidates (Johnson et al. 2017; Petigura et al. 2017b). The new KOIs with a disposition of PC are found at all periods, but only 10 have MES ≥ 10.

Zoom In Zoom Out Reset image size

We compare the DR25 KOI catalog to two sets of Kepler exoplanets: the confirmed exoplanets and the certified FPs. In both of these cases, additional observations and careful vetting are used to verify the signal as either a confirmed exoplanet or a certified FP (Bryson et al. 2017). It is worth comparing the Robovetter to these catalogs as a sanity check.

We use the confirmed exoplanet list from the Exoplanet Archive⁴⁹ on 2017 May 24. A total of 2279 confirmed planets are in the DR25 KOI catalog. The DR25 Robovetter fails 44 of these confirmed planets, or less than 2%. Half of these FPs are not-transit-like fails, 16 are stellar eclipse fails, six are centroid offsets, and one is an ephemeris match. Twelve fail owing to the LPP metric; all of these 12 have periods less than 50 days. The LPP metric threshold was set to improve the reliability of the long-period KOIs, an act that sacrificed some of the short-period KOIs. The reason the Robovetter failed each confirmed planet is given in the "koi_comment" column at the Exoplanet Archive (see Section B).

For the vast majority of the Robovetter FPs on the confirmed planet list, careful inspection reveals that there is no doubt that the Robovetter disposition is incorrect. As an example, Kepler-10b (Batalha et al. 2011; Fogtmann-Schulz et al. 2014), a rocky planet in a 0.84-day orbit, was failed owing to the LPP metric. This occurred because the detrending algorithm (the harmonic identification and removal algorithm in TPS; see Jenkins 2017) used by the Kepler Pipeline significantly distorts the shape of the transit, a known problem for strong, short-period signals (Christiansen et al. 2015). The LPP metric, which compares the shape of the folded light curve to known transits, then fails the TCE.

In some cases the Robovetter may be correctly failing the confirmed planet. Many of the confirmed planets are only statistically validated (Rowe et al. 2014; Morton et al. 2016). In these cases no additional data exist proving the physical existence of a planet outside of the transits observed by Kepler. It is possible that the DR25 light curves and metrics have now revealed evidence that the periodic events are caused by noise or a binary star. For example, Kepler-367c (Rowe et al. 2014), Kepler-1507b (Morton et al. 2016), and Kepler-1561b (Morton et al. 2016) (KOI-2173.02, KOI-3465.01, and KOI-4169.01, respectively) were all confirmed by validation and have now failed the Robovetter because of the new ghost metric (see Appendix A.5.2), indicating that the events are caused by a contaminating source not localized to the target star. These validations should be revisited in the light of these new results.

It is also worth noting that none of the confirmed circumbinary planets (e.g., Doyle et al. 2011; Orosz et al. 2012) are in the DR25 KOI catalog. However, the eclipsing binary stars that they orbit are listed as FPs. The timing and shape of the circumbinary planet transits vary in a complicated manner, making them unsuitable for detection by the search algorithm used by the Kepler Pipeline to generate the DR25 obsTCE list. As a result, this catalog cannot be used for occurrence rate estimates of circumbinary planets, and their absence in the KOI catalog should not cast doubt on their veracity.

We use the Certified False Positive table⁵⁰ downloaded from the Exoplanet Archive on 2017 July 11 to evaluate the performance of the Robovetter at removing known FPs. This table contains objects known to be FPs based on all available data, including ground-based follow-up information. The Robovetter passes 106 of the 2713 certified FPs known at the time, only 3.9%. Most of those called PCs by the Robovetter are high S/N, and more than half have periods less than 5 days. The most common reason they are certified FPs is that there is evidence they are eclipsing binaries. In some cases, external information, like radial velocity measurements, provides a mass that determines that the KOI is actually a binary system. The other main reason for the discrepancy between the tables is that the certified FPs often show evidence of significant centroid offsets. In crowded fields the Centroid Robovetter (Appendix A.5.1) will not fail observed offsets because of the potential for confusion. For the Certified False Positive table, individual cases are examined by a team of scientists who determine when there is sufficient proof that the signal is indeed caused by a background eclipsing binary.

To evaluate the performance of the Robovetter and to measure the catalog completeness and reliability, we run the Robovetter on the injTCEs, invTCEs, and scrTCEs. As a high-level summary, Figure 8 provides the completeness, effectiveness (E), and reliability for a 3 by 3 grid across period and MES. If the same figure is made for only the FGK-dwarf-type stars ($\mathrm{log}g$ ≥ 4.0 and 4000 K ≥ T_⋆ < 7000 K), the long-period, low-MES bin improves substantially. Giant stars are inherently noisy on timescales of planet transits (see Figure 9 of Christiansen et al. 2012), causing more FPs and also causing more real transits to be distorted by the noise. For FGK-dwarf stars and only considering candidates with periods between 200 and 500 days and MES < 10, C = 76.7%, 1 − E = 1.1%, and R = 50.3%, which is a 13.1 percentage-point improvement in reliability and 3% point improvement in completeness compared to all stars in the same period and MES range.

Zoom In Zoom Out Reset image size

7.3.1. Completeness

The completeness of the vetting is measured as the fraction of injTCEs that are dispositioned as PCs. We discuss here the detection efficiency of the Robovetter, not the Kepler Pipeline (see Section 8 for a discussion of the pipeline completeness). Across the entire set of recovered injTCEs that have periods ranging from 0.5 to 500 days, the Robovetter dispositioned 85.2% as PC. As expected, the vetting completeness is higher for transits at shorter periods and higher MES and lower for longer periods and lower MES. The right column of Figure 9 shows how the completeness varies with period, expected MES, number of transits, and transit duration. Note that expected MES is the average MES at which the injected transit signal would be measured in the target light curve, given the average photometric noise of that light curve and the depth of the injected transit signal—see Christiansen (2017) for more details. The small drop in completeness just short of 90 days is likely caused by the odd–even metric (Appendix A.4.1.4), which only operates out to 90 days, confusing true transits for binary eclipses.

Zoom In Zoom Out Reset image size

Because most planet occurrence rate calculations are performed using period and radius (e.g., Burke et al. 2015), we show the measured completeness binned by period and radius in Figure 10. The plot is linear in period and radius in order to emphasize the long-period planets. Planetary radius is not a natural unit to understand the performance of the Robovetter since it combines the depth of the transit, the noise level of the light curve, and the stellar radius. At the longest periods the Robovetter is more likely to disposition larger injected planets as FPs than the smaller counterparts, though the trend is reduced when only considering the FGK-dwarf stars. The reason for this is that the largest-radius planets in the injTCE population are entirely around giant stars; large planets were not injected onto the dwarf stars (Christiansen 2017). The giant stars are notoriously noisy. As a result, the largest-radius planets in the injTCEs are more likely to be dispositioned incorrectly. Also, even when only considering the dwarf stars, a larger fraction of the big planets will be around larger, more massive stars (in comparison to the small planets, which will mostly be found around smaller stars). This results in a population of planets that produce longer transit durations. The Robovetter performs less well for long transit durations (see Figure 9). For more figures showing the Robovetter effectiveness across different parameters, see Coughlin (2017b).

Zoom In Zoom Out Reset image size

7.3.2. Effectiveness

7.3.3. Reliability

The reliability is measured according to the method described in Section 4.1 using the effectiveness measured from the combined scrTCE and invTCE data sets and the number of observed PCs. If one bins over the entire data set, the overall reliability of the catalog is 97%. However, as Figure 9 demonstrates, the reliability for long-period and especially low-MES planets is significantly smaller. For periods longer than 200 days and MES less than 10, the reliability of the catalog is approximately 37%, i.e., 6 out of 10 PCs are caused by noise. As with completeness, we plot the reliability as a function of period and planet radius in Figure 11. The least reliable planets are at long periods and have radii less than 4 R_⊕.

Zoom In Zoom Out Reset image size

The uncertainty in the reliability is likely dominated by how well the false alarms in the scrTCE and invTCE sets match the false alarms in the obsTCE data set (see Section 4.2 for further discussion on their similarity). One way to get a handle on the uncertainty on reliability is to calculate the reliability in three different ways for the long-period (200–500 days), low-MES (<10) obsTCEs. First, we use only the invTCEs to measure the effectiveness at removing false alarms. This results in a lower reliability, namely, R = 24% with E = 98.5%. Second, we use only the scrTCEs to measure the effectiveness. This results in a higher reliability, R = 51% with E = 99.1%. Third, we select, at random, half of the combined population of false alarms (scrTCE and invTCE) and calculate the reliability. After doing this random selection 100 times, we obtained R = 38% with a standard deviation of 8%, and the distribution appears symmetric and basically Gaussian in shape.

The Robovetter is less effective at removing the false alarms produced by inversion than those by scrambling the data. Inversion finds false alarms with periods near 372 days, which are frequently caused by image artifacts. Scrambling underpopulates these types of false alarms, and since they are difficult to eliminate, it is not surprising that the reliability measured by inversion is worse than scrambling. The truth likely lies somewhere in between. We encourage users of these data sets to consider ways to optimize the reliability measurement, and the error budget associated with them, when doing occurrence rate calculations. We remind the reader that this analysis only concerns the reliability against the false alarms that are present in the scrTCEs and invTCEs. Previous studies (e.g., Santerne et al. 2012) discuss the reliability of previous KOI catalogs against short-period eclipsing binaries. However, since the Robovetter logic has changed considerably for this catalog (specifically, the V-shaped metric was introduced and tuned to account for these false positives), the eclipsing binary false-positive rate should be reevaluated for this DR25 KOI catalog.

7.3.4. High Reliability Using the Disposition Score

The disposition scores discussed in Section 3.2 can be used to select a more reliable, though more incomplete, sample of planet candidates. In Figure 12 we show the distribution of disposition scores for the PCs and FPs from the observed, inverted, scrambled, and on-target planet injection populations. (Note that the inverted and scrambled populations have been cleaned as discussed in Section 2.3.3.) For all populations, the PC distribution tends to cluster near a score of 1.0 with a tail that extends toward lower score values. Lower MES values tend to have a greater proportion of lower score values. Similarly, the vast majority of FPs have a score of 0.0, with only a small fraction extending toward higher score values (note that the y-axis for the FPs is logarithmic, while the y-axis for PCs is linear). Comparing the populations, the on-target planet injections have a greater concentration of score values toward 0.5 for both the PCs and FPs than other populations. Both the inverted and scrambled populations have very few PCs near high-score values. We can exploit the relative distribution of PC and FP score values for the different populations to select a higher reliability catalog.

Zoom In Zoom Out Reset image size

At the top of Figure 13 we show how the completeness and reliability of the catalog vary for different cuts on the disposition score for MES < 10 and periods between 200 and 500 days. The effectiveness of the Robovetter increases as the score threshold is increased. The reliability values also depend on the number of observed PCs that remain, which is why reliability does not change in step with the effectiveness. Selecting the PC sample by choosing those with a disposition score above 0.6 (see the point labeled 0.6 on the top of Figure 13) yields an 85% reliability and a completeness that is still above 50%. Doing a score cut in this way not only removes those dispositioned as a PC from the sample but also causes a few obsTCEs that are formally dispositioned as FPs to now be included in the sample. An FP with a high score occurs when a TCE marginally fails a single metric.

Zoom In Zoom Out Reset image size

It is interesting to note that the number of inferred candidates, i.e., the number of candidates after accounting for the Robovetter completeness and catalog reliability, does not change significantly with the score cut. In the bottom panel of Figure 13 we plot both the observed number of PCs and the corrected number of PCs that have periods between 200 and 500 days and MES less than 10. The correction is done by taking the number of PCs and multiplying by the reliability and dividing by the completeness. The error bars only include the Poisson counting error in the number of observed PCs and do not include errors in the measured completeness or reliability. The corrected number of PCs only varies by approximately 1σ regardless of the score cut used.

Lissauer et al. (2014) argue that almost all multiplanet, transit systems are real. Forty-seven, or 21%, of the new DR25 PCs are associated with targets with multiple PCs. One of the new PCs, KOI-82.06, is part of a six-candidate system around the star Kepler-102. Five candidates have previously been confirmed (Marcy et al. 2014; Rowe et al. 2014) in this system. The new candidate is the largest radius-confirmed planet in the system. It also lies a bit outside the 4:3 resonance, possibly adding to the excess of planets found just wide of such first-order resonances (Lissauer et al. 2011a). If verified, this would be only the third system with six or more planets found by Kepler. The other new candidate within a high-multiplicity system is KOI-2926.05. The other four candidates in this system around Kepler-1388 have been validated by Morton et al. (2016). This new candidate also orbits just exterior to a first-order mean motion resonance with one of the four previously known planets.

Kepler is NASA's first mission capable of detecting Earth-size planets around Sun-like stars in 1 yr orbits. One of its primary science goals is to determine the occurrence rate of potentially habitable, terrestrial-size planets—a value often referred to as eta-Earth. Here we use the concept of a habitable zone to select a sample of planet candidates that are the right distance from their host stars and small enough to possibly have a rocky surface. A point that bears repeating is that no claims can be made regarding planetary habitability based on size and orbital distance alone. This sample is, however, of great value to the occurrence rate studies that enable planet yield estimates for various designs of future life-detection missions (Stark et al. 2015). This eta-Earth sample is provided in Table 7 and shown in Figure 14.

Zoom In Zoom Out Reset image size

Table 7.  Habitable Zone Terrestrial-sized Planet Candidates

KOI	KIC	Kepler	Period	Rp	Sp	T⋆	R⋆	MES	Disp.
			(days)	(R⊕)	(S⊕)	(K)	(R☉)		Score
172.02	8692861	Kepler-69c	242.46130	${1.73}_{-0.22}^{+0.21}$	${1.59}_{-0.45}^{+0.59}$	${5637}_{-101}^{+113}$	${0.94}_{-0.12}^{+0.12}$	18.0	0.693
238.03	7219825	⋯	362.97828	${1.96}_{-0.29}^{+0.33}$	${1.81}_{-0.60}^{+0.87}$	${6086}_{-133}^{+133}$	${1.22}_{-0.18}^{+0.20}$	11.9	0.784
438.02	12302530	Kepler-155c	52.66153	${1.87}_{-0.12}^{+0.11}$	${1.28}_{-0.25}^{+0.26}$	${3984}_{-86}^{+71}$	${0.54}_{-0.04}^{+0.03}$	30.6	1.000
463.01c	8845205	Kepler-560b	18.47763	${1.55}_{-0.29}^{+0.32}$	${1.21}_{-0.47}^{+0.72}$	${3395}_{-67}^{+74}$	${0.28}_{-0.05}^{+0.06}$	78.0	0.001
494.01	3966801	Kepler-577b	25.69581	${1.70}_{-0.33}^{+0.21}$	${2.30}_{-1.10}^{+1.17}$	${3787}_{-204}^{+163}$	${0.48}_{-0.09}^{+0.06}$	35.9	1.000
571.05a	8120608	Kepler-186f	129.94410	${1.18}_{-0.14}^{+0.11}$	${0.23}_{-0.06}^{+0.07}$	${3751}_{-84}^{+75}$	${0.44}_{-0.05}^{+0.04}$	7.7	0.677
701.03	9002278	Kepler-62e	122.38740	${1.72}_{-0.07}^{+0.10}$	${1.24}_{-0.19}^{+0.27}$	${4926}_{-98}^{+98}$	${0.66}_{-0.03}^{+0.04}$	35.9	0.994
701.04d	9002278	Kepler-62f	267.29100	${1.43}_{-0.06}^{+0.08}$	${0.44}_{-0.07}^{+0.09}$	${4926}_{-98}^{+98}$	${0.66}_{-0.03}^{+0.04}$	14.3	0.000
812.03	4139816	Kepler-235e	46.18420	${1.83}_{-0.15}^{+0.12}$	${1.32}_{-0.30}^{+0.29}$	${3950}_{-86}^{+70}$	${0.49}_{-0.04}^{+0.03}$	18.0	1.000
854.01	6435936	Kepler-705b	56.05608	${1.94}_{-0.22}^{+0.12}$	${0.69}_{-0.19}^{+0.15}$	${3593}_{-86}^{+71}$	${0.49}_{-0.06}^{+0.03}$	19.3	0.996
947.01	9710326	Kepler-737b	28.59914	${1.83}_{-0.21}^{+0.16}$	${1.87}_{-0.53}^{+0.52}$	${3755}_{-84}^{+75}$	${0.46}_{-0.05}^{+0.04}$	45.7	1.000
1078.03	10166274	Kepler-267d	28.46465	${1.87}_{-0.22}^{+0.14}$	${1.95}_{-0.55}^{+0.49}$	${3789}_{-82}^{+75}$	${0.46}_{-0.05}^{+0.04}$	22.2	0.992
1298.02d	10604335	Kepler-283c	92.74958	${1.87}_{-0.10}^{+0.08}$	${0.78}_{-0.14}^{+0.15}$	${4141}_{-91}^{+83}$	${0.58}_{-0.03}^{+0.03}$	10.7	0.000
1404.02	8874090	⋯	18.90609	${0.87}_{-0.21}^{+0.16}$	${3.03}_{-1.67}^{+2.29}$	${3751}_{-219}^{+219}$	${0.45}_{-0.11}^{+0.08}$	10.1	0.955
1422.02b	11497958	Kepler-296d	19.85029	${1.52}_{-0.23}^{+0.19}$	${1.83}_{-0.62}^{+0.68}$	${3526}_{-78}^{+71}$	${0.38}_{-0.06}^{+0.05}$	25.1	1.000
1422.04	11497958	Kepler-296f	63.33627	${1.18}_{-0.18}^{+0.15}$	${0.39}_{-0.13}^{+0.15}$	${3526}_{-78}^{+71}$	${0.38}_{-0.06}^{+0.05}$	9.1	0.927
1422.05	11497958	Kepler-296e	34.14211	${1.06}_{-0.16}^{+0.13}$	${0.89}_{-0.30}^{+0.33}$	${3526}_{-78}^{+71}$	${0.38}_{-0.06}^{+0.05}$	10.5	0.984
1596.02	10027323	Kepler-309c	105.35823	${1.87}_{-0.17}^{+0.13}$	${0.41}_{-0.10}^{+0.09}$	${3883}_{-93}^{+69}$	${0.50}_{-0.04}^{+0.04}$	16.5	0.738
2162.02	9205938	⋯	199.66876	${1.45}_{-0.18}^{+0.18}$	${2.06}_{-0.59}^{+0.76}$	${5678}_{-102}^{+113}$	${0.92}_{-0.12}^{+0.12}$	11.1	0.920
2184.02e	12885212	⋯	95.90640	${2.17}_{-0.12}^{+0.07}$	${1.63}_{-0.29}^{+0.20}$	${4620}_{-82}^{+73}$	${0.74}_{-0.04}^{+0.02}$	8.92	0.638
2418.01	10027247	Kepler-1229b	86.82952	${1.68}_{-0.21}^{+0.12}$	${0.35}_{-0.11}^{+0.08}$	${3576}_{-85}^{+71}$	${0.46}_{-0.06}^{+0.03}$	11.7	0.937
2626.01	11768142	⋯	38.09707	${1.58}_{-0.21}^{+0.20}$	${0.81}_{-0.25}^{+0.30}$	${3554}_{-80}^{+71}$	${0.40}_{-0.05}^{+0.05}$	14.6	0.999
2650.01	8890150	Kepler-395c	34.98978	${1.14}_{-0.10}^{+0.07}$	${1.71}_{-0.42}^{+0.35}$	${3765}_{-83}^{+75}$	${0.52}_{-0.05}^{+0.03}$	10.1	0.985
2719.02	5184911	⋯	106.25976	${1.50}_{-0.16}^{+0.10}$	${1.99}_{-0.58}^{+0.53}$	${4827}_{-144}^{+129}$	${0.82}_{-0.09}^{+0.06}$	10.0	0.990
3010.01	3642335	Kepler-1410b	60.86610	${1.39}_{-0.10}^{+0.07}$	${0.84}_{-0.16}^{+0.17}$	${3808}_{-76}^{+69}$	${0.52}_{-0.04}^{+0.03}$	12.7	0.996
3034.01	2973386	⋯	31.02092	${1.66}_{-0.17}^{+0.12}$	${1.70}_{-0.45}^{+0.40}$	${3720}_{-81}^{+73}$	${0.48}_{-0.05}^{+0.03}$	11.9	1.000
3138.01b	6444896	Kepler-1649b	8.68909	${0.49}_{-0.00}^{+0.00}$	${0.47}_{-0.00}^{+0.00}$	${2703}_{-0}^{+0}$	${0.12}_{-0.00}^{+0.00}$	12.0	1.000
3282.01	12066569	Kepler-1455b	49.27684	${1.75}_{-0.13}^{+0.09}$	${1.28}_{-0.26}^{+0.26}$	${3899}_{-78}^{+78}$	${0.53}_{-0.04}^{+0.03}$	14.7	0.996
3284.01	6497146	Kepler-438b	35.23319	${0.97}_{-0.07}^{+0.06}$	${1.62}_{-0.34}^{+0.37}$	${3749}_{-84}^{+75}$	${0.52}_{-0.04}^{+0.03}$	11.9	1.000
3497.01	8424002	Kepler-1512b	20.35972	${0.80}_{-0.16}^{+0.12}$	${1.38}_{-0.58}^{+0.58}$	${3419}_{-76}^{+67}$	${0.34}_{-0.07}^{+0.05}$	19.6	1.000
4005.01a	8142787	Kepler-439b	178.13960	${2.25}_{-0.16}^{+0.22}$	${1.70}_{-0.31}^{+0.47}$	${5431}_{-81}^{+81}$	${0.88}_{-0.06}^{+0.09}$	17.8	0.997
4036.01	11415243	Kepler-1544b	168.81133	${1.69}_{-0.06}^{+0.10}$	${0.80}_{-0.12}^{+0.17}$	${4798}_{-95}^{+95}$	${0.71}_{-0.03}^{+0.04}$	14.8	0.965
4087.01	6106282	Kepler-440b	101.11141	${1.61}_{-0.08}^{+0.10}$	${0.65}_{-0.11}^{+0.14}$	${4133}_{-82}^{+74}$	${0.56}_{-0.03}^{+0.03}$	15.7	1.000
4356.01a	8459663	Kepler-1593b	174.51028	${1.74}_{-0.20}^{+0.14}$	${0.28}_{-0.09}^{+0.09}$	${4367}_{-155}^{+124}$	${0.45}_{-0.05}^{+0.04}$	11.0	0.976
4427.01	4172805	⋯	147.66173	${1.59}_{-0.14}^{+0.12}$	${0.23}_{-0.05}^{+0.06}$	${3788}_{-84}^{+76}$	${0.49}_{-0.04}^{+0.04}$	10.8	0.969
4460.01	9947389	⋯	284.72721	${2.02}_{-0.29}^{+0.30}$	${1.41}_{-0.44}^{+0.55}$	${5497}_{-74}^{+82}$	${1.08}_{-0.16}^{+0.16}$	10.7	0.972
4550.01	5977470	⋯	140.25194	${1.84}_{-0.12}^{+0.05}$	${1.28}_{-0.24}^{+0.17}$	${4821}_{-86}^{+76}$	${0.79}_{-0.05}^{+0.02}$	9.6	0.934
4622.01	11284772	Kepler-441b	207.24820	${1.56}_{-0.06}^{+0.09}$	${0.30}_{-0.05}^{+0.06}$	${4339}_{-87}^{+78}$	${0.55}_{-0.02}^{+0.03}$	9.7	0.975
4742.01	4138008	Kepler-442b	112.30530	${1.30}_{-0.05}^{+0.07}$	${0.79}_{-0.11}^{+0.15}$	${4401}_{-78}^{+78}$	${0.59}_{-0.02}^{+0.03}$	12.9	0.993
7016.01	8311864	Kepler-452b	384.84300	${1.09}_{-0.10}^{+0.20}$	${0.56}_{-0.15}^{+0.32}$	${5579}_{-150}^{+150}$	${0.80}_{-0.07}^{+0.15}$	7.6	0.771
7223.01	9674320	⋯	317.06242	${1.59}_{-0.12}^{+0.27}$	${0.54}_{-0.13}^{+0.29}$	${5366}_{-144}^{+160}$	${0.71}_{-0.05}^{+0.12}$	10.3	0.947
7706.01	4762283	⋯	42.04952	${1.19}_{-0.16}^{+0.08}$	${2.00}_{-0.68}^{+0.55}$	${4281}_{-140}^{+115}$	${0.48}_{-0.06}^{+0.03}$	7.5	0.837
7711.01	4940203	⋯	302.77982	${1.31}_{-0.12}^{+0.34}$	${0.87}_{-0.22}^{+0.66}$	${5734}_{-154}^{+154}$	${0.80}_{-0.07}^{+0.21}$	8.5	0.987
7882.01	8364232	⋯	65.41518	${1.31}_{-0.12}^{+0.08}$	${1.79}_{-0.47}^{+0.49}$	${4348}_{-130}^{+130}$	${0.65}_{-0.06}^{+0.04}$	7.2	0.529
7894.01	8555967	⋯	347.97611	${1.62}_{-0.15}^{+0.49}$	${0.97}_{-0.27}^{+0.87}$	${5995}_{-181}^{+163}$	${0.88}_{-0.08}^{+0.27}$	8.5	0.837
7923.01	9084569	⋯	395.13138	${0.97}_{-0.10}^{+0.12}$	${0.44}_{-0.13}^{+0.20}$	${5060}_{-174}^{+192}$	${0.87}_{-0.09}^{+0.10}$	10.0	0.750
7954.01	9650762	⋯	372.15035	${1.74}_{-0.14}^{+0.46}$	${0.69}_{-0.18}^{+0.52}$	${5769}_{-172}^{+155}$	${0.81}_{-0.07}^{+0.21}$	8.9	0.839
8000.01	10331279	⋯	225.48805	${1.70}_{-0.14}^{+0.43}$	${1.20}_{-0.30}^{+0.90}$	${5663}_{-152}^{+169}$	${0.78}_{-0.07}^{+0.19}$	8.7	0.975
8012.01	10452252	⋯	34.57372	${0.42}_{-0.12}^{+0.17}$	${0.37}_{-0.19}^{+0.47}$	${3374}_{-82}^{+112}$	${0.22}_{-0.06}^{+0.09}$	7.7	0.989
8174.01	8873873	⋯	295.06066	${0.64}_{-0.07}^{+0.07}$	${0.70}_{-0.21}^{+0.28}$	${5332}_{-144}^{+160}$	${0.76}_{-0.09}^{+0.09}$	7.4	0.665

Notes.

^aConfirmed planet properties from NASA Exoplanet Archive on 2017 May 31 place object within HZ. ^bConfirmed planet properties from NASA Exoplanet Archive on 2017 May 31 place object exterior to the HZ. ^cConfirmed planet with vetting score less than 0.5. ^dConfirmed planet dispositioned as false positive in DR25. ^eThe erratum to Mathur et al. (2017) reduces planet size, now placing the object in the eta-Earth sample.

A machine-readable version of the table is available.

Download table as:  DataTypeset image

7.5.1. Selecting the Eta-Earth Sample

Before applying thresholds on planet properties, we first select a sample based on disposition score (see Section 3.2) in order to produce a sample of higher-reliability planets orbiting G-type stars. At long orbital period and small radius, we are vulnerable to instrumental false alarms despite the significant improvements afforded us by the latest versions of metrics like Marshall, Skye, Rubble, Chases, and Model-Shift. This is evident in the FGK dwarf sample of Figures 10 and 11 by comparing the relatively low reliability (45%–74%) and completeness (74%–88%) measurements in the bottom right boxes to others at shorter period and larger radius. Removing candidates with score <0.5 results in a significant improvement in the sample reliability with a small degradation in the sample completeness (Figure 13). The candidates reported in Table 7 are ≈80% reliable for the G-type stars and even higher for the K- and M-type stars. Note that there is only one late F-type star in the sample. Kepler was not designed to find small planets in the habitable zones of F-type stars, and those in the DR25 catalog are of low reliability and have disposition scores less than 0.5.

The DR25 catalog uses the transit depth and period, along with the DR25 stellar table of Mathur et al. (2017), to derive the planet radius and the semimajor axis of the planet's orbit. From these we calculate the insolation flux in units of the Earth's insolation flux,

$$\begin{eqnarray}&&{S}_{p}=\displaystyle \frac{{R}_{\star }^{2}\cdot {({T}_{\star }/5777)}^{4}}{{a}^{2}},\end{eqnarray} \tag{ 9 }$$

where a is the semimajor axis of the planet's orbit in au, T_⋆ is the host star temperature in kelvin, 5777 K is the effective temperature of the Sun, R_⋆ is the radius of the star in units of R_☉, and thus S_p is in units relative to the Earth's insolation flux. The errors for both insolation flux and radii include the errors from the DR25 stellar catalog. The habitable zone represents a range of orbits where the flux received by the host star allows for the possibility of surface liquid water on an Earth-size planet. While the insolation limits for the habitable zone depend on the stellar temperature, it roughly falls from 0.2 to 1.7 S_⊕ (see Figure 14). We use the empirical (recent Venus/early Mars) habitable zone of Kopparapu et al. (2013). To err on the side of inclusiveness, we include candidates whose 1σ error bars on the insolation flux overlap this empirical habitable zone.

Finally, we include only those candidates that satisfy the size constraint ${R}_{p}-{\sigma }_{{R}_{p},\mathrm{low}}\lt 1.8\,{R}_{\oplus }$. The purpose of the size constraint is to identify candidates likely to have a bulk composition similar to terrestrial planets in the solar system. The 1.8 R_⊕ upper limit is taken from Fulton et al. (2017), who report a distinct gap in the radius distribution of exoplanets for planets in orbital periods of less than 100 days. The authors argue that the gap is the result of two (possibly overlapping) population distributions: the rocky terrestrials and the mini-Neptune-size planets characterized by their volatile-rich envelopes. Within this framework, the center of the gap marks a probabilistic boundary between having a higher likelihood of a terrestrial composition and a higher likelihood of a volatile-rich envelope. However, this boundary was identified using planets in orbital periods of less than 100 days, and it may not exist for planets in longer-period orbits. Also, it is not entirely clear that planets on the small side of this gap are all terrestrial. Rogers (2015) examined small planets with density measurements with periods less than ≈50 days and showed that less than half of planets with a radius of 1.62 R_⊕ have densities consistent with a body primarily composed of iron and silicates. For our purposes of highlighting the smallest planets in this catalog, we chose to be inclusive and set the threshold at 1.8 R_⊕.

To summarize, Table 7 lists those candidates with scores greater than 0.5 and whose error bars indicate that they could be smaller than 1.8 R_⊕ and lie in the habitable zone. The table also includes KOI-2184.02 because the erratum to Mathur et al. (2017; see Section 2.5 of this paper) reduces the stellar and planet radii so that the PC now lies in our sample. Note that the same erratum also reduces the planet radii of KOI-4460.01 and KOI-4550.01 to 2.0 and 1.65 R_⊕, respectively. The values reported in Table 7 are identical to those in the KOI table at the NASA Exoplanet Archive and do not include the values reported in the erratum to Mathur et al. (2017). Also, in order to make Table 7 complete, we include any Kepler terrestrial-size confirmed planet that falls in the habitable zone of its star according to the confirmed planet table at the Exoplanet Archive (downloaded on 2017 May 15). The objects are included and denoted with footnotes, even if the DR25 catalog dispositions them as FPs or if the DR25 planetary parameters place them outside the habitable zone. However, note that statistical inferences like occurrence rates should be based on a uniform sample drawn exclusively from the DR25 catalog and its self-consistent completeness and reliability measurements (see Section 8).

We plot the eta-Earth sample candidates in Figure 14, using only the information in the DR25 KOI catalog. Notice that this final search of the Kepler data not only identified previously discovered candidates around the M-dwarf stars but also yielded a handful of highly reliable candidates around the GK-dwarf stars. These GK-dwarf candidates have fewer transits and shallower depths, making them much more difficult to find. Despite their lower S/N, because we provide a measure of the reliability against false alarms (along with the completeness), these candidates are available to further study the occurrence rates of small planets in the habitable zone of GK-dwarf stars.

7.5.2. Notes on the Eta-Earth Sample

When selecting candidates from the KOI catalog for further study, as we did for the eta-Earth sample (Section 7.5), it is important to remember a few caveats. First, even with a high cut on disposition score, the reliability against false alarms is not 100%. Some candidates may still be caused by false alarms, especially those around the larger, hotter stars. Also, this reliability number does not include the astrophysical reliability. Many of our tools to detect astrophysical false positives do not work for long-period, low-MES candidates. For example, it is nearly impossible to detect the centroid offset created from a background eclipsing binary, and secondary eclipses are not deep enough to detect for these stars.

Second, the measured radius and semimajor axis of each planet depend on the stellar catalog. As discussed in Section 2.5 and Mathur et al. (2017), the stellar radii and masses are only known to a certain precision, and the quality of the data used to derive these stellar properties varies between targets. These unknowns are reflected in the 1σ error bars shown in Figure 14 and listed in the KOI table. The uncertainty in the stellar information limits our knowledge of these planets. As an example, for Kepler-452 (KIC 8311864), the DR25 stellar catalog lists a temperature of 5579 ± 150 K and stellar radius of ${0.798}_{-0.075}^{+0.150}\,{R}_{\odot }$, while the values in the confirmation paper (Jenkins et al. 2015) after extensive follow-up are 5757 ± 85 K for the effective temperature and ${1.11}_{-0.09}^{+0.15}$ for the stellar radius. As a result, the planet Kepler-452b is given as 1.6 ± 0.2 R_⊕ in Jenkins et al. (2015) and ${1.09}_{-0.1}^{+0.2}\,{R}_{\oplus }$ in the DR25 catalog. The radii and stellar temperature differ by less than 2σ, but those differences change the interpretation of the planet from a super-Earth in the middle of the habitable zone of an early G-dwarf host to an Earth-size planet receiving about half the amount of flux from a late K star. As follow-up observations of each candidate star are obtained and errors on the stellar parameters decrease, we expect this population to change in significant ways.

Third, high-resolution imaging has proven crucial for identifying light from background and bound stars that add flux to the Kepler photometric time series (Furlan et al. 2017). When this occurs, unaccounted-for extra light dilutes the transit, causing the radii to be significantly underestimated (Ciardi et al. 2015; Furlan & Howell 2017). As a result, we fully expect that once follow-up observations are obtained for these stars, several of the PCs in this catalog, including those listed in the eta-Earth sample, will be found to have radii larger than reported in this catalog.

Any measure of the catalog completeness must include the completeness of the Robovetter and the Kepler Pipeline. The pipeline's detection efficiency has been explored in two ways: using pixel-level transit injection and using flux-level transit injection. In the former, a simulated transiting planet signal is injected into the calibrated pixels of each Kepler target, which are then processed through the pipeline. This experiment provides an estimate of the average detection efficiency over all the stars that were searched. A full description of the signals that were injected and recovered can be found in Christiansen (2017). The pixel-level measurements have the advantage of following transit signals through all the processing steps of the Kepler Pipeline, and the recovered signals can be further classified with the Robovetter, as demonstrated in Section 7.3. Figure 15 shows the average pipeline detection efficiency for a sample of FGK stars: the left panel shows the pipeline detection efficiency, and the right panel shows the combined Kepler Pipeline and Robovetter detection efficiency, calculated by taking the injections that were successfully recovered by the pipeline and processing them through the Robovetter. A gamma cumulative distribution function is fit to both (see Equation (1) of Christiansen et al. 2016). Notice that the detection efficiency decreases by 5–10 percentage points (of the entire set that were injected) for all MES, as expected given the results shown in Figure 9.

Zoom In Zoom Out Reset image size

Since the pixel-level transit injection includes only one injection per target, it does not examine potential variations in the pipeline completeness for individual targets owing to differences in stellar properties or astrophysical variability. To probe these variations, a small number of individual stars had a large number of transiting signals (either several thousand or several hundred thousand, depending on the analysis) injected into the detrended photometry, which was processed only through the transit-search portion of the TPS module. The flux-level injections revealed that there are significant target-to-target variations in the detection efficiency. The flux-level injections and the resulting detection efficiency are available for the sample of stars that were part of this study. For more information on the flux-level injection study, see Burke & Catanzarite (2017c). All products associated with the flux-level and pixel-level injections can be found at the NASA Exoplanet Archive (see footnote 43).

We have described the reliability of the DR25 candidates with regard to the possibility that the observed events are actually caused by stellar or instrumental noise. See Section 7.3 for how this reliability varies with various measured parameters. However, even if the observed signal is not noise, other astrophysical events can mimic a transit. Some of these other astrophysical events are removed by carefully vetting the KOI with Kepler data alone. Specifically, the Robovetter looks for significant secondary eclipses to rule out eclipsing binaries and for a significant offset in the location of the in- and out-of-transit centroids to rule out background eclipsing binaries. Morton et al. (2016) developed the vespa tool, which considers the likelihood that a transit event is caused by various astrophysical events, including a planet. The False Positive Probabilities (FPP) table⁵² provides the results of applying this tool to the KOIs in the DR25 catalog. It provides a probability that the observed signal is one of the known types of astrophysical false positives. The FPP table results are only reliable for high-S/N (MES ⪆ 10) candidates with no evidence that the transit occurs on a background source. For more information on this table see the associated documentation at the NASA Exoplanet Archive.

To robustly determine whether a KOI's signal originates from the target star, see the Astrophysical Positional Probabilities Table.⁵³ Using a more complete catalog of stars than the original Kepler Input Catalog (Brown et al. 2011), Bryson & Morton (2017) calculate the probability that the observed transit-like signal originates from the target star. Note that these positional probabilities are computed independent of the results from the Centroid Robovetter and are not used by the Robovetter.

To help understand the astrophysical reliability of the DR25 KOIs as a population, we have provided data to measure how well the Robovetter removes certain types of FPs. As part of the pixel-level transit injection efforts, we injected signals that mimic eclipsing binaries and background eclipsing binaries. Those that were recovered by the Kepler Pipeline can be used to measure the effectiveness of the Robovetter at removing this type of FP. A full description of these injections and an analysis of the Robovetter's effectiveness in detecting these signals can be found in Coughlin (2017b).

For those doing occurrence rates, another issue to consider is whether the measured size of the planet is correct. As discussed in Section 2.5, the stellar catalog (i.e., radii and temperatures) provided by Mathur et al. (2017) typically has errors of 27% for the stellar radii. Results from Gaia (Gaia Collaboration et al. 2016a, 2016b) are expected to fix many of the shortcomings of this catalog. Also, the dilution from an unaccounted-for bound or line-of-sight binary (Ciardi et al. 2015; Furlan et al. 2017) can cause planet radii to be larger than what is reported in the DR25 catalog. For occurrence rate calculations this dilution also has implications for the stars that have no observed planets because it means that the search did not extend to planet radii that are as small as the stellar catalog indicates. For this reason, any correction to the occurrence rates that might be applied needs to consider the effect on all searched stars, not just the planet hosts.

As mentioned in Section 1.2, for all of the Robovetter tests that require a phased light curve and model fit, we utilize two different detrendings and model fits (named ALT and DV). Both were also used by the DR24 Robovetter. Every test that is applied to the DV phased light curves is also applied to the ALT detrending, albeit with different thresholds for failure. Failing a test using either detrending results in the TCE being classified as an FP.

In the Kepler Pipeline, the DV module produces a harmonic-removed, median-detrended, phased flux light curve, along with a transit model fit (Wu et al. 2010; Jenkins 2017). However, the harmonic removal software is known to suppress or distort short-period (≲3 days) signals, causing short-period eclipsing binaries with visible secondaries to appear as transiting planets with no visible secondaries (Christiansen et al. 2013). It can also make variable stars with semicoherent variability, such as starspots or pulsations, appear as transit-like signals. As an alternative, we implement the ALT detrending method that utilizes the PDC time series light curves and the nonparametric penalized least-squares detrending method of Garcia (2010), which includes only the out-of-transit points when computing the filter. This ALT detrending technique is effective at accurately detrending short-period eclipsing binaries and variable stars, i.e., preserving their astrophysical signal. These ALT detrended light curves are phased and fit with a simple trapezoidal transit model.

If a TCE under examination is not the first one in a system, the Robovetter checks whether there exists a previous TCE with a similar period that was designated as an FP due to a stellar eclipse (see Appendix A.4). (Note that TCEs for a given system are ordered from highest MES to lowest MES, and the Robovetter runs on them in this order.) To compute whether two TCEs have the same period within a given statistical threshold, we employ the period matching criteria of Coughlin et al. (2014, see Equations (1)–(3)), σ_P, where higher values of σ_P indicate more significant period matches. We restate the equations here as

$$\begin{eqnarray}&&{\rm{\Delta }}P=\displaystyle \frac{{P}_{A}-{P}_{B}}{{P}_{A}}\end{eqnarray} \tag{ 10 }$$

$$\begin{eqnarray}&&{\rm{\Delta }}{P}^{{\prime} }=\mathrm{abs}({\rm{\Delta }}P-\mathrm{rint}({\rm{\Delta }}P))\end{eqnarray} \tag{ 11 }$$

$$\begin{eqnarray}&&{\sigma }_{P}=\sqrt{2}\cdot \mathrm{erfcinv}({\rm{\Delta }}{P}^{{\prime} }),\end{eqnarray} \tag{ 12 }$$

where P_A is the period of the shorter-period TCE, P_B is the period of the longer-period TCE, rint() rounds a number to the nearest integer, abs() yields the absolute value, and erfcinv() is the inverse complementary error function. We consider any value of σ_P > 3.5 to indicate significantly similar periods.

If the current TCE is (1) in a system that has a previous TCE dispositioned as an FP due to a stellar eclipse, (2) matches the previous TCE's period with σ_P > 3.5, and (3) is separated in phase from the previous TCE by at least 2.5 times the transit duration, then the current TCE is considered to be a secondary eclipse. In this case, it is designated as an FP and is classified into both the not-transit-like and stellar eclipse FP categories—a unique combination that can be used to identify secondary eclipses while still ensuring that they are not assigned KOI numbers (see Section 6). Note that since the Kepler Pipeline generally identifies TCEs in order of their S/N, from high to low, sometimes a TCE identified as a secondary can have a deeper depth than the primary, depending on their relative durations and shapes. Also note that it is possible that the periods of two TCEs will meet the period matching criteria but be different enough to have their relative phases shift significantly over the ≈4 yr mission duration. Thus, the potential secondary TCE is actually required to be separated in phase by at least 2.5 times the previous TCE's transit duration over the entire mission time frame in order to be labeled as a secondary. Also, the Kepler Pipeline will occasionally detect the secondary eclipse of an eclipsing binary at one-half, one-third, or some smaller integer fraction of the orbital period of the system, such that the epoch of the detected secondary coincides with that of the primary. Thus, when a non-1:1 period ratio is detected, we do not impose criterion 3, the phase separation requirement. Note that Equations (10)–(12) allow for integer period ratios.

A very large fraction of false-positive TCEs have light curves that do not resemble a detached transiting or eclipsing object. These include quasi-sinusoidal light curves from pulsating stars, starspots, and contact binaries, as well as more sporadic light curves due to instrumental artifacts. The first step in the catalog process is to determine whether each TCE is not transit-like. All transit-like obsTCEs are given KOI numbers, which are used to keep track of transit-like systems over multiple Kepler Pipeline runs. We employ a series of algorithmic tests to reliably identify these not-transit-like FP TCEs, as shown by the flowchart in Figure 16.

Zoom In Zoom Out Reset image size

A.3.1. The LPP Metric

A.3.2. Sine Wave Event Evaluation Test (SWEET)

A.3.3. TCE Chases

A.3.4. The Model-Shift Uniqueness Test

If a TCE under investigation is truly a PC, there should not be any other transit-like events in the folded light curve with a depth, duration, and period similar to the primary signal, in either the positive or negative flux directions, i.e., the transit event should be unique in the phased light curve. Many FPs are due to noisy, quasi-periodic signals (see Section 2) and thus are not unique in the phased light curve. In order to identify these cases, we developed a "model-shift uniqueness test" and used it extensively for identifying false positives in the Q1–Q12 (Rowe et al. 2015b), Q1–Q16 (Mullally et al. 2015), and DR24 (Coughlin et al. 2016) planet candidate catalogs.

See Section 3.2.2 of Rowe et al. (2015b) and page 23 of Coughlin (2017a) for figures and a detailed explanation of the "model-shift uniqueness test," as well as the publicly available code.⁵⁵ Briefly, after removing outliers, the best-fit model of the primary transit is used as a template to measure the best-fit depth at all other phases. The deepest event aside from the primary (pri) transit event is labeled as the secondary (sec) event, the next-deepest event is labeled as the tertiary (ter) event, and the most positive (pos) flux event (i.e., shows a flux brightening) is labeled as the positive event. The significances of these events (σ_Pri, σ_Sec, σ_Ter, and σ_Pos) are computed assuming white noise as determined by the standard deviation of the light-curve residuals. Also, the ratio of the red noise (at the timescale of the transit duration) to the white noise (F_Red) is computed by examining the standard deviation of the best-fit depths at phases outside of the primary and secondary events.

When examining all events among all TCEs, assuming Gaussian noise, the minimum threshold for an event to be considered statistically significant is given by

$$\begin{eqnarray}&&{\mathrm{FA}}_{1}=\sqrt{2}\cdot \mathrm{erfcinv}\left(\displaystyle \frac{{T}_{\mathrm{dur}}}{P\cdot {N}_{\mathrm{TCEs}}}\right),\end{eqnarray} \tag{ 13 }$$

where T_dur is the transit duration, P is the period, and N_TCEs is the number of TCEs examined. (The quantity P/T_dur represents the number of independent statistical tests for a single target.) When comparing two events from the same TCE, the minimum difference in their significances in order to be considered distinctly different is given by

$$\begin{eqnarray}&&{\mathrm{FA}}_{2}=\sqrt{2}\cdot \mathrm{erfcinv}\left(\displaystyle \frac{{T}_{\mathrm{dur}}}{P}\right).\end{eqnarray} \tag{ 14 }$$

We compute the following quantities to use as decision metrics:

$$\begin{eqnarray}&&{\mathrm{MS}}_{1}={\mathrm{FA}}_{1}-{\sigma }_{\mathrm{Pri}}/{F}_{\mathrm{Red}}\end{eqnarray} \tag{ 15 }$$

$$\begin{eqnarray}&&{\mathrm{MS}}_{2}={\mathrm{FA}}_{2}-({\sigma }_{\mathrm{Pri}}-{\sigma }_{\mathrm{Ter}})\end{eqnarray} \tag{ 16 }$$

$$\begin{eqnarray}&&{\mathrm{MS}}_{3}={\mathrm{FA}}_{2}-({\sigma }_{\mathrm{Pri}}-{\sigma }_{\mathrm{Pos}}).\end{eqnarray} \tag{ 17 }$$

In the Robovetter, we disposition a TCE as a not-transit-like FP if either MS₁ > 1.0, MS₂ > 2.0, or MS₃ > 4.0 in the DV detrending, or if either MS₁ > −3.0, MS₂ > 1.0, or MS₃ > 1.0 in the ALT detrending. These criteria ensure that the primary event is statistically significant when compared to the systematic noise level of the light curve, the tertiary event, and the positive event, respectively. We also fail TCEs as not-transit-like if σ_Pri exactly equals zero in both the DV and ALT detrendings. A value of zero indicates that the fit failed for both detrendings and suggests that something is fundamentally flawed with the TCE.

A.3.5. Dominated by Single Event

The depths of individual transits of planet candidates should be equal to each other, and thus assuming constant noise levels, the S/N of individual transits should be nearly equivalent as well. In contrast, most of the long-period FPs that result from three or more equidistant systematic events are dominated in S/N by one event. The Kepler Pipeline measures detection significance via the MES, which is calculated by combining the SES of all the individual events that compose the TCE—both the MES and SES are measures of S/N. Assuming that all individual events have equal SES values,

$$\begin{eqnarray}&&\mathrm{MES}=\sqrt{{N}_{\mathrm{Trans}}}\cdot \mathrm{SES},\end{eqnarray} \tag{ 18 }$$

where N_Trans is the number of transit events that compose the TCE. Thus, SES/MES = 0.577 for a TCE with three transits, and less for a greater number of transits. If the largest SES value of a TCE's transit events, SES_Max, divided by the MES is much larger than 0.577 (regardless of the number of transits), this indicates that one of the individual events dominates when calculating the S/N.

In the Robovetter, for TCEs with periods greater than 90 days, if SES_Max/MES > 0.8, it is dispositioned as a not-transit-like FP. The period cutoff of 90 days is applied because short-period TCEs can have a large number of individual transit events, which dramatically increases the chance of one event coinciding with a large systematic feature, thus producing a large SES_Max/MES value despite being a valid planetary signal.

A.3.6. Previous TCE with Same Period

A.3.7. Individual Transit Metrics

A new approach implemented in DR25 is to examine individual transit events for each TCE and determine whether they are transit-like. After rejecting these "bad" transit events, we check if either

• 1.  
  there are less than 3 "good" events left; or
• 2.  
  the recomputed MES using only "good" events is <7.1.

If either of these conditions is met, then the TCE is failed as not-transit-like. This is in line with the Kepler mission requirement of at least three valid transit events with an MES ≥ 7.1 in order to generate a TCE. In the following subsections we list the various tests we apply to each individual transit event.

A.3.7.1. Rubble—Missing Data

A number of TCEs from the Kepler Pipeline are based on transit events that are missing a significant amount of data either in transit or just before and/or after. These tend to be false positives that are triggering on edges of gaps, or cases where a large amount of data has been removed and a TCE is being created from the residuals of previous TCEs in the system. We thus devised the Rubble metric to clean up these fragments from the TCE list. The Rubble value for each individual transit is computed by dividing the number of Kepler cadences that are available in the DV time series by the number of cadences expected across two transit durations given Kepler's regular 29.42-minute cadence and the transit duration provided by the DV fit. If the Rubble value for the transit falls below threshold, then that transit is not counted as a valid transit. We adopted a threshold value of 0.75 to generate the DR25 KOI Catalog.

A.3.7.2. Marshall—Transit Shape

In the DR24 KOI Catalog, Coughlin et al. (2016) used the Marshall algorithm (Mullally et al. 2016) to identify and reject false-alarm TCEs caused by short-period transients in the data. Marshall fits the proposed transit with models of various transients and uses a Bayesian Information Criterion (BIC) to decide which model is the best explanation for the data. Simulations in Mullally et al. (2016) showed that Marshall was 95% complete for TCEs with periods >150 days and correctly rejected 66% of simulated artifact events. The limit on Marshall's effectiveness at eliminating false alarms was that it used a parabola to describe the out-of-transit flux, which failed to capture much of the real observed stellar variability. To ensure high completeness, Marshall was tuned to prevent a variable continuum from causing true transits to be rejected, at the cost of a lower effectiveness.

For the DR25 KOI catalog, we use a Gaussian process approach (GP; Rasmussen & Williams 2006) to provide an improved continuum model and increase our effectiveness, while maintaining our high completeness. Briefly, our approach aims to model the covariance in the light curve to better fit the trends in our data. A similar approach was used by Foreman-Mackey et al. (2016) to model single transits due to very long period planets (P > 1000 days).

Our procedure is as follows. For each individual proposed transit event, we select a snippet of PDC data 30 times the reported transit duration centered on the event. Where the event happens near the start (or end) of a quarter, we take a snippet of similar length anchored at the start (or end) of the quarter. We use the George package (Ambikasaran et al. 2014) to fit the covariance of the out-of-transit flux with an exponential squared function, $\mathrm{Cov}{(\delta t)=A\exp (\delta t/{\ell })}^{2}$, where A and ℓ are tunable parameters.

We next fit four models to the entire snippet:

$$\begin{eqnarray}&&G(t| A,{\ell })+{y}_{0}\\ &&G(t| A,{\ell })+{y}_{0}+S(t)\\ &&G(t| A,{\ell })+{y}_{0}+S(t)(1-\exp \beta t)\\ &&G(t| A,{\ell })+{y}_{0}+S(t-\tau /2)-S(t+\tau /2),\end{eqnarray} \tag{ 19 }$$

where G is the GP model with the tunable parameters held fixed to those found earlier, and y₀ is a constant offset. S(t) is given by

$$\begin{eqnarray}&&S(t)=\displaystyle \frac{d}{1+{e}^{-\gamma (t-{t}_{0})}},\end{eqnarray} \tag{ 20 }$$

where d and t₀ are tunable parameters and γ is a positive constant. This function, known as a sigmoid (or logistic) function, has asymptotes of 0 for t ≪ t₀ and d for t ≫ t₀. The function transitions quickly, but smoothly, between the two states near t = t₀, where it takes on a value of d/2.

By using a sigmoid and avoiding the discontinuities present in the models used by the original Marshall algorithm (Mullally et al. 2016), we can use the L-BFGS-B algorithm (Byrd et al. 1995) available in the Scipy package⁵⁶ instead of the less robust Nelder–Mead.

The second function in Equation (19) models a discrete jump in the data. We fit this model seeded with a negative-going dip at the predicted time of ingress and also with a positive-going spike at the predicted egress, as we see both types of features in Kepler data. The third model fits an SPSD event, probably caused by a cosmic-ray hit on the detector. The last model approximates a box transit. By varying the parameter γ, we could in principle model transit ingress and egress, but we find that that extra degree of freedom is not necessary to fit the low-S/N events of most concern.

For each transit the Marshall method returns the BIC score, the preferred model, and the difference between the BIC scores of the preferred model and the sigmoid box fit. A transit is considered sufficiently bad when this difference (also known as the Marshall score) exceeds a particular threshold, as with the original Marshall algorithm. However, in a few cases the GP fails and yields extremely large, unbelievable BIC values. In these cases the transit is set to always pass. Also, for low-MES transits, the expected SES of a transit is sufficiently low that Marshall will be unable to distinguish between the "no transit" model and a low-S/N transit. Because of this, the Robovetter declares that a specific transit is not valid if all of the following criteria are met:

• 1.  
  The BIC score of the best-fitting nontransit model is at least 10 lower than the BIC of the transit model.
• 2.  
  The BIC score of the best-fitting nontransit model is less than 1.0E6.
• 3.  
  Either $\mathrm{MES}/\sqrt{{N}_{\mathrm{RealTrans}}}\gt 4.0$ or the lowest BIC model is for the constant offset model.

Note that N_RealTrans is the total number of observed transit events for the TCE. The Marshall code used for the DR25 KOI catalog is available on sourceforge.⁵⁷

A.3.7.3. Chases—SES Artifacts

The Chases metric was developed to chase down non-transit-like events on long-period, low-MES TCEs. Qualitatively, the metric mimics the human vetting preference to classify a TCE as a PC when individual transit events "stand out" as a unique, transit-like signal from a visual inspection of the Kepler flux time series data. In order to quantify this human vetting preference, we developed the Chases algorithm. Chases uses the SES time series generated by the TPS module of the Kepler Pipeline (Jenkins 2017). The SES time series measures the significance of a transit signal centered on every cadence. Details of calculating the SES time series are given in Jenkins et al. (2002), and illustrative examples are given in Tenenbaum et al. (2012). A transit produces a peak in the SES time series (as do systematic signals). TPS searches the SES time series for equally spaced peaks indicative of a series of transits. The series of individual peaks in the SES time series are combined to form the MES employed as the primary threshold for detecting a transit signal (Jenkins et al. 2002; Twicken et al. 2016; Jenkins 2017).

The Chases metric quantifies how well the SES peaks contributing to a TCE approximate the expected shape and significance (relative to neighboring data) of a bona fide transit signal. Figure 17 shows the detrended flux time series (top panel) and the corresponding SES time series (bottom panel) for a clear single transit event contributing to the TCE detection of K03900.01 on target KIC 11911580. The flux time series, with a very clear decrement during in-transit cadences (orange points), has the archetypal SES time series of a strong central peak with two low-amplitude, symmetric side troughs (caused by the way TPS uses wavelets to modify the model transits when calculating the SES; see Jenkins 2017).

Zoom In Zoom Out Reset image size

The Chases metric for an individual transit event is formulated by identifying the maximum SES value for cadences in transit, SES_max (in Figure 17, SES_max ≈ 20). Next, excluding cadences within 1.5τ_dur of midtransit (to avoid the symmetric side troughs), where τ_dur is the detected transit duration, the SES time series is searched for Δ_t, the temporally closest feature to midtransit in the absolute value of the SES time series, $| \mathrm{SES}|$. A feature is defined as when $| \mathrm{SES}| \,\gt f\,{\mathrm{SES}}_{\max }$, where f represents a tunable fraction of the peak in the SES time series. Finally, we define a maximum window Δ_tmax = P_orb/10 with which to search for a comparable peak in $| \mathrm{SES}|$, and we form the final Chases metric for an individual transit event as C_i = min(Δ_t, Δ_tmax)/Δ_tmax.

A value of C_i = 1 indicates that there is no comparable peak/trough in the SES time series within f of SES_max over the interval Δ_tmax of the transit signal. Thus, Ch_i = 1 is consistent with a unique, transit-like signal. A value of Ch_i ≈ 0 indicates that a comparable strength feature is present in the SES time series temporally close to the transit event and is consistent with the human vetting tendency to dismiss such signals as spurious. Figure 18 shows an example of a spurious TCE detection on the target KIC 11449918. The target is on a detector suffering from elevated levels of the "rolling-band" image artifacts as described in Appendix A.3.7.4. The neighboring peak of comparable strength in the SES time series would result in Ch_i ≈ 0 for this individual transit event. The Chases metric is also sensitive to the shape of the transit signal as illustrated in Figure 19. The SPSD shown in Figure 19 is a spurious instrumental signal with an asymmetric shape. Because Chases uses the absolute value of the SES, Ch_i ≈ 0 for these types of events.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

For each TCE with five or fewer transit events contributing to the signal, Ch_i is calculated for every transit event. With more than five transit events, the individual transit events are not expected to be statistically significant, and the assumptions of the Chases metric no longer apply. The individual transit event Ch_i values were used to recalculate the MES (see Appendix A.3.7). Transit events with Ch_i < 0.01 were excluded from the Robovetter's MES calculation.

A.3.7.4. Skye—Image Artifacts Clustered by Skygroup

As discussed in 2.1, there are a number of TCEs caused by rolling-band image artifacts. These artifacts are caused by a spatial pattern in the CCD bias level that moves across the chip in response to changes in the temperature of the chip (for more details, see Van Cleve & Caldwell 2009). If a number of individual transit events from TCEs on different targets but the same skygroup (region of the sky that falls on the same CCD each quarter) occur at the same time, they are very likely systematic in origin. The metric called Skye looks for an excess in the number of individual events occurring at the same time in the same skygroup. If an excess is identified, we consider these events to be caused by artifacts.

More specifically, for each skygroup we bin the individual events into 1.0-day bins. We only use those obsTCEs with periods greater than 45 days (∼half a Kepler quarter) for each skygroup. The reason for the period cut is that the long-period obsTCEs are likely to be affected by rolling-band systematics, but the short-period ones are not. Including shorter-period TCEs would dramatically increase the number of individual transits and would reduce the significance of the anomalous peaks. See Figure 20 for an example of the anomalous peaks seen in some skygroups when the data are binned in this way.

Zoom In Zoom Out Reset image size

To determine which events are anomalous, for each skygroup we compute the average rate (R) of transits, by dividing the overall number of individual transit events in the skygroup by the number of 1.0-day bins. Assuming that the majority of transits are randomly distributed in time and utilizing Poisson counting statistics, any peaks greater than

$$\begin{eqnarray}&&\mathrm{threshold}=R+N\cdot \sqrt{R}\end{eqnarray} \tag{ 21 }$$

are statistically significant and indicative of temporal clustering, given a chosen value for N. We choose a value of N = 3.0 and robustly determine the rate for each skygroup by first computing the threshold using all the bins and then iteratively rejecting all bins with a height greater than threshold and recomputing threshold until it converges and does not change with further iterations.

For each skygroup and its threshold, we identify the individual times of transit for TCEs belonging to the skygroup that fall in bins that are above the threshold. We assign Skye a value of 1.0 to these individual transits to indicate that they are bad transits. The Skye value for all other transit times is set to zero. The Skye code is publicly available on github.⁵⁸

A.3.7.5. Zuma—Negative Significance

A valid transit-like TCE should be composed of individual events that correspond to flux decrements. If any event instead shows an increase of flux, then that event is suspect. We thus designate any individual transit event with SES < 0 as "bad."

A.3.7.6. Tracker—Ephemeris Slip

After the TPS module of the Kepler Pipeline detects a TCE, it is sent to DV to be fit with a full transit model. DV allows the period and epoch to vary when fitting in order to provide as accurate a fit as possible. Sometimes the TPS ephemeris and DV ephemeris can end up significantly different. When this occurs, it indicates that the underlying data are not transit-like and the TCE is likely due to quasi-sinusoidal systematics, which cause the ephemeris to wander when fitting.

Tracker measures (i.e., keeps track of) the time difference between the TPS and DV linear ephemerides in units of the TCE's duration for each transit. When Tracker is greater than 0.5T_dur for any transit, we designate the transit as bad.

A.3.8. Fraction of Gapped Events

A.3.9. No Data Available

If a TCE is deemed transit-like by passing all of the tests presented in Appendix A.3 on both detrendings, it is given a KOI number (see flowchart in Figure 16). However, many of these KOIs are FPs owing to eclipsing binaries and contamination from nearby variable stars. We employ a series of robotic tests to detect systems that are due to stellar companions, as shown by the flowchart in Figure 21.

Zoom In Zoom Out Reset image size

A.4.1. Secondary Eclipse

One of the most common methods to detect a stellar system is the presence of a significant secondary in the light curve. With the exception of some hot-Jupiter-type planets (e.g., HAT-P-7; Borucki et al. 2009), the visibility of a secondary eclipse in Kepler data is a telltale sign of a stellar eclipsing binary.

A.4.1.1. Subsequent TCE with Same Period

Once the Kepler Pipeline detects a TCE in a given system, it removes the data corresponding to this event and re-searches the light curve. It is thus able to detect the secondary eclipse of an eclipsing binary as a subsequent TCE, which will have the same period, but different epoch, as the primary TCE. Thus, using Equations (10)–(12), the Robovetter dispositions a TCE as a stellar system FP if its period matches a subsequent TCE within the specified tolerance (σ_P > 3.25) and they are separated in phase by at least 2.5 times the transit duration. For clarity, we note again that it is sometimes possible that the periods of two TCEs will meet the period matching criteria but be different enough to have their epochs shift significantly in phase over the ∼4 yr mission duration. The phase separation requirement must be upheld over the entire mission duration in order to disposition the TCE as an FP due to a stellar eclipse.

Occasionally the Kepler Pipeline will detect the secondary eclipse of an eclipsing binary at one-half, one-third, or some smaller integer fraction of the orbital period of the system. In these cases, the epoch of the TCE corresponding to the secondary will overlap with that of the primary. These cases are accounted for by not requiring a phase separation of at least 2.5 transit durations when a period ratio other than unity is detected. (Note that Equations (10)–(12) allow for integer period ratios.) While this approach will likely classify any multiplanet system in an exact 2:1 orbital resonance as an FP due to a stellar eclipse, in practice this is nonexistent. Exact 2:1 orbital resonances, where "exact" means that the period ratio is close enough to 2.0 over the ∼4 yr mission duration to avoid any drift in relative epoch, appear to be extremely rare (Fabrycky et al. 2014). Also, they might produce strong TTVs, which would likely preclude their detection. The Kepler Pipeline employs a strictly linear ephemeris when searching for TCEs, and thus while planets with mild TTVs (e.g., deviations from a linear ephemeris less than the transit duration) are often detected, planets with strong TTVs (e.g., deviations from a linear ephemeris greater than the transit duration) are often not detected.

A.4.1.2. Secondary Detected in Light Curve

There are many cases when a secondary eclipse does not produce its own TCE, most often when its MES is below the Kepler Pipeline detection threshold of 7.1. The model-shift uniqueness test, discussed in Appendix A.3.4, is well suited to automatically detect secondary eclipses in the phased light curve, as it searches for the next two deepest events aside from the primary event. It is thus able to detect the best-candidate secondary eclipse in the light curve and assess its significance. We compute the following quantities to use as secondary detection metrics:

$$\begin{eqnarray}&&{\mathrm{MS}}_{4}={\sigma }_{\mathrm{Sec}}/{F}_{\mathrm{Red}}-{\mathrm{FA}}_{1}\end{eqnarray} \tag{ 22 }$$

$$\begin{eqnarray}&&{\mathrm{MS}}_{5}=({\sigma }_{\mathrm{Sec}}-{\sigma }_{\mathrm{Ter}})-{\mathrm{FA}}_{2}\end{eqnarray} \tag{ 23 }$$

$$\begin{eqnarray}&&{\mathrm{MS}}_{6}=({\sigma }_{\mathrm{Sec}}-{\sigma }_{\mathrm{Pos}})-{\mathrm{FA}}_{2}.\end{eqnarray} \tag{ 24 }$$

Recall that σ indicates a significance and was defined in Appendix A.3.4. If MS₄ > 1, MS₅ > 0, and MS₆ > 0, in either the DV or alternate detrendings, the Robovetter dispositions the TCE as a stellar system FP. These criteria ensure that the secondary event is statistically significant when compared to the systematic noise level of the light curve, the tertiary event, and the positive event, respectively.

A.4.1.3. Candidates with Stellar Eclipses

There are two exceptions when the above-mentioned conditions are met but the Robovetter does not designate the TCE as an FP. First, if the primary and secondary widths and depths are statistically indistinguishable and the secondary is located at phase 0.5, then it is possible that the TCE is a PC that has been detected at twice the true orbital period. Thus, the Robovetter labels a TCE with a stellar eclipse as a PC when σ_Pri − σ_Sec < FA₂ and the phase of the secondary is within 1/4 of the primary transit's duration of phase 0.5. Second, hot Jupiter PCs can have detectable secondary eclipses owing to planetary occultations via reflected light and thermal emission (Christiansen et al. 2010; Coughlin & López-Morales 2012). Thus, a TCE with a detected stellar eclipse is labeled as a PC with the stellar eclipse flag (in order to facilitate the identification of hot Jupiter occultations) when the geometric albedo required to produce the observed secondary eclipse is less than 1.0, the planetary radius is less than 30 R_⊕, the depth of the secondary is less than 10% of the primary, and the impact parameter is less than 0.95. The additional criteria beyond the albedo criterion are needed to ensure that this test is only applied to potentially valid planets and not grazing eclipsing binaries. We calculate the geometric albedo by using the stellar mass, radius, and effective temperature from the DR25 stellar catalog (Mathur et al. 2017) and the values of the period and radius ratio from the original DV fits.

A.4.1.4. Odd/Even Depth Difference

If the primary and secondary eclipses of eclipsing binaries are similar in depth and the secondary is located near phase 0.5, the Kepler Pipeline may detect them as a single TCE at half the true orbital period of the eclipsing binary. In these cases, if the primary and secondary depths are dissimilar enough, it is possible to detect it as an FP by comparing the depths of the odd- and even-numbered transit events and their associated uncertainties, via the following statistic:

$$\begin{eqnarray}&&{\sigma }_{\mathrm{OE}}=\displaystyle \frac{\mathrm{abs}({d}_{\mathrm{odd}}-{d}_{\mathrm{even}})}{\sqrt{{\sigma }_{\mathrm{odd}}^{2}+{\sigma }_{\mathrm{even}}^{2}}},\end{eqnarray} \tag{ 25 }$$

where d_odd is the measured depth using the odd-numbered transits, with associated uncertainty σ_odd, d_even is the measured depth using the even-numbered transits, with associated uncertainty σ_even, and abs() returns the absolute value.

We use two different methods to compute d_odd, σ_odd, d_even, σ_even, and thus σ_OE, for both the DV and ALT detrending. For the first method, the depths are computed by taking the median of all the points near the center of all transits, and the uncertainty is the standard deviation of those points, both using only the odd- or even-numbered transits. For the ALT detrending with a trapezoidal fit, we use all points that lie within ±30 minutes of the central time of transit, as well as any other points within the in-transit flat portion of the trapezoidal fit. For the DV detrending, we use all points within ±30 minutes of the central time of transit. (This threshold corresponds to the long-cadence integration time of the Kepler spacecraft. Including points farther away from the central time of transit degrades the accuracy and precision of the test.) If σ_OE > 1.1 for either the DV or ALT detrending, then the TCE is labeled as an FP due to a secondary eclipse and given the DEPTH_ODDEVEN_DV and/or DEPTH_ODDEVEN_ALT flag(s). The value of 1.1 was empirically derived using manual checks and transit injection. This method is very robust to outliers and systematics but not extremely sensitive, as it does not take into account the full transit shape to measure the depth.

The second method measures the depths and uncertainties by running the model-shift test separately on the portions of the light curve within half a phase of the odd- and even-numbered transits. The model-shift test measures the depths and associated uncertainties using the entire transit model and taking into account the measured noise level of the entire light curve. This method is more sensitive to small odd/even differences but also more sensitive to outliers and light-curve systematics compared to the above method. If σ_OE > 11.2 for the DV detrending, or >19.8 for the ALT detrending, then the TCE is labeled as an FP due to a stellar eclipse and given the MOD_ODDEVEN_DV and/or MOD_ODDEVEN_ALT flag(s). The thresholds of 11.2 and 19.8 were empirically derived using manual checks and transit injection. This method is susceptible to outliers and systematics (and why the thresholds are set fairly high) but can also detect small yet significant odd/even differences that the other method listed above cannot.

A.4.2. Out-of-eclipse Variability

A.4.3. V-shape Metric

A.5.1. Centroid Robovetter

A.5.2. Ghost Diagnostic

Another method for detecting FPs owing to contamination is to compare the ephemerides (periods and epochs) of TCEs to each other, as well as other known variable sources in the Kepler field. If two targets have the same ephemeris within a specified tolerance, then at least one of them is an FP due to contamination. Coughlin et al. (2014) used Q1–Q12 data to compare the ephemerides of KOIs to each other and eclipsing binaries known from both Kepler- and ground-based observations. They identified over 600 FPs via ephemeris matching, of which over 100 were not known as FPs via other methods. They also identified four main mechanisms of contamination. The results of Coughlin et al. (2014) were incorporated in Rowe et al. (2015b, see Section 3.3), and with some small modifications to Mullally et al. (2015, see Section 5.3) and Coughlin et al. (2016).

We modified the matching criteria used in previous catalogs to improve performance. We use the results of the transit injection run (Section 2.3) to measure the ability of the original DV fits by the Kepler Pipeline to recover period and epoch as a function of period. (Note that while the DV fits do produce an error on the measured period, it is not a robustly measured error, and thus not sufficient for our purposes.) In Figure 22 we show, in the top two panels, the difference in the injected and recovered period and epoch, as a function of the injected period. The bottom panels show the measured standard deviation of the difference as a function of period, in linear and logarithmic space, respectively. The red line is the result of a best-fit power law.

Zoom In Zoom Out Reset image size

When comparing two objects, A and B, where A is defined to have the shorter period, the new matching metrics we use, S_P and S_T for period and epoch, respectively, are

$$\begin{eqnarray}&&{S}_{P}=\displaystyle \frac{| {P}_{r}\cdot {P}_{A}-{P}_{B}| }{\sqrt{2}\cdot {\sigma }_{P}({P}_{A})}\end{eqnarray} \tag{ 26 }$$

$$\begin{eqnarray}&&{S}_{T}=\displaystyle \frac{| {T}_{A}-{T}_{B}-{T}_{r}\cdot {P}_{A}| }{\sqrt{2}\cdot {\sigma }_{T}({P}_{A})},\end{eqnarray} \tag{ 27 }$$

where P_A and P_B are the periods of objects A and B, T_A and T_B are similarly the epochs of objects A and B, and σ_P(P_A) and σ_T(P_A) are the errors in period and epoch, given period P_A, derived from the best-fit power law to the standard deviation of the injected versus recovered periods and epochs, respectively. The period ratio, P_r, and epoch ratio, T_r, are defined by

$$\begin{eqnarray}&&{P}_{r}=\mathrm{rint}\left(\displaystyle \frac{{P}_{B}}{{P}_{A}}\right)\end{eqnarray} \tag{ 28 }$$

$$\begin{eqnarray}&&{T}_{r}=\mathrm{rint}\left(\displaystyle \frac{{T}_{A}-{T}_{B}}{{P}_{A}}\right),\end{eqnarray} \tag{ 29 }$$

where rint() rounds a number to the nearest integer. Thus, a perfect match has S_P = 0 and S_T = 0, with worse matches having increasingly larger values of S_P and S_T.

We consider matches with S_P < 5 and S_T < 5, with period ratios of 50 or less (P_r < 50), to be statistically significant enough to constitute a match. We also require the following:

• 1.  
  The two objects do not have the same KIC ID.
• 2.  
  The two objects satisfy at least one of the following conditions:

  • (a)  
    A separation distance less than d_max arcseconds, where

    $$\begin{eqnarray}&&{d}_{\max }(^{\prime\prime} )=55\cdot \sqrt{{10}^{6}\cdot {10}^{-0.4\cdot {m}_{\mathrm{kep}}}+1}\end{eqnarray} \tag{ 30 }$$

    with the Kepler magnitude of the brighter source being used for m_kep.
  • (b)  
    Located on opposite sides of the field-of-view center, but equidistant from the center to within a 100'' (25 pixel) tolerance.
  • (c)  
    Located on the same CCD module and within 5 pixels of the same column value in any of the four quarters.
  • (d)  
    Located on the same CCD module and within 5 pixels of the same row and column value in any of the four quarters.

Criterion 1 ensures that no star is ever matched to itself. Criterion 2a is a semiempirically determined formula derived to account for direct PRF contamination and reflection off the field flattener lens, assuming that the average wings of a Kepler point-spread function can be approximated by a Lorentzian distribution. The formula allows for any two stars to match within a generous 55'' range, but it allows for bright stars to match to larger distances, e.g., a 10th mag star could match up to 550'' away, and a 5th mag star could match up to 5500'' away. Criterion 2b accounts for antipodal reflection off the Schmidt Corrector. Criterion 2c accounts for the column anomaly (see Section 3.5 of Coughlin et al. 2016), and criterion 2d accounts for video cross talk.

In this Q1–Q17 DR25 catalog, we match the ephemerides of all Q1–Q17 DR25 TCEs (Twicken et al. 2016), including rogue TCEs, to the following sources:

• 1.  
  Themselves.
• 2.  
  The list of 8826 KOIs from the NASA Exoplanet Archive cumulative KOI table after the closure of the Q1–Q17 DR24 table and publication of the last catalog (Coughlin et al. 2016).
• 3.  
  The Kepler Eclipsing Binary Working Group list of 2605 "true" eclipsing binaries found with Kepler data as of 2016 October 13 (Prša et al. 2011; Slawson et al. 2011; Kirk et al. 2016).
• 4.  
  J. M. Kreiner's up-to-date database of ephemerides of ground-based eclipsing binaries as of 2016 October 13 (Kreiner 2004).
• 5.  
  Ground-based eclipsing binaries found via the TrES survey (Devor et al. 2008).
• 6.  
  The General Catalog of Variable Stars (Samus et al. 2009) list of all known ground-based variable stars, published 2016 October 05.

The ephemeris matching code used for the DR25 catalog is publicly available on github.⁶⁰

Via ephemeris matching, we identify 1859 Q1–Q17 DR25 TCEs as FPs. Of these, 106 were identified as FPs only owing to ephemeris matching. We list all 1859 TCEs in Table 8, as this information is valuable for studying contamination in the Kepler field. In this table each TCE is identified by its KIC ID and planet number, separated by a dash. We also list in Table 8 each TCE's most likely parent, the period ratio between child and parent (P_rat), the distance between the child and parent in arcseconds, the offset in row and column between the child and parent in pixels (ΔRow and ΔCol), the magnitude of the parent (m_Kep), the difference in magnitude between the child and parent (ΔMag), the depth ratio of the child and parent (D_rat), the mechanism of contamination, and a flag to designate unique situations. In Figure 23 we plot the location of each FP TCE and its most likely parent, connected by a solid line. TCEs are represented by filled black circles, KOIs are represented by filled green circles, eclipsing binaries found by Kepler are represented by filled red circles, eclipsing binaries discovered from the ground are represented by filled blue circles, and TCEs due to a common systematic are represented by open black circles. The Kepler magnitude of each star is shown via a scaled point size. Most parent–child pairs are so close together that the line connecting them is not easily visible on the scale of the plot.

Zoom In Zoom Out Reset image size

Since Kepler does not observe every star in its field of view, it can often be the case that a match is found between two objects, but given their relative magnitude, distance, and depths, it is clear that neither is the parent of the other, so these are classified as "bastards" (Coughlin et al. 2014). To identify the bastards due to direct PRF contamination, we performed a robust fit of the Kepler PRF model described by Equations (9) and (10) of Coughlin et al. (2014) to the depth ratio, magnitude difference, and distance between each object identified as due to direct PRF contamination and its most likely parent. After iteratively rejecting outliers greater than 4.0 times the standard deviation, the fit converged with values of α = 693 and γ = 0358. Outliers greater than 4.0 times the standard deviation of the final iteration, with these resulting fit parameters, were labeled as bastards. For the mechanism of column anomaly and reflection, if the depth ratio of the two objects is between 0.01 and 100, then it is labeled as a bastard, as these mechanisms should produce depth ratios of at least 1E-3 or 1E3. All bastards are identified with a flag of 1 in Table 8. Additionally, it can sometimes be the case that objects are matched via the column anomaly but are on different outputs of the same module—these cases likely involve the column anomaly working in conjunction with cross talk, and thus are complicated and given a flag of 2 in Table 8. Finally, a flag of 3 indicates a combination of flags 1 and 2.

There are a couple tests that the Robovetter performs that do not influence the disposition of a TCE. While failing one of these tests indicates a likely FP, it is not reliable enough to declare a TCE an FP. Instead, TCEs that fail these tests are given information-only flags (see Section B) as a way to notify users that a manual inspection of the TCE and the Robovetter results is likely warranted.

A.7.1. Planet in Star

A.7.2. Seasonal Depth Differences

Due to the Kepler spacecraft's rotation every ≈90 days, each target and the surrounding stars will fall on a new CCD every quarter and return to the same CCD once every four quarters. All of the quarters that correspond to the same CCD are labeled as being in a given season (e.g., Q2, Q6, Q10, and Q14 belong to Season 0, Q3, Q7, Q11, and Q15 belong to Season 1, etc.; Thompson et al. 2016b). The shape and size of the optimal aperture for a given star are seasonally dependent and can change significantly from season to season. As a result, a target will have differing amounts of third light in its optimal aperture from nearby stars. If the source of the signal that triggers a TCE is not from the target star, but rather from another source (as just discussed in Appendix A.5 and Appendix A.6), the level of contamination, and thus observed depth of the TCE, will have significant seasonal variation. Observation of seasonal depth differences is usually a good indication that the target is contaminated and a centroid offset is likely. However, depth differences can also arise when the signal is truly coming from the target but significant third light exists in the aperture and the seasonal variations are not sufficiently corrected.

In order to automatically detect seasonal depth differences, if a TCE has been dispositioned as transit-like (the NT flag is not set), we measure the depth and associated error of the primary event in each season utilizing the first method described in the second paragraph of Appendix A.4.1.4, i.e., we compute the median and standard deviation of all the points within ±15 minutes of the center of transit. We then obtain an average depth over all seasons, D_a, by computing the mean of the depths of all four seasons.

The significance of the seasonal depth differences, S_Diff, is then computed via

$$\begin{eqnarray}&&{S}_{\mathrm{Diff}}=\displaystyle \frac{{\displaystyle \sum }_{n=0}^{3}\,| {D}_{n}-{D}_{a}| }{\sqrt{{\displaystyle \sum }_{n=0}^{3}\,{\sigma }_{n}^{2}+N\cdot {\sigma }_{a}^{2}}},\end{eqnarray} \tag{ 31 }$$

where n denotes a particular season (0, 1, 2, or 3), N is the total number of seasons with a measured depth and uncertainty, D_n is the measured depth in a given season, σ_n is the measured error on the depth in a given season, D_a is the measured averaged depth, and σ_a is the measured error of the average depth, given by

$$\begin{eqnarray}&&{\sigma }_{a}=\displaystyle \frac{\sqrt{{\displaystyle \sum }_{n=0}^{3}\,{\sigma }_{n}^{2}}}{N}.\end{eqnarray} \tag{ 32 }$$

For either the DV or ALT detrending, if S_Diff > 3.6, then the TCE is flagged as having significant seasonal depth differences via the flag SEASONAL_DEPTH_(ALT∣DV).

A.7.3. Period Aliasing

In some cases, the Kepler Pipeline detects a signal (and produces a TCE) that is at an integer multiple of the signal's true period. In most cases, this is due to the presence of seasonal depth differences, as the pipeline ends up only locking onto events in the quarters with the strongest (deepest) signal. While this usually indicates an FP due to a centroid offset, as discussed in Appendix A.7.2, it is not a definitive measure. Also, the pipeline will detect real planets with significant TTVs at longer (near integer multiple) periods.

In order to detect a period alias, we utilize the model-shift results—if the TCE's period is an integer multiple of the signal's true period, then several, equally spaced events should be visible in the phased light curve. If the TCE has been dispositioned as transit-like (the NT flag is not set), the Robovetter first checks whether the model-shift test detected significant secondary and tertiary events, by ensuring that σ_Sec/F_Red > FA₁ and σ_Ter/F_Red > FA₁. If so, the phases of the secondary and tertiary events, ϕ_Sec and ϕ_Ter, are then expressed as the absolute value of their distance in phase from the primary event, i.e., constrained to be between 0.0 and 0.5. (For example, if secondary and tertiary events were initially detected at phases of 0.1 and 0.7, then ϕ_Sec = 0.1 and ϕ_Ter = 0.3.) If period aliasing is present, then ϕ_Sec and ϕ_Ter should be ≈n/N, where N is the integer multiple of the true signal that the pipeline detected it at, and n is an integer between 1 and N − 1 that is different for the secondary and tertiary events. (For example, in the case of ϕ_Sec = 0.1 and ϕ_Ter = 0.3, this implies N = 10, n = 1 for ϕ_Sec, and n = 3 for ϕ_Ter.)

We derive metrics to measure how close ϕ_Sec and ϕ_Ter each are to an exact integer period alias, called S_Sec and S_Ter. Specifically,

$$\begin{eqnarray}{S}_{\mathrm{Sec}} & = & \sqrt{2}\cdot \mathrm{erfcinv}\left(\left|\displaystyle \frac{1}{{\phi }_{\mathrm{Sec}}}-\mathrm{rint}\left(\displaystyle \frac{1}{{\phi }_{\mathrm{Sec}}}\right)\right|\right)\\ {S}_{\mathrm{Ter}} & = & \sqrt{2}\cdot \mathrm{erfcinv}\left(\left|\displaystyle \frac{1}{{\phi }_{\mathrm{Ter}}}-\mathrm{rint}\left(\displaystyle \frac{1}{{\phi }_{\mathrm{Ter}}}\right)\right|\right),\end{eqnarray} \tag{ 33 }$$

where erfcinv() is the inverse complementary error function and rint() rounds a number to the nearest integer. The higher the values of S_Sec and S_Ter, the more closely the measured phases of the significant secondary and tertiary events correspond to an integer period ratio. These computations are performed independently for the DV and ALT detrendings. If S_Sec > 2.0 and S_Ter > 2.0, for either detrending, the Robovetter considers a period alias detected, and the TCE is flagged as PERIOD_ALIAS_(ALT∣DV).