The Active Asteroids Citizen Science Program: Overview and First Results

HARVEST runs as a series of steps that are composed of constituent tasks; tasks are executed in series or, when possible, in parallel. Tasks are primarily written in python 3 code, with some compiled programs called as specified in the subsections below. We optimized the pipeline for execution on high-performance computing clusters that employ the Slurm task scheduler (Yoo et al. 2016), so the top-level pipeline steps are conducted via Bash shell scripts. Key concepts needed to understand HARVEST are provided here. Advanced technical considerations are discussed in Chandler (2022).

Throughout HARVEST, we implement an "Exclusion" system that is essential to optimizing the chances of success that volunteers will identify activity in an image they examine. For example, we do not want to submit images to volunteers for classification that we determine (via automated algorithm) contain no source at the center of the frame. These are described in the corresponding pipeline subsections below.

HARVEST makes use of a custom MySQL relational database of our own design. The database is composed of numerous tables to optimize memory usage. Here, we describe the key elements essential to the pipeline.

Observations are records holding the UT observation date and time, as well as the identity of the telescope, instrument, broadband filter, Principal Investigator (PI) name, and proposal ID. Each Observation record can have one or more associated Field records, each containing air mass, angular separation from the pointing center to the Moon's center, and R.A. and decl. sky coordinates.

Data files are the records specific to a particular version of a produced data file, such as exposure time and release date (when the data became or will become publicly available). We store our computed depth estimate here (discussed further in Section 2.4). Each data-file record may have many thumbnail records, one for each of the individual cutouts centered on a known minor planet we produce. We strive to keep only one thumbnail per unique combination of observation and solar system object, despite the necessity to download different versions of data files in cases where the archive-provided data file was corrupted.

Solar system objects are records containing compiled information about individual bodies of the solar system, including orbital elements and discovery circumstances. Skybot results are the tabular data returned by the Institut de Mécanique Céleste et de Calcul des Éphémérides (IMCCE) Skybot Service (Section 2.5), such as computed sky position and apparent V-band magnitude, geocentric and heliocentric distances, phase angle, and solar elongation.

In this step, we query astronomical image archives for metadata pertaining to observations. The essential elements include sky coordinates, exposure UT date/time, exposure time, broadband filter selection, release date (when the data becomes public), and data location (URL). We primarily query instrument archives that hold calibrated data with well-calibrated World Coordinate System (WCS) header information. The two archives we query are the National Optical and Infrared Laboratory (NOIRLab) AstroArchive and the Canadian Astronomy Data Centre (CADC) data archive. Our pipeline produces thumbnail images for several instruments, with Dark Energy Camera (DECam) the sole data source we have made use of thus far for our Active Asteroids Citizen Science program.

We query external resources for information about known minor planets, including orbital elements (e.g., semimajor axis a, eccentricity e, inclination i), identity information (e.g., minor-planet numbers, provisional designations), and discovery circumstances (e.g., date, site). These include the Minor Planet Center (MPC), JPL Small Body Database, and Lowell Observatory's AstOrb database (Moskovitz et al. 2022). The Ondrêjov web page lists objects discovered at Ondejov Observatory (site code 557), ²⁰ and includes identifiers sometimes not found at the MPC but that may be returned by SkyBot (Section 2.5). We note the late Kazuo Kinoshita's comet page is no longer being updated, but it is included here as we have incorporated his work. ²¹

We exclude observations (i) taken at an air mass greater than 3.0, (ii) calculated to have a pointing center <4° from the Moon's center, (iii) with invalid pointing coordinates (e.g., R.A. >360°), and (iv) acquired with broadband filters typically unfavorable to activity detection. We exclude data files that (i) are uncalibrated (i.e., raw) as activity is harder to detect and the embedded WCS is likely insufficient to place the object at the center of our cutouts, and (ii) are stacked (coadded) images that typically eliminate moving objects.

Magnitude estimates. We compute a rough estimate of image depth, a value not necessarily provided with the archival data. We employ functions that are instrument specific and, wherever possible, are based upon an observatory-supplied exposure-time calculator (ETC). In cases where no ETC was available, we applied our DECam-derived estimator, adjusting the mirror area as needed. We estimate the magnitude limit achievable for a minimum detection at a 10:1 signal-to-noise ratio (S/N). To compare the depth estimate with object-specific magnitudes computed by ephemeris services (e.g., JPL Horizons), which are always provided in the Johnson V band, we apply a rudimentary apparent magnitude offset from measured apparent Vega magnitudes of the Sun (Willmer 2018). The difference between ephemeris magnitude and depth we call delta magnitude (Δm; see Section 2.5). This allows us to exclude thumbnails for which an object and potential activity is fainter than the detection limit of a given exposure.

Version selection. Archives may provide multiple data-file versions for an observation, such as InstCal and Resampled images via AstroArchive. We choose a single data file to work with and exclude all others. If we later encounter a problematic (i.e., corrupt) data file, we can select a different version.

NASA JPL object data. We maintain an internal table of NASA JPL-provided minor-planet parameters (e.g., semimajor axis) that may not be provided by other services we utilize. We query both JPL Horizons and the JPL Small Body Database (Giorgini et al. 1996).

Solar system object parameters. Here, we assemble a consolidated set of dynamical elements (semimajor axis a, inclination i, eccentricity e, perihelion distance q, and aphelion distance Q) and compute the Tisserand parameter with respect to Jupiter T_J, needed for dynamically classifying objects. The classes are listed below, and the methods are discussed in Section 6.

Object classification. Each minor planet in our database is labeled with a single associated class from the following: Comet, Amor, Apollo, Aten, Mars-crosser, inner main belt (IMB), middle main belt (MMB), outer main belt (OMB), Cybele, Hungaria, Jupiter-family comet (JFC), Hilda, Trojan, Centaur, Damocloid, Trans-Neptunian object (TNO)/Kuiper Belt object (KBO), Phocaea, or interstellar object. These are dynamical classes, with the notable exception of comets, which are classified as such when visible activity has been reported. We use the class name provided by the IMCCE Quaero Service as these are included with SkyBot results—but we intervene to reclassify some objects as long as they are not labeled as Trojan asteroids. Specifically, following the procedures described in Section 6, minor planets with (i) a Tisserand parameter with respect to Jupiter (Section 6) 2 ≤ T_J < 3 we reclassify as a JFC, (ii) T_J < 2 we reclassify as a Damocloid, or (iii) a_J < a < a_N (a semimajor axis a between those of Jupiter and Neptune, a_J and a_N, respectively) are labeled a Centaur. We note that we treat classifications in HARVEST as approximate as they are not based upon custom dynamical simulations. However, the rough fit is adequate for our purposes (e.g., selecting images for a subject set; see Section 3.3).

Here, we perform tasks specific to a unique combination of telescope pointing and UT date/time, internally stored as Field records. As noted earlier, multiple records can exist for a single Observation record because different process types (e.g., InstCal) can result in slightly different WCS information, though our database should only maintain one nonexcluded Field per Observation as a result of version selection (Section 2.4).

SkyBot. The IMCCE SkyBot (Berthier et al. 2006) service returns a table listing the solar system objects that may be found within a given combination of sky coordinates, UT date/time, and field of view (FOV). We construct each query as a "cone" (circular field) or "polygon" (rectangular field), depending on the instrument FOV, and query SkyBot for all new fields (Section 2.3) added to our database during the daily HARVEST schedule. For computational and service call efficiency (i.e., to avoid excessive queries to the SkyBot service), instead of querying all fields via SkyBot daily, we only periodically (every ∼90 days) resubmit fields to SkyBot to search for minor planets discovered since we previously queried the field via SkyBot.

Delta magnitudes. During the SkyBot phase we calculate a metric to estimate how many magnitudes brighter (or fainter) an object will appear in a field, by

$$\begin{eqnarray}&&{{\rm{\Delta }}}_{\mathrm{mag}}={V}_{\mathrm{JPL}}-{V}_{\mathrm{ITC}},\end{eqnarray} \tag{ 1 }$$

where V_JPL is the object's apparent V-band magnitude as computed by the JPL Horizons ephemeris service (typically Johnson V), and V_ITC is our computed V-band depth (Section 2.4). Objects with Δ_mag < 0 are above a S/N of 10 and should be detectable, whereas Δ_mag > 0 would likely not be detectable. We exclude SkyBot results from our database that have Δ_mag > −1 because our goal is to detect activity, and our experience has been that at least one additional magnitude of depth is necessary for this task. We acknowledge that activity outbursts could result in a significantly brighter apparent magnitude than our estimate, but maintain this threshold to eliminate low-probability detection events among a high volume of extraneous images volunteers will examine. Roughly 57% (∼21 million) of SkyBot results in HARVEST have been excluded because of our chosen Δ_mag threshold. Additional considerations for adjusting this threshold are discussed in Section 8.2.1.

Trail length. Making use of the ephemeris-supplied apparent rate of motion on the sky, we estimate trail lengths for each object given the exposure time. We used this measurement for constructing the project Field Guide and, as of 2023 August 15, we have suspended submitting images with trails >15 pixels for examination, as these have proven to be a common source of false-positive detections by volunteers.

Data download. Here, we generate scripts to download data from astronomical archives. Downloads occur when new data have become publicly available, or a new object was found in an existing field. The transfer of data is handled by daemons we constructed for this purpose, each dedicated to downloading data from a single archive (e.g., AstroArchive, CADC). As the process can take days, HARVEST continues operations without waiting for these tasks to finish; these data can be processed during a subsequent execution of the pipeline.

Chip corners. For every image we download we record the sky coordinates of each corner of all camera chips. For DECam there are >60 chips, which make up a mosaic that covers a roughly circular area on the sky. This step eliminates the need to check every chip corner for each thumbnail image to be produced, thereby enabling an order-of-magnitude compute time savings during thumbnail extraction (Section 2.7). This also allows us to determine if an object falls outside of any detector area (e.g., chip gap), negating the need to redownload a file from an archive, such as when a new object is discovered and in a field (Section 2.5).

FITS thumbnails. We extract Flexible Image Transport System (FITS) format cutouts for each SkyBot record that was not excluded by searching for the object in the chip corners table. Cutouts have a 126'' by 126'' FOV which, for DECam, results in a 480 × 480 pixel image, each requiring ∼1 Mb of disk space. We preserve WCS in thumbnail images, as well as primary headers and the headers for the specific chip from which the cutout was extracted.

PNG thumbnails. Here, we convert the FITS thumbnail images to Portable Network Graphics (PNG) format for submission to Zooniverse or examination by our team. We employ an iterative rejection contrast enhancement scheme (Chandler et al. 2018) to facilitate activity detection. Each PNG thumbnail requires ∼512 Kb of storage.

Source analysis. We produce tables of sources found within each cutout with SExtractor (Bertin & Arnouts 2010) and apply exclusions based on our analysis of these data. The following exclusions are applied, with representative statistics derived 2022 July 10, when HARVEST contained 22,004,739 nonexcluded thumbnail records. (i) No source was detected within the center 20 × 20 pixel region; 16% (4,248,133 thumbnails) excluded. (ii) >150 sources were found in the center 270 × 270 pixel region; 4% (952,289) met this criteria. (iii) >5 blended (overlapping) sources were detected at the cutout center; 0.4% (84,697 thumbnails) were affected.

Source tallying. All tasks that perform exclusions have concluded. We perform tallying to optimize reporting, and consistency checks: (i) Objects per Field, the number of nonexcluded solar system objects in each field; and (ii) SkyBot source density, the tally of nonexcluded SkyBot results associated with each field.

SkyBot reports. We generate plots and tables describing how recently each field has been submitted to SkyBot, primarily for diagnostic purposes.

Objects per Field. This diagnostic aid quantifies valid objects in each pointing that are passed on for processing. This helps us, for example, project our future Citizen Science project completeness (Section 8.2.1).

Data file checks. We check image files we have downloaded for integrity by querying the HARVEST database for images that have been marked as "bad data files" by other tasks, typically those failing the AstroPy FITS verification process. Files we diagnose as corrupt go through a process where we download the data again and, upon a second failure, we identify a replacement if another version is available (Section 2.4).

Data file exclusion by property. Here, we exclude from the HARVEST database all data files with invalid properties, such as exposure times <1 s or NULL values. While this screening is also done during the Catalog step (Section 2.3), we routinely repeat screening as a safety measure.

Purge data files. Once all thumbnail images have been extracted from a downloaded image and all subsequent analysis processes have completed, we purge the file from disk as we do not have the requisite storage necessary to keep all of the downloaded image data.

We chose to host our project, Active Asteroids, on the online Citizen Science platform Zooniverse because of their proven track record of supporting successful astronomy-related projects. Their team also provides developmental support for project customization, which is important for our project workflow (Section 3.4).

The overall process for Active Asteroids, from launch to ongoing operations, is as follows. (i) Prepare Zooniverse project (see sections below). (ii) Test project viability via a Zooniverse "beta release." (iii) Formally launch Active Asteroids for public use. (iv) In a cyclic fashion, (a) interact with volunteers, (b) download and analyze results, (c) prepare and upload a new batch of images, (d) notify volunteers of new data and other news, and (e) investigate activity candidates.

We formally launched Active Asteroids on 2021 August 31. ²² Since then more than 8300 volunteers have examined over 430,000 images, carrying out a total of some 6,700,000 classifications (including both sample and training data).

The project "workflow" is the task volunteers are asked to perform. At present, we have one concise workflow where we ask volunteers if they see activity (i.e., a tail or coma) coming from the central object, marked by a green reticle like that shown in Figure 1. The first time participants begin classifying images they are shown a tutorial we produced that demonstrates images of activity along with tips for avoiding activity lookalikes (e.g., background galaxies). During the classifying process, users may return to the tutorial at any time. Also available during the classification process is our comprehensive Field Guide, which discusses phenomena participants may encounter, such as cosmic rays.

Zoom In Zoom Out Reset image size

The Zooniverse web structure includes several other areas important to project success. An "About" section includes pages describing (i) our research and science justification, (ii) project team members, (iii) a listing of results (e.g., publications) stemming from the project, and (iv) a frequently asked questions (FAQ) page. The "Talk" discussion boards (forums) provide a place for participants and the science team to interact and build relationships. Surprisingly, we have made discoveries that first come to light on the Talk pages, well before the subject set was fully retired (discussed below).

A "subject set" is a collection of images and associated metadata (e.g., image names, object designations). We try to select a subject set size (i.e., number of images) that balances preparation overhead with turnaround time to complete subject set retirement. Smaller subject sets are fully examined by volunteers in fewer days, but each batch requires significant overhead—both effort and time—for our team to (i) prepare each batch (described below) and (ii) analyze classification data (Section 4). Conversely, large batches take longer to complete. We found a good balance to be a subject set size of ∼22,000 images, which typically needed 4 to 8 weeks for volunteers to examine (Section 8.2).

To create subject sets, we (i) assign images from HARVEST based upon selection criteria (described below), and (ii) gather images and prepare them for upload to Zooniverse by adding a green reticle (Figure 1). The ability to select objects by criteria is motivated by the need to show volunteers a variety of images, and to optimize the discovery of activity. We fully recognize that our choices impart biases, but err on the side of making the best use of volunteer efforts.

We assemble each batch as a collection of members from different dynamical classes. As of 2023 August 17, we have submitted 19 subject sets for examination, as aforementioned typically containing ∼22,000 images. The composition has changed too as we have exhausted the images of some minor-planet classes, and have de-emphasized others (e.g., near-Earth objects, NEOs) that have proven problematic for activity identification.

To improve chances for identifying activity, we prioritize selecting images of objects closer to their perihelion passage, with the assumption that activity is more likely to be present around this point in an object's orbit. We achieve this effect by sorting HARVEST images by our simple metric, "percentage to perihelion" (Chandler et al. 2018), given by

$$\begin{eqnarray}&&{ \% }_{T\to q}=\left(\displaystyle \frac{Q-d}{Q-q}\right)\cdot 100 \% ,\end{eqnarray} \tag{ 2 }$$

where d, q, and Q are its orbital, perihelion, and aphelion distances, respectively. This metric is more efficiently sorted than the more familiar true anomaly angle, though %_T→q does not describe the direction (i.e., inbound to, or outbound from, perihelion) of the object.

By default, we only show one image of an object in a given batch, such that volunteers are examining the maximum number of individual minor planets. In cases where we have few object images remaining (e.g., Centaurs), we do increase this number. Conversely, as of 2022 October 22, we have had the option to skip objects entirely that have already been examined by volunteers at least once. This is especially useful for populations like the main-belt asteroids, where we have in our collection tens of thousands of images of unique minor planets, all essentially at perihelion.

As of 2023 August 18, we always apply further delta magnitude limits (Section 2.5). We typically require ≥2 magnitudes brighter than our computed exposure depth (i.e., Δ_mag ≤ −2). We consider this threshold reasonable given the project's current classification rate and projected completeness timescales (Section 8.2).

The training system implemented by Zooniverse for Active Asteroids is designed to teach volunteers how to identify activity. We show in Section 4 that this system measurably improves activity detection ability for the vast majority of participants. The system also served to validate that the project was functioning as intended during the launch phase, and the training system continues to serve that function today.

For training purposes, we created a subject set consisting of images known to show an object displaying cometary activity at the center. To achieve this, we manually examined ∼10,000 images of known active bodies produced by the HARVEST pipeline and assigned a score to each image. The subjective scoring system, introduced in Chandler (2022), is as follows: (0) unidentifiable/missing; (1) point-source appearance; (2) vaguely fuzzy; (3) fuzzy, activity unlikely; (4) inconclusive activity indicators (coma and/or tail); (5) likely active, some ambiguity remains; (6) activity, not very ambiguous, but faint; (7) definitely active, medium-strength indicators; (8) definitely active, strong activity evidence; (9) definitely active, overwhelming activity indicators. All training images in Active Asteroids are derived from those images to which we applied a score of ≥5, our minimum threshold, for which we consider the activity to be highly likely.

Active Asteroids is configured with two training features. The first is a system that periodically shows the user a training image, at an interval that decays with user experience (determined by the number of images N they have classified), given by the probability

$$\begin{eqnarray}{P}_{{\rm{T}}}(N)=\left\{\begin{array}{rr}50 \% & 1\leqslant N\leqslant \ 10\ \\ 20 \% & 10\lt N\leqslant \ 50\\ 10 \% & 50\lt N\leqslant 100\\ 5 \% & 100\lt N\lt \infty \ \end{array}\right\}.\end{eqnarray} \tag{ 3 }$$

The second feature is a feedback system, wherein immediate feedback is given to the user about their training image classification, whether their classification was "correct" or not. While this serves as a direct training mechanism for new participants, it also serves to reinforce the abilities of experienced users and helps keep volunteers engaged in the classification process.

Initially, we computed, for each image i, a simple activity likelihood metric M₀(i) as the ratio of "yes" classifications for the image, Y_i , and the total number of classifications, i.e., the sum of yes and no, N_i , responses for that image, as

$$\begin{eqnarray}&&{M}_{0}(i)={Y}_{i}/({Y}_{i}+{N}_{i}).\end{eqnarray} \tag{ 4 }$$

For the development of the new metrics (discussed in the subsequent section), we validated underlying premises (e.g., users become more experienced through time) as we developed the methods. The exception was this naïve metric, which served as the starting point from which we set out to improve our classification analyses.

Here, we also define the minimum "threshold," L_min, of a metric. This serves to differentiate between images likely to show activity—and thus qualify as candidates that our team will investigate—from those that are not. For the naïve (unjustified) threshold, we chose L₀ ≥ 80%. From this initial combination of metric and threshold we set out to test and improve upon our initial selection of metric and threshold.

Weighting based upon assessment of volunteer trends has been employed by other Citizen Science programs. For example, Gollan et al. (2012) found Citizen Scientists may on average not perform as well as professional scientists when performing the same tasks, but also that some individuals do meet or exceed that same standard.

Metric 1: Training image accuracy. This metric considers users who perform well with training data as having more expertise than those who perform poorly, thus the more expert users should be given more weight. To quantify training image performance, we measure the ratio of a user's successful training image classification, Y_training, to their total number of training image classifications, T_training, as

$$\begin{eqnarray}&&{M}_{1}={Y}_{\mathrm{training}}/{T}_{\mathrm{training}}.\end{eqnarray} \tag{ 5 }$$

Metric 2: log10 number of classifications. As users classify more images, they generally become more experienced, so they should be given greater weight. This metric quantifies "experience" as a log₁₀ of the total number of classifications for a given user. We placed an upper limit of 10,000 classifications, and scaled weights to span the range of 0–1 by dividing all weights by 4 (i.e., ${\mathrm{log}}_{10}10,000$):

$$\begin{eqnarray}&&{M}_{2}=\displaystyle \frac{1}{4}{\mathrm{log}}_{10}({T}_{\mathrm{total}}),\end{eqnarray} \tag{ 6 }$$

where T_total is the total number of images the user had examined, including training images.

Metric 3: Optimism debiasing. We found that some users identify activity much more often than would normally be expected, thus their weight needed to be lowered accordingly. Noting that the activity occurrence rate among main-belt asteroids is estimated to be roughly 1 in 10,000 (Hsieh et al. 2015; Jewitt et al. 2015; Chandler et al. 2018), we expect a low rate of "yes" classifications. Moreover, from our cursory examinations, we estimated no more than ∼1% of images should warrant being flagged as likely active. To search for bias, we first described the fraction of classifications a user, u, submits as positive by

$$\begin{eqnarray}&&{F}_{u,Y}={Y}_{u}/({Y}_{u}+{N}_{u}),\end{eqnarray} \tag{ 7 }$$

where Y_u and N_u are the total number of "yes" and "no" classifications for that user, respectively. Around 35% of users (2400) clicked "yes" over 20% of the time, indicating an optimism bias is present. Any weight based purely upon training accuracy is skewed for users selecting "yes" to most images they classify, with a potential resultant weight of unity for training accuracy while incorrectly reflecting activity detection ability. For this metric, the more frequently a volunteer sees activity, the lower their weight becomes, via

$$\begin{eqnarray}&&{M}_{3}=1-\left({Y}_{\mathrm{sample}}/{T}_{\mathrm{sample}}\right),\end{eqnarray} \tag{ 8 }$$

where Y_sample is the number of times a user saw activity in sample images, and T_sample is the total number of sample images the user classified.

In order to test the efficacy of each metric (or combination of metrics), we maintained a "control list" of images that our team vetted and labeled as strong activity candidates. We set a threshold L_min for each metric (i.e., L₁, L₂, and L₃) by iteratively increasing or decreasing the threshold L in 10% increments until all control list images appeared in the final output list of candidates. We arrived at an ${L}_{\min }=40 \%$ threshold that resulted in control list completeness for all metrics; henceforth, this served as our initial threshold when testing metrics and combinations thereof. Throughout, a secondary goal was to minimize the number of extraneous (inactive) images flagged as promising candidates, while still including those from the control list.

Figure 2 shows combined user weights over time for 10 randomly selected users, where time here is measured only by the number of images classified. User weights typically improved, but not always (e.g., user #7 of Figure 2), indicating that one or both of the metrics not solely dependent on classification count (i.e., M₁, training accuracy, or M₃, optimism debiasing) must be significantly altering the weight. This finding showed (i) a need to evaluate metrics temporally, (ii) each metric may need a multiplier (weight), and (iii) we cannot assume user abilities improve over time. To capture this time-dependent weight we employed a 5th-order polynomial fit for each user's weight over time.

Zoom In Zoom Out Reset image size

We tried combinations of metric weights, ranging from 0 to 10, for each metric (M₁, M₂, M₃), plus 100×, 1000×, and 10,000× to test extrema. For computational efficiency, we (i) eliminated weight combinations that were integer multiples of each other that would yield identical scores, and (ii) selected only even-number weight multiples, thereby reducing the number of required compute tasks while still covering the full range of weights. We combined the weighted metrics via

$$\begin{eqnarray}&&w=\displaystyle \frac{{W}_{1}{M}_{1}+{W}_{2}{M}_{2}+{W}_{3}{M}_{3}}{{W}_{1}+{W}_{2}+{W}_{3}},\end{eqnarray} \tag{ 9 }$$

where W is the combined weight for a user, W₁ is the weight of M₁ (training image accuracy), W₂ the weight for M₂ (log₁₀ of classification count), and W₃ is the weight for M₃ (optimism debiasing).

We created an overall weighted likelihood score, L, computed as a ratio of the sum of an image's weighted user "yes" classifications, w_Y , to the sum of all the users' weights w, given by

$$\begin{eqnarray}&&L=\displaystyle \frac{{\sum }_{i=1}^{m}{w}_{Y,i}}{{\sum }_{j=1}^{k}{w}_{j}},\end{eqnarray} \tag{ 10 }$$

with m the number of users who classified the image as showing activity, and k is the total number of users who classified the image.

For each set of weight combinations we (i) calculated a score for all sample (nontraining) images using that set of weights, (ii) determined the threshold L_min needed to include the images of our control list (Section 4.3), and (iii) recorded the number of images, I_f, that received a score $L\geqslant {L}_{\min }$ for that set of weights, including images not part of the control list.

We evaluated each metric independently and in combination, and compared these to the naïve metric M₀. Our newly crafted method for determining which images warrant further investigation performed markedly better than the naïve method (Section 4.1). The naïve method resulted in a threshold of ${L}_{\min }=46.66 \%$ for I_f = 2513 images (1.48% of the classified images), 795 more than our weighted method. Moreover, employing any one standalone metric alone underperformed when compared to the combined approach (I_f = 1718 images (1.01%), ∼1%): M₁ (training accuracy) gave I_f = 1807 images (1.06%), M₂ (number of classifications) I_f = 1972 images (1.16%), and M₃ (optimism debiasing) returned I_f = 1995 images (1.17%).

We selected a final weight combination of W₁ = 7, W₂ = 2, and W₃ = 1, with a threshold ${L}_{\min }=47.3 \%$. This combination resulted in 1718 activity candidate images for our team to examine, ∼1% of the ∼170,000 images (discussed at the beginning of this section) used for the classification analysis improvements.

We evaluated the 15 image retirement criteria (Section 3.3) by varying the number of classifications required per image prior to subject retirement. We calculated the combined weighted score for every image, considering only the first n classifications for each image, for 1 ≤ n ≤ 15. The number of extraneous images flagged for investigation declined through n = 14. While n = 14 unexpectedly performed marginally better (233 images) than n = 15, we interpret this result as indicating that 14 classifications per image represents a necessary minimum to achieve the best results for our project. We chose to keep n = 15 for the live project, in keeping with the original Zooniverse recommendation.

Our archival investigation process typically involves first querying our internal HARVEST database for additional images of the object. This first pass enables us to quickly rule out some false-positive candidates, for example those with apparent activity that we recognize as a background source when the object is viewed in an image sequence.

For the remaining candidates, we next query external services via three pipelines we have written for the purpose, and manually query two additional sources. There are significant drawbacks to these systems as compared to HARVEST, most notably a high fraction of "junk" images that are either too faint to see the candidate, or where the candidate is not captured by a detector. Nonetheless, this in-depth search can yield images of activity that are not available through the HARVEST pipeline.

We query the following archives (see Acknowledgments for additional references) as part of the aforementioned pipelines.

CADC SSOIS. We query the CADC Solar System Object Information Search (SSOIS; Gwyn et al. 2012) for DECam, MegaPrime, Kitt Peak National Observatory instruments, Southern Astrophysical Research Telescope, SkyMapper, Las Campanas Observatory, European Space Organization instruments (e.g., Very Large Telescope Survey Telescope, VST, OMEGACam), Near-Earth Asteroid Tracking (NEAT) Ground-Based Electro-Optical Deep Space Surveillance, the Sloan Digital Sky Survey (SDSS), Subaru Suprime-Cam and Hyper Suprime-Cam, Wide-field Infrared Survey Explorer (WISE), and the Cambridge Astronomy Survey Unit Astronomical Data Centre for the Isaac Newton Telescope's Wide Field Camera data.

IRSA. With their Moving Object Search Tool (MOST), we query the NASA/CalTech Infrared Science Archive (IRSA) for Zwicky Transient Facility (ZTF) and Palomar Transient Factory data.

ZTF alert stream. We download ZTF alert stream data (Patterson et al. 2018) and keep only solar system data.

Manual queries. We manually query (i) the Keck Observatory Archive via their MOST, and (ii) the Comet Asteroid Telescopic Catalog Hub tool, which spans several instruments, including NEAT (Pravdo et al. 1999) and SkyMapper (Keller et al. 2007). After downloading the relevant data, many sources require pre- and post-processing to, for example, perform astrometry to replace an inadequate (or absent) plate solution. We perform astrometry as needed via Astrometry.net (Lang et al. 2010), which makes use of multiple source catalogs, including Gaia (Collaboration et al. 2018) and the SDSS (Ahn et al. 2012). We produce thumbnail images in FITS and PNG formats, and record sky position angle (PA) information indicating the antisolar and anti-motion vectors, as computed by JPL Horizons.

A member of the science team visually examines all of the thumbnail images produced by our follow-up pipelines and searches for activity indicators, such as tails and comae. Thus far, we have examined over two million thumbnail images as Active Asteroids follow-up and in developing the HARVEST pipeline. These data are from myriad sources and vary greatly in image quality and character (e.g., chip gaps, image orientation), and the vast majority of these data do not contain any useful information. Thus, we do not submit images from these secondary pipelines for volunteer examination.

Objects we deem appropriate for follow-up observations are added to an internal list of candidates needing further telescope observations. Hereafter, we refer to telescopes by their name or the instrument: DECam, the Inamori-Magellan Areal Camera and Spectrograph (IMACS), the Gemini Multi-Object Spectrograph (GMOS), the Large Binocular Telescope (LBT), the Lowell Discovery Telescope (LDT), and the Vatican Advanced Technology Telescope (VATT). Table 1 lists the telescopes and facilities our team employs to carry out follow-up observations of activity candidates. We make use of ground-based facilities in both hemispheres to maximize decl. coverage. For target selection, we prioritize objects that are near perihelion (i.e., true anomaly angles of f ≥ 290° and f ≤ 70°).

7.1.1. Active Asteroid (62412) 2000 SY178

7.1.2. Active Asteroid (6478) Gault

In 2019 January, asteroid (6478) Gault (Figure 4(a); Prop. ID 2012B-0001, PI: Frieman, observers SK, DT, NFM) was reported to be displaying activity (Hui et al. 2019; Jewitt et al. 2019; Marsset et al. 2019; Moreno et al. 2019; Smith et al. 2019; Ye et al. 2019; Devogèle et al. 2021). For the first time, our team made use of the HARVEST pipeline, which was not yet complete, to identify images of Gault in DECam data. In Chandler et al. (2019), we reported our subsequent discovery that Gault had been active during multiple prior epochs. We found Gault's activity was not correlated with perihelion passage, and we postulated that Gault is recurrently active due to rotational spin-up, supported by Kleyna et al. (2019) findings of YORP-induced effects on Gault. Our findings stemmed from tools we created to help us understand potential observational biases and correlation effects with perihelion passage and activity outbursts.

Zoom In Zoom Out Reset image size

Even with this strategy in place, both observability (the number of hours an object is above the horizon as observed from a given observatory) and perihelion passage must be coincident to maximize the chances an object will be shown to volunteers. For example, only the 2016 perihelion passage coincided with a peak in observability. At other times (e.g., 2013, 2019) Gault was highly observable, but it was not near perihelion, or Gault was minimally observable (or not observable) at perihelion, as was the case in 2012.

7.1.3. Active Centaur C/2014 OG₃₉₂ (Pan-STARRS)

7.1.4. Main-belt Comet 433P

For the remainder of this section, we discuss 16 objects, all classified as active by Active Asteroids volunteers and brought to our team's attention as a result. A representative thumbnail showing activity for each object is provided in Figure 4. We classify each object into a dynamical class, and in the process refer to (i) object-specific properties (e.g., inclination, perihelion distance), and (ii) the gallery of Jupiter corotating reference frame plots (Figure 5). Table 3 provides a unified collection of data pertaining to the observed activity, most notably the date ranges of activity along with corresponding heliocentric distances r_H and true anomaly angles f.

Zoom In Zoom Out Reset image size

With the exception of 282P/(323137) 2003 BM₈₀, all objects are referred to by their primary provisional designation, with full and alternate designations for each object listed in corresponding subsections. In the tables and subsections provided, we generally group objects by dynamical class first, then sort objects by provisional designation within each dynamical class.

7.2.1. Active Asteroids

7.2.2. Quasi-Hilda Objects

7.2.3. Jupiter-family Comets

We describe here a brief preliminary analysis of the Active Asteroids classifications and results. We caution that (i) the inferences herein have not been debiased in any way, and we impart significant biases in our subject selection process (e.g., we sort objects submitted for classification by distance from perihelion; see Section 3.3); (ii) that the classifications are incomplete (e.g., ∼241,000 of ∼1.1 million main-belt asteroids have been examined by the project thus far); (iii) our investigation into newfound activity epochs is ongoing; and (iv) some dynamical classifications require dynamical simulations (Section 5), and thus may have been incorrectly labeled in the past. While we primarily make use of the object class returned by the Quaero service (Berthier et al. 2006), some classes contain objects with ambiguous membership, e.g., JFCs that also qualify as NEOs.

Table 2 shows metrics by object class, with analyses considering subjects (images) and unique objects. Overall, Active Asteroids volunteers classified 1.3% of the images as showing activity, with our team concurring 33.3% of the time (i.e., 0.04% of all images examined). We investigated these candidates (Section 5) except for training images (Section 3.4), which are included in Table 2 to indicate, for example, volunteer expertise (Section 4).

Despite the disclaimers mentioned above, especially regarding biases and classification incompleteness, we can still make some rough inferences. Of the 240,989 unique asteroids examined by volunteers, they labeled 25 (0.010%) as showing activity, consistent with prior activity occurrence rate estimates, roughly 1 in 10,000 (Hsieh et al. 2015; Jewitt et al. 2015; Chandler et al. 2018), or 0.01%. Active Asteroids volunteers examined 193 Centaurs, of which 11 (5.7%) qualified as activity candidates by our enhanced classification analysis (Section 4).

To assess the Centaur activity occurrence rate in the context of the broader Centaur population, we first needed a list of Centaurs from which we shall make our comparisons. We also must define a Centaur as these objects are described by multiple definitions in the literature. Derived from Jewitt (2009), we define a Centaur as an object (i) with a semimajor axis a and perihelion distance q between the aphelion distances of Jupiter and Neptune (i.e., Q_J < a < Q_N, Q_J < q < Q_N), and (ii) not in 1:1 MMR with a giant planet. This latter requirement excludes the 24 Neptune Trojans and two Uranus Trojans known as of UT 2023 December 9, and the a and q constraints exclude objects that cross the orbit of Jupiter.

We queried the MPC list of Centaurs and scattered-disk objects ²⁴ and the JPL Small Body Database via their query tool ²⁵ for objects that match our a and q requirements. The results largely overlapped, noting that (i) the MPC did not include objects with comet designations in their list, and (ii) in two cases (2010 HM₂₃ and 2015 FZ₃₉₇), orbital element disagreement between the two services (e.g., 2010 HM₂₃ a = 32.35 au via JPL and a = 27.90 via the MPC) caused an object to appear on one list but not the other. In both cases, we included the objects on our final list. Subsequently, we removed the known Trojans.

Of the 346 Centaurs on our list, 31 are active Centaurs, indicating an activity occurrence rate among the Centaurs of about 9%. This figure is in agreement with Peixinho et al. (2020), and lower than the 13% rate of Jewitt (2009) that was measured when only 92 Centaurs were known (of which 12 were active). Because Active Asteroids volunteers only examined images of about half of the known Centaurs, it is unsurprising that the 5.7% identification rate differs from our 9% rate.

Similarly, we consider the ratio of JFCs to ACOs, where the former have shown activity and the latter have not (Licandro et al. 2006, 2016). We queried the JPL Small Body Database for objects with Tisserand parameters with respect to Jupiter 2 < T_J < 3, the canonical range for JFCs, and excluded the Jupiter Trojans as well as all objects from our Centaur list. We note that we only counted one object per parent designation (i.e., we excluded fragments except for the primary designation). We flagged each object on our list as either active or inactive, based on their cometary designation or lack thereof, and we also flagged the 13 qualifying objects included in this work as active, plus another established active object, 2008 GO₉₈ (García-Migani & Gil-Hutton 2018). Of the 14,407 minor planets on our ACO+JFC list, 668 have been observed to be active. Thus, we find the apparent occurrence rate (i.e., observed fraction) of active objects in the ACO+JFC population to be 4.6%. We reiterate our query did not restrict our population selection by any physical property (e.g., albedo or color; Licandro et al. 2016) other than observed activity, and we did not limit our ACOs or JFCs populations using the Tancredi (2014) method.

The other classes cannot be meaningfully evaluated at this time due to, for example, currently unresolved ambiguities in overlapping class definitions (e.g., JFC, QHC).

We carried out analyses to better understand performance when classifying data. Here, we discuss the total number of submitted thumbnail images, as well as metrics from select dynamical classes of relevance to this discussion. At the time of these analyses, there were 406,082 sample (i.e., nontraining) images in the Active Asteroids project on Zooniverse. Of these, 4171 (1.03%) of the images qualified as candidates by our analysis system (Section 4). Our team flagged 526 (12.6%) of these candidates as warranting additional investigation, which we define as reaching a threshold of ≥4 based on our activity likelihood score (Section 3.4).

If we do not limit our assessment to candidates flagged by volunteers, we found an additional 138 thumbnail images that our team had previously flagged as candidates that the project had not; these images are members of dynamical classes that we examined extensively during project preparations.

The lowest fraction of objects classified as candidates by volunteers were the main-belt asteroids. Of the 300,526 images of main-belt asteroids submitted for classification (Table 7.3), 2707 (0.9%) qualified as candidates based on analysis of volunteer classifications. Of these, our team flagged 258 (9.5%) as warranting follow-up. Conversely, the highest fractions occurred with the comets. Of the 1150 sample (nontraining) images of known comets, 300 (26.1%) were flagged as candidates, and 267 (89.0%) of these our team also classified as warranting follow-up.

As of 2023 July 3—the date of our last Zooniverse data export—volunteers averaged 12,770 ± 10,750 classifications day^–1, with a maximum rate of 129,338 classifications day^–1 taking place on the project launch date, 2021 August 31. These figures include both training and sample images, and exclude dates with <1000 classifications, which typically occur when no new sample data are available on the project for volunteers to classify. Of the 6,543,368 classifications in this data export, 353,058 (5.4%) were training images. Taking this training fraction into account, the mean retirement rate (nominally 15 classifications per image; see Section 3) is 805 ± 678 sample images day^–1, and our nominal peak rate covers ∼8000 sample images day^–1.

Some of the variation we observe in the classification rate is due to external factors, such as media attention and publications of our findings. Internal factors are driven by when we email newsletters to participants, and lulls between subject sets are due to pauses instilled while we examine previous results and prepare a new batch. The cause of the remaining variation is unknown, though we have speculated that seasonal societal effects, such as vacation times, for example, may be partially responsible. Nevertheless, with nearly 2 yr of data to draw upon, we will consider the average and peak rates mentioned above for the remainder of this discussion.

It is worth mentioning here that we typically consider three modes of optimizing results through our Citizen Science project. All three modes can either increase the number of images examined or reduce the time it takes to complete the examination of an entire subject set. (i) Additional participation by existing volunteers, or an increased number of participants. (ii) Optimized analysis of classification data that is capable of reducing the time to retirement for at least a portion of subjects in a subject set. (iii) Reducing the amount of data needing classification, through either (a) more advanced automated vetting, or (b) measured decisions to exclude certain data. The discussion that follows focuses on decisions to reduce the volume of data. Yet we continue to work toward improving all three areas, especially techniques involving applications of artificial intelligence (AI). However, those areas are still under development and outside the scope of this manuscript.

8.2.1. Current DECam Dataset

8.2.2. Considering the Legacy Survey of Space and Time

With the help of thousands of volunteers, Active Asteroids has produced over 20 discoveries thus far, resulting in numerous publications. Further, dozens more candidate objects are actively under investigation by our team. Clearly, Active Asteroids is successful at accomplishing its primary goal of making active-body discoveries while engaging the public in the scientific endeavor. Our engagement with volunteers continues to grow, with several Active Asteroids participants now included as authors on publications, including this manuscript. As we continue to innovate optimizations for the entire process, from HARVEST pipeline to Citizen Science to follow-up investigation, we optimistically anticipate discoveries to only increase in frequency as the project moves forward. Moreover, these optimizations include improved image vetting designed for both the current project and the upcoming LSST. Anyone with an internet connection who can visually examine images can participate by visiting the Active Asteroids website. ²⁶