Resident space objects classification by machine learning techniques

In recent years the number of resident space objects in near-Earth space has been increasing drastically, which can lead to a “cascade collisions” effect and other catastrophic consequences. Therefore, it is crucial to monitor resident space objects, maintain and update catalogues on time, including the identification of unknown space objects to estimate their location and potential orbit. It is necessary to perform resident space object identification quickly and accurately to avoid possible satellite collisions resulting in large clouds of uncontrollable space debris fragments. That is why the task of resident space object preliminary classification by TLEs is stated and solved by applying machine learning approaches to achieve a higher classification quality and speed. This issue is a highly imbalanced classification problem, which narrows and specifies the used models. Applying the proposed resident space object classification model to space surveillance systems can decrease the time required for the object identification significantly.


Intoduction
There are thousands of registered resident space objects (RSO) in near-Earth space called near-Earth objects (NEOs).Technogenic NEOs include many space objects, such as operational artificial Earth satellites and defunct ones, broken spacecraft, upper-stage blocks, and other launch vehicle parts, debris, etc.In fact, functional on-orbit satellites make up only a small part of all registered RSOs with a diameter greater than 5-10 cm [1,2].However, the RSOs amount has been uncontrollably increasing throughout the previous years, which can lead to a collisional cascading (or Kessler syndrome [3]) and other disastrous consequences for space missions [2].For example, this effect can cause irreversible disruption in the operation of satellite constellations and a general impossibility for the further space exploration development [4].According to several scientists' reports, this cascading process of space debris collisions has already started in the Low-Earth orbits (LEO) resulting in an increase of the RSOs amount in orbits with altitudes from 800 to 1000 km.While the moment when collisional cascading begins is not defined precisely yet due to the large uncertainty of observations [5], it is established that the growth of NEOs' amount is going to be steady in the coming decades even if no new spacecraft is launched into space [3,4].
The first consequences of such an increase in satellites number and a possible leak of regulations and control in this area could already be witnessed in the previous century.In particular, the first collision resulting in a fault of an operational satellite took place in 1996, when the French satellite CERISE collided with a debris item.Moreover, the first satellite collision happened in 2009, when the American communication satellite Iridium 33 and the derelict Russian Kosmos-2251 crashed.This collision led to a cloud of debris formation consisting of at least 2200 pieces [5], which were catalogued upon it.Although the number of RSOs has been increasing since the first satellite launch in 1957, research into the problem of space debris began only in the 90s of the previous century [6].
In addition to effective active methods ways of space object removal being developed [4], it is necessary to observe and control catalogued objects to prevent potentially risky situations such as satellite collisions.In this case, it is necessary to record and analyse the provided observations of the RSOs in near-Earth space.The whole process of data analysis can be divided into three stages: direct observations, processing of the gained data, and trajectory prediction of space objects.Each of these steps is closely connected with the data analysis and the big data processing tasks, where processing accuracy and speed are supposed to be crucial parameters for ensuring space safety.Therefore, various data processing algorithms should be analysed and developed, using machine learning methods which can speed up and increase the quality of processing large amounts of near-Earth object observations.Space object observations processing requires the identification of an object by its heterogeneous observations while matching them with the catalogued ones.Nowadays the identification problem is usually solved by calculating correlations between simulated orbits of catalogued objects and the observed trajectory [7,8].The method for preliminary RSO classification by its type or satellite constellation, proposed in this article, narrows the number of potential object matches several times, which can increase the speed of the whole object identification process significantly.TLE records can be used to classify space objects as they contain information about orbital elements and some characteristics of the object's dynamics.For instance, the statistical analysis of TLE [9] distinguishes 9 remote sensing satellite constellation categories.Besides that, an ensemble of machine learning models (an autoencoder and a decision tree) can be applied to classify satellites in LEO by their orbital states [10].This conducted research demonstrates the high accuracy of the classifications (>90%) obtained after training ML models on the historical sets of TLE records.
Thus, this work is going to explore and compare machine learning methods applied for classifying objects in low Earth orbits in keeping with their sets of TLE elements.For this purpose, LEO space objects are divided into 12 groups related to their type (an active satellite, a piece of debris, or an unassigned object) or a satellite group or a constellation (OneWeb, Starlink, Cosmos, etc.).This kind of RSO grouping is a novel feature of this work, as it separates both debris from working satellites and satellite constellations from each other.In contrast, only rough classifications were proposed in related articles.For solving the stated classification problem, the training data for further models fitting and assessment was collected from the open-source celestrack.orgpage.Finally, various machine learning methods for solving the imbalanced multi-classification task applied to the space observations specific data are investigated and compared, mostly related to the balanced (or weighted) models.

RSO classification problem
TLE (Two-Line elements) are used in this work as features for the RSO classification.Sets of TLE records include, for example, the name of the satellite as well as the unique number of the satellite in the NORAD database [11].The name of a satellite usually contains the name of the corresponding satellite constellation or a satellite series of launches with similar purposes and characteristics.The common part of the name for groups of satellites (further on referred to as a "type") in different cases corresponds to a type, grouping, series of satellites, or another distributed space system (DSS) of satellites.The proposed kind of RSO splitting into groups makes sense because satellites of the same type are expected to have orbits with similar characteristics, therefore, it should be possible to train a classifier model using TLE observations for the NEOs classification.For this reason, this hypothesis is explored and checked in further sections of this work.
Therefore, a set of TLE records is used in this work to solve the problem of multi-class satellite classification by their types using machine learning methods.The latest records of LEO observations available by 01/06/2023 were obtained using the space-track.orgAPI and used as training and validation datasets.Due to machine learning models being specific, these raw data should be preprocessed before the model training starts [11], as well as deeply analysed to choose more suitable models for optimisation and comparison.At the first stage of data processing, the downloaded records were converted to the TLE format with the help of the tletools library, and the names of the satellites as well as their categories were extracted from TLE records.After that, only the most common categories were left.To be precise, 10 classes with at least 100 records per class were extracted, while other classes were merged into the OTHER class, whereas objects whose names include "RB" (rocket body) or "DEB" (debris) substring were allocated to a separate DEBRIS category.Overall, all objects are divided into 12 classes: COSMOS, CZ, DEBRIS, FLOCK, IRIDIUM, LEMUR, ONEWEB, OTHER, SL, SPACEBEE, STARLINK, YAOGAN.
In addition to merging objects with the known identifiers by categories or series of launches, two separate classes of objects without identifiers (with TBA or OBJECT specified as names) can be established.These two types, TO BE ASSIGNED and OBJECT, respectively, are excluded from the training dataset due to the lack of information about their type.Most likely, space objects of the first type were identified as resident space objects, but a more precise description was not established, whereas it is even unknown whether or not the second type belongs to RSO.In the final sections of this article resulting classifier predictions on these types of objects are also provided to check this hypothesis.Thus, the training dataset of objects with the known categories consists of 19521 records whereas the remaining part of unassigned objects includes 946 records.It means that only about 5% of data was removed from the training set and, consequently, there is still enough data to analyse and to fit classifiers properly.
It should be noted that TLE records contain the satellite's name and its unique number as well as information about its launch time, which cannot be used for the model training because these values are unknown before the object is identified.Therefore, in the next stage of data preprocessing corresponding columns are removed before the model training is started to avoid its overfitting.Besides that, TLE data also contains information about the object's location in space, its dynamics and orbit characteristics.To be precise, orbital elements, the observation time, the object position and motion characteristics are included in the records, which allows using TLE to calculate the coordinates of a space object in the spherical coordinate system, as well as the semi-major axis and the true anomaly of its orbit.In the final stage of data preprocessing these values are added to the dataset using the tletools and astropy Python libraries.After data preprocessing is complete, it is necessary to analyse the available data fully to take into account its specifics and simplify the model fitting and testing.
As a first step of the analysis, the average feature values for different classes are compared.It was found out that the mean values of some features are nearly the same for different RSO categories (for example, 'epoch_day').It makes these columns potentially meaningless for the space object classification, while other columns ('ecc', 'rev_num', 'bstar', etc.) will most likely be more useful.It is clear that some features can be removed from the data to speed up the model fitting without a decrease in quality.Also, some columns, such as 'argp' and 'M', seem to be strongly correlated, the same as 'n' and 'a'.To test this hypothesis a correlation matrix between all numerical features is constructed, which is shown in figure 1.According to this matrix, some pairs of features are indeed highly correlated, which proves that some columns can be excluded from the training dataset.Removing such features from the dataset will help make the classifier model lighter and faster.It will also help to reduce the dimension of training data without a reduction in accuracy [12].
It should also be mentioned that several meaningful features are given in a degree scale (inclination, ascending node longitude, perigee argument, anomaly, spherical coordinates, etc.), which means that, for example, 0.5 and 359.5 values actually mean nearly the same.It is necessary to take into account this data characteristic as most ML models interpret values in terms of their numerical similarity, which in this case will result in a poor classification quality despite the way of data normalisation for such models.Nevertheless, it seems that the tree-based models and, consequently, ensembles of trees should work correctly with this type of data, which makes them more suitable and potentially successful for space object classification by TLE data.Finally, one of the most significant characteristics of the data is its imbalance, according to the distribution shown in figure 2. As is shown in this plot, the most common class is debris, which includes 10K records (about 50% of the entire set), while half of the classes make up less than 3% of observations in total.This issue should be taken into consideration while selecting and training a model because a few ML models can work effectively with such imbalanced data.One of the most common methods of imbalance classification solution is using weighted models (where class fractions are considered to be weights in the loss function).Fortunately, the basic tree-based models in the scikit-learn library support corresponding balanced modifications.Therefore, the first, baseline model is going to be a random forest classifier as it is more stable and complicated than the decision tree, but still meets all of the mentioned criteria and is quite a light-weight model.

Model selection
Before the model selection is started, it is necessary to define the model comparison criteria.To be precise, several classification metrics should be chosen to measure and compare the classifiers' quality.Obviously, these metrics should take into account the category imbalance, otherwise their values could be misleading.Typically, accuracy, precision and recall are used for measuring classification quality as well as their combinations (e.g., F1-score) and modifications.None of them indicates the entire classification correctness but they all reflect different quality indicators and are easy to estimate even in the weighted form adapted for imbalanced datasets.
That is why it seems optimal to start with estimating the balanced accuracy, precision, recall and F1score values to improve them further on.The quality measuring method should also be adjusted to compare resulting well-optimised models; its updated version is described in the upcoming sections of the article.A minimum F1-score among all classes was also added to metrics for the quality assessment.

Figure 2. Resident space objects target class distribution
As has already been mentioned, the random forest classifier is going to be used both to get a preliminary classification quality assessment and to estimate the importance of the data features, as it can help to remove unimportant columns from the data.For this purpose, the dataset is divided into training and validation parts in a ratio of 3:1.Precisely, the training dataset includes 13079 records, and the test one -6442 records.This model was fitted on the training part of the dataset, and after that metrics were calculated on the model's predicted classes on the validation set.The balanced metric values are as follows: accuracy ~ 0.794, precision ~ 0.927, recall ~ 0.929, F1-score ~ 0.824 As expected, even the first chosen model looks effective enough, but it is necessary to compare the random forest classifier to other basic ML models to check this hypothesis.For this purpose, linear balanced models (SVM and logistic regression) and other tree-based models (a decision tree and gradient boosting classifiers -AdaBoost and CatBoost) were also trained and tested, and the metrics received on the validation dataset are shown in table 1.Indeed, all tree-based methods are superior in the classification quality but not in training speed.Thus, only tree-based balanced models will be optimised further on to achieve a better quality of RSO imbalanced classification to make the space objects identification process more accurate and faster.Returning to the model optimization issue, unnecessary feature removal is a way to speed up the process of training without any decrease in quality.A fitted instance of the random forest classifier provides weighted coefficients for all data features, which represent the importance of each feature for this model.After checking the quality of a model trained on a cropped dataset, only 9 most important features of all 24 were left.It is noticeable that all these features relate to the object dynamics or orbit characteristics, regardless of the observation time.
After that, several hyperparameters of the chosen tree-based models (random forest and CatBoost as models with the highest metric values) are optimised using the grid search and cross-validation procedures.However, the CatBoost classifier model is already more complicated and the grid search selection is less suitable in this case, whereas other methods of automated ML model optimisation seem to be more efficient, such as hyperopt or optuna frameworks.Thus, the CatBoost classifier was tuned using the optuna library, unlike the previous models.Finally, all resulting classifiers should be compared more accurately.

Model comparison
All previously chosen metrics were calculated for the random forest and CatBoost models, and it turns out that CatBoost is slightly superior in quality, according to the metrics shown in table 2. However, it seems that this method of the model comparison is insufficient at this stage, and a more detailed analysis of the model predictions should be performed due to the high imbalance in the dataset which can affect metric values.For this purpose, a full classification report by target classes and confusion matrix is generated for each model, shown in figure 3. The classification report from sklearn includes precision, recall and F1-scores calculated for each class separately as well as their weighted and mean values.In general, this report indicates whether any problems with individual RSO types exist.For instance, the classification report for the random forest shows little recall and F1-score for the small CZ and LEMUR classes, whereas the CatBoost model demonstrates more stable metric values with a slight increase in average metric values.A confusion matrix is a matrix of shape 12 x 12, where each cell with an index [TYPE_A, TYPE_B] contains a number of records related to objects of TYPE_A which are predicted by the model to be TYPE_B objects.Therefore, this matrix does not only visualise possible problems with individual classes similar to the classification report but also indicates which classes are confused by the model.For example, for the CatBoost classifier, sometimes it is difficult to distinguish small classes from the major DEBRIS class, in keeping with figure 4. But overall values on the major diagonal of the matrix are significantly greater than the other ones, which means that the CatBoost classifier model manages to find and learn the differences between the space object classes, and is quite effective and accurate.As a result, a more sophisticated model analysis shows that more complex models perform better solving the imbalanced classification problem by heterogeneous TLE records data.To be precise, the gradient boosting (CatBoost) model shows fewer systematic errors and is slightly more accurate than the random forest one.Even though the process of training a CatBoost model is more time-consuming, it is optimal to use this model for nearly real-time RSO classification by categories or types because model retraining can be done offline, if necessary.

Unassigned objects classification
In addition to the quality assessment, it is useful to check whether the chosen model's predictions are similar to the expected ones.To be precise, a reliable ML-based classifier should in general act in an understandable way for a human specialist even in extraordinary circumstances.For example, the classifier should not get confused by TLE records of other possible NEO types, even if they contain feature values not from the training data features distribution.Therefore, for testing the reliability and overall adequacy of the classifier, another test is performed with the use of already mentioned data.
In the first stage of data preprocessing only records with assigned identifiers and categories were left for the model fitting and assessment.But the remaining data with the OBJECT or TBA (To Be Assigned) types can be used for checking the model's stability to the unknown data.Due to the fact that these two categories were not presented in the training and validation datasets, the model's quality and performance on this part of the data have not been estimated yet.Thus, predictions for records of both not assigned classes of objects were obtained from the optimised CatBoost classifier model.According to the distribution of the predictions shown in figure 5, the model assumes that most of the TBA objects are pieces of debris and few others are operating satellites from different constellations, whereas the OBJECT ones are mostly assigned to be OTHER objects, which means these records can represent unknown types of objects (such as natural objects).This experiment proves that the resulting model does not get confused with the data which was not indicated in the training dataset, that is why it can be used as a real-time high-accuracy classification application robust to the unknown data.

Conclusion
It is difficult to overestimate the importance of the space debris monitoring problem as it helps to avoid satellite collisions which can lead to the "cascade effect" and other catastrophic consequences for the space programme.For this purpose, RSO catalogue maintenance and updates should be performed on time, and the process of space object identification should be made in a fast and accurate way to forecast its orbit properly.This process is performed while calculating possible correlations of the new object's orbit with simulated or real trajectories of the catalogued objects.Obviously, several methods can be used to speed up this procedure by reducing the number of potential candidates' trajectories.For example, the classification of RSOs by their categories significantly decreases the number of objects which should be compared to a new one.
In this work, a new ML-based method for the classification of RSOs on low Earth orbits by TLE records is presented.Data for the model selection and training was collected from the open-source website.After a full analysis of their TLE records RSOs were divided into 12 categories depending on their type (a piece of debris, a functioning satellite or an unassigned object), a related satellite constellation or series of launches.It was expected that tree-based models should work more effectively with this kind of data compared to other methods because the dataset is highly imbalanced and contains several features given in the degree scale.Indeed, the CatBoost classifier as a tree ensemble turns out to be the most efficient model for solving the RSO imbalanced classification task by analysing TLE records and orbit characteristics calculated from the related records.
The resulting classification model shows more than 93% accuracy and about 83% weighted F1-score on the validation set.Moreover, the high quality of model predictions is proved by calculating the confusion matrix and the classification report.For the final test for compliance of model predictions with expectations and ideas the classifier was retrained on all records with assigned categories and forecasts of the type for the unknown objects were obtained.As was expected, the OBJECT ones were mostly classified as debris and the minor part of them are classified as satellites of different constellations, whereas the TBA (To Be Assigned) objects were predicted by the model to be other objects, neither debris defunct satellites.Therefore, the proposed model learned effectively to distinguish RSOs of different categories and all of them from other space objects and can be used effectively for the real-time RSO classification to speed up the space object identification process.

Figure 1 .
Figure 1.Correlation matrix of numerical features from the RSOs dataset.

Figure 3 .
Figure 3. Classification reports for the random forest and CatBoost classifiers.

Figure 4 .
Figure 4. Confusion matrix for the CatBoost classifier predictions on the validation dataset.

Figure 5 .
Figure 5. Classification estimations for the unassigned objects from the RSOs dataset.

Table 1 .
Baseline classification weighted metrics for different models.

Table 2 .
Classification metric values for the optimised models.