Selection of Three (Extreme)Ultraviolet Channels for Solar Satellite Missions by Deep Learning

Over the past 25 yr, ultraviolet (UV) and extreme UV (EUV) phenomena of the Sun have been continuously observed in multi-wavelengths. The Solar and Heliospheric Observatory (SOHO; Domingo et al. 1995), launched in 1995, observed the low corona in four channels of 171, 195, 284, and 304 Å of the Extreme ultraviolet Imaging Telescope (EIT; Delaboudinière et al. 1995). The Transition Region and Coronal Explorer (TRACE; Handy et al. 1999), launched in 1998, provided observations from the solar photosphere to the upper atmosphere in seven channels of 1700, 1600, 1550, 1216, 284, 195, and 171 Å. The Extreme Ultraviolet Imager (EUVI; Wuelser et al. 2004; Howard et al. 2008) on board the Solar Terrestrial Relationships Observatory (STEREO; Kaiser et al. 2008), launched in 2006, has made it possible to observe the farside of the Sun in 171, 195, 284, and 304 Å, similar to SOHO/EIT. Launched in 2010, the Atmospheric Imaging Assembly (AIA; Lemen et al. 2012) on board the Solar Dynamics Observatory (SDO; Pesnell et al. 2012) has provided high-resolution full-disk images with high time cadences of about 12 s. The AIA has nine channels of 1700, 1600, 335, 304, 211, 193, 171, 131, and 94 Å.

Such a large amount of single- or multi-channel paired data allows us to do various applications of deep learning (DL), one of the artificial intelligence (AI) models. First, super-resolution methods that enhance the original resolution of data have been developed (Díaz Baso & Asensio Ramos 2018; Jia et al. 2019; Rahman et al. 2020). Second, reconstruction and synthesis of data have been considered (Felipe & Asensio Ramos 2019; Galvez et al. 2019; Kim et al. 2019; Park et al. 2019; Salvatelli et al. 2019; Szenicer et al. 2019; Jeong et al. 2020; Shin et al. 2020; Zhang et al. 2020; Lee et al. 2021a). Third, processes of reducing noise in data have been proposed (Díaz Baso et al. 2019; Park et al. 2020). Fourth, predictions of future data have been studied (Galvez et al. 2019; Ji et al. 2020; Lee et al. 2021b).

There are two types of approaches in the reconstruction and synthesis of solar data. First, several authors have studied the translation between data from currently operated instruments. Galvez et al. (2019) translated Helioseismic and Magnetic Imager (HMI) vector magnetograms into AIA nine-channel observations. Park et al. (2019) also generated AIA nine-channel images using HMI line-of-sight magnetograms. Szenicer et al. (2019) mapped from AIA images to spectral irradiance measurements. Salvatelli et al. (2019) reconstructed one AIA channel from three other channels among 94, 171, 193, and 211 Å. Second, there have been several studies using DL models to compensate for the absence of observational data. Kim et al. (2019) generated farside magnetograms from STEREO/EUVI 304 Å. Felipe & Asensio Ramos (2019) developed a method to improve the quality of farside seismic maps by using HMI magnetograms and farside phase-shift maps. Jeong et al. (2020) constructed extrapolated global magnetic fields using observed frontside and DL-generated farside magnetograms from STEREO/EUVI 171, 195, and 304 Å. Shin et al. (2020) generated past HMI-like magnetograms using Ca ii K images from the Rome Observatory. Lee et al. (2021a) generated HMI-like magnetograms and AIA-like (E)UV images in 1612 using Galileo sunspot drawings. Zhang et al. (2020) generated Nobeyama Radioheliograph image from SDO/AIA 171, 193, 211, 304, and 335 Å.

The above studies presented a sufficient possibility of a "virtual observatory" that can generate real-like observational data sets (Salvatelli et al. 2019). In this study, we address a question of what combination of channels can best translate other channels in UV and EUV observations. For this, we use data from the nine SDO/AIA channels. Here, we consider three types of DL models: (1) 72 models with single-channel input and single-channel output, (2) 56 models with double-channel input and single-channel output, and (3) 42 models with triple-channel input and single-channel output. We also compare the performance of these 170 models in view of a metric. This Letter is organized as follows. In Section 2, we describe the data and method. Section 3 gives the results of the translations with discussion. A brief summary and conclusion are given in Section 4.

SDO/AIA is so far the most suitable instrument to consider a large number of multi-(E)UV observations. Thus, we use the AIA observations in two UV channels (1600 and 1700 Å) and seven EUV channels (94, 131, 171, 193, 211, 304, and 335 Å) with 12 hr cadence from 2011 to 2017. These nine channels observe from the photosphere to the corona and cover the plasma temperature between about 5000 K and 20 MK (Lemen et al. 2012). All data are pre-processed by calibrating, rotating, centering, exposure correction, and degradation correction as mentioned in Park et al. (2019). All nine-channel data are set to have the same solar disk size, scaled down to 1024 by 1024.

Considering the solar cycle phase, we divide the data set into training, validation, and test as follows. Data for September and October of each year are used for the test, and data for July and August of each year for the validation, thereby separating the training and test sets. The remaining data are for training. The number of data per channel for training, validation, and testing are 3,299, 825, and 821, respectively.

We adjust the dynamic range of each wavelength using the aia_intscale. pro routine to make physically important features of the Sun stand out well at each wavelength. For this reason, the model outputs could be used for quantitative study within the limited dynamic range. Note that it takes much time to set up 170 DL models with full dynamic ranges, which is reserved for a future study. After the dynamic range is adjusted for each channel, all the pixel values are scaled by 255. Then, they are considered as the input of the model. The output of the model is data with values between 0 and 255 like the input. Between inputs and outputs, the data is scaled between −1 and 1 and processed. The output data is inversely transformed to the dynamic range for each channel before calculating a metric in the test set.

In this study, three types of models are considered as follows. We first consider models that translate from single-channel input to single-channel output. The number of single-channel models for nine channels is 72, which can be calculated using a permutation that selects two out of nine elements without overlapping, ₉ P₂. Among the single-channel models, we select one channel, referred to as Channel-A, that best translates the other channels on average. Second, we consider models that translate from double-channel input to single-channel output while keeping Channel-A as an input. The number of double-channel models is 56, ₈ P₂. Then, among the double-channel models, we select one combination, referred to as Channel-A+Channel-B, showing the best performance. Lastly, we consider models that translate from triple-channel input to single-channel output while keeping Channel-A and Channel-B as an input. The number of triple-channel models is 42, ₇ P₂. A total of 170 models are trained for this study.

We consider a DL method called "Model B" in Park et al. (2019) based on pix2pix of Isola et al. (2017) with two networks "generator" and "discriminator". A detailed description of this method is presented in GitHub ³ (Park 2019) and the Appendix in Park et al. (2019). We use the same hyperparameters as in Appendix C of Park et al. (2019). Because this method is for single-channel input, we modify it to be able to input multiple channels for this study.

We compare 170 models using the average pixel-to-pixel correlation coefficient (CC) between DL model outputs and the corresponding real AIA images within the solar disk for the test data set. The first to the ninth rows in Table 1 shows the average CC values for single-channel models. The rms errors between the CC value of each test data and the average CC value are represented as uncertainties. The channels that best translate 94, 131, 171, 193, 211, and 335 Å are 335, 171, 131, 211, 193, and 211, respectively. They are all EUV channels. This is not surprising in that the peak temperatures of 131 and 171 Å (about 0.4 MK at lower peak temperature and 0.6 MK, respectively), and 193 and 211 Å (about 1.6 MK and 2 MK, respectively) in the response function are almost similar to each other (Lemen et al. 2012). Likewise, each UV channel has the best performance to generate the other UV channel. The 304 Å has a higher average CC with the EUV channels than the UV channels. Among the nine channels, 131 Å is considered as the Channel-A as it gives the highest average CC. Another interesting result is as follows. The average CC (0.59) for the translation from the higher atmospheric channels (94, 131, 171, 193, 211, and 335 Å) to the lower atmospheric channels (1600 and 1700 Å) is lower than that (0.78) for the translation from the lower atmospheric channels to the higher atmospheric channels. The result implies that observations from the lower atmosphere of the Sun may have more information to infer the characteristics of the higher atmosphere. This can be understood from the origin of the solar activity coming from lower atmosphere.

The tenth to the seventeenth rows in Table 1 shows the average CC values with their uncertainties for double-channel models with 131 Å and another channel as inputs. The performances of the models are much better than the single-channel models. The average CCs of single-channel models with EUV channels are distributed between 0.8 and 0.84, whereas those of double-channel models are between 0.9 and 0.91. Since the combination of 131 and 1600 Å best translates other channels on average, 1600 Å is selected as the Channel-B. The eighteenth to the last rows in Table 1 show the average CCs for triple-channel models with 131, 1600 Å and another channel as inputs. The performances of these models are all similar or better than those of the single- and double-channel models. The triple-channel models with the input including 171, 193, 211, 335, and 304 Å give the highest average CC of 0.97. Among these, the model with 131, 1600, and 304 Å is selected as the best model because it has the highest minimum CC (0.96).

Interestingly, the three channels are representative coronal, upper photospheric, and chromospheric channels, respectively. Of the three channels, the 1600 Å has been considered in TRACE and SDO, and 304 Å has been considered in SOHO, STEREO, and SDO (Delaboudinière et al. 1995; Handy et al. 1999; Howard et al. 2008; Lemen et al. 2012). However, 131 Å is selected first by the SDO (Lemen et al. 2012). The reason why these three channels are selected from the perspective of the metric could be understood as follows. The 131 Å is selected first as it can observe not only the transition region but also the corona, i.e., it has double peaks at low and high temperatures (about 0.4 and 10 MK) in the response function. The 1600 Å is selected second because it can generate the 1700 Å much better than the other channels and is the most unpredictable channel among the double-channel models. The 304 Å seems to be considered third because it is also the most difficult channel to be translated from the other channels of AIA among the triple-channel models. There is a possibility that the three prime channels could be selected even with the average CC results of the single-channel models. Because of the substantial difference between UV and EUV channels, one channel in the UV needs to be the second channel. 1600 Å can be better transferred to 1700 Å than the reverse, therefore 1600 Å is selected. Then, in order to choose the third channel among the rest of the channels, one just needs to choose the channel with the lowest average CC (0.8) column-wise (i.e., harder to be reproduced by other channels) in Table 1, which here are 171 and 304 Å. It is difficult to select one out of 171 and 304 Å by CC value alone, but 304 Å could be selected to represent the chromospheric activity. In terms of qualitative scientific viewpoints, another combination could be selected. For example, if a scientific target is coronal holes (CHs), then the 193 Å is preferable as the first selection channel because CHs are well detected in 193 Å (O'Dwyer et al. 2010; Caplan et al. 2016). The remaining two channels can be determined in a similar way as described above.

We compare real images and AI-generated ones by the following three models: the model with single-channel (131 Å) input (hereafter Model 1), the model with double-channel (131 and 1600 Å) input (hereafter Model 2), and the model with triple-channel (131, 1600, and 304 Å) input (hereafter Model 3). Figure 1 shows the full-disk EUV images at 00:00 UT on 2014 October 31. It is found that the EUV channels are well translated in view of the location and shape of the overall structures, even if the single-channel input is used. In addition, we compare local regions of interest such as active regions (ARs) and CHs. For this, we consider 193 and 171 Å, where CHs and coronal loops of ARs are most visible, respectively. We select 82 CHs detected by the spatial possibilistic clustering algorithm (SPoCA; Verbeeck et al. 2014) during the test period. 90 ARs detected as HMI Active Region Patch (HARP; Bobra et al. 2014) are selected during the test period between 2012 and 2014, corresponding to the solar maximum phase. The average CC within the 90 CHs is 0.84 for Model 1, 0.88 for Model 2, and 0.91 for Model 3. For 90 ARs, the average CC is 0.89 for Model 1, 0.93 for Model 2, and 0.95 for Model 3. These results show that Model 3 has performs better translations for the ARs and CHs than Model 1 and Model 2.

Zoom In Zoom Out Reset image size

The first and second rows in Figure 2 shows the full-disk images and the CH regions (SPoCA 2453) of the AIA 193 Å at 00:00 UT on 2011 September 11. It is shown that Model 3 better translates the overall size and shape of the CH region than Model 1 and Model 2. The average CC within the CH region is 0.92 for Model 1, 0.92 for Model 2, and 0.96 for Model 3. As in Table 1, the single-channel that best translates 193 Å is 211 Å. Thus, 211 Å may give a better reconstruction of CHs. The third and fourth rows in Figure 2 shows the full-disk images and the AR regions (HARP 3119) of the AIA 171 Å at 00:00 UT on 2013 September 3. Compared to the results of Model 1 and Model 2, the result of Model 3 gives more similar intensities to the observed intensities and shows more clear coronal loops. The average CC within the AR is 0.89 for Model 1, 0.96 for Model 2, and 0.97 for Model 3. Salvatelli et al. (2019) investigated translations from three channels into one channel among the AIA 94, 171, 193, and 211 Å. They showed that the translation error for 171 Å out of four channels is the lowest. Thus, an input combination of 94, 193, and 211 Å using our DL method may be better for 171 Å translation than Model 1, 2, and 3. Figure 3 shows the comparison between cutout real images and AI-generated ones by the three models for the AIA 1700 Å observations at 00:00 UT on 2014 October 31. The result of Model 1 does not well translate the location and size of a sunspot and faculae, but these features are greatly improved in the results of Models 2 and 3 that consider 1600 Å as the input. The average CC within the region is 0.68 for Model 1, 0.98 for Model 2, and 0.98 for Model 3.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

In order to demonstrate the performance of the DL model, we consider a simple base model that is a multiple linear regression of the intensities in log scales for the 131, 1600, and 304 Å as follows.

$$\begin{eqnarray}&&\begin{array}{l}\mathrm{log}{I}_{\mathrm{pred},\mathrm{base}}=a\mathrm{log}{I}_{\mathrm{obs},131}+b\mathrm{log}{I}_{\mathrm{obs},1600}\\ \,+\,c\mathrm{log}{I}_{\mathrm{obs},304}+d,\end{array}\end{eqnarray} \tag{ 1 }$$

where I is the intensity of the pixel, a, b, and c are a slope, and d is a constant of a regression. The data used to make the base model are pre-processed and scaled down to 1024 by 1024 as in the DL models, but have a full dynamic range for each channel. The data for the base model are chosen with 10 days cadence in the training data set since the use of all data are too much time consuming. For comparison with the base model, we develop a DL model using data with the full dynamic range. This model only consider the input combination of 131, 1600, and 304 Å (Model 3 that gives the highest CC among the DL models). Hereafter, Model 3 refers to the DL model considering the data with the full dynamic range.

For the quantitative comparison of the two-dimensional distribution, we consider the joint probability density function (JPDF) of the observed intensities (I_obs) and the model predicted intensities (I_pred) as suggested in Salvatelli et al. (2019). The JPDF is a function used to characterize the probability distribution of a finite domain ΔI_obsΔI_pred. The sum of JPDFs in all domains is 1. If the model predicts perfectly, the JPDF would be lie along the diagonal line.

Figure 4 shows the JPDF of the observed intensities and model predicted intensities in the test set for the base model and Model 3. For all six channels, the results of Model 3 are much more distributed near the diagonal line than those of the base model. This indicates that Model 3 can translate the intensity more precisely within the corresponding range than the base model. To quantify the distribution, we calculate the sum of JPDFs along the diagonal (∑_i=j JPDF(i, j)). A higher value means better translation within the observed intensity range. For the base model, ∑_i=j JPDF(i, j) are 0.18, 0.2, 0.16, 0.11, 0.17, and 0.42 for 94, 171, 193, 211, 335, and 1700 Å, respectively. For Model 3, ∑_i=j JPDF(i, j) are 0.2, 0.29, 0.32, 0.31, 0.23, and 0.52 for 94, 171, 193, 211, 335, and 1700 Å, respectively. It can be found that the performance of the DL model is better than the base model as shown in Figure 4. Among the six channels of Model 3 and the base model, the 1700 Å has the higher sum of JPDFs along the diagonal line. In this case, the average CC (0.98) is also the highest, as shown in Table 1.

Zoom In Zoom Out Reset image size

Although the DL model outperforms the base model in reproducing the UV and EUV images, the accuracy is not good enough to fully replace the real measurements based on the quantitative comparison shown in Figure 4. To be useful for reducing the telemetry of future solar missions, the DL model accuracy has to be improved. The current DL model may be further improved by considering the following. The first is to use the data of SDO/AIA with the original size 4096 by 4096. Then the information loss caused by binning could be reduced. The second is to apply the "pix2pixHD" method that is specifically devised for high-resolution image translation tasks (Wang et al. 2017). In addition, another DL method that can precisely translate the coronal loop strand should be considered in the future.

In this Letter, we have addressed the question of what combination of channels can best translate other channels in UV and EUV observations. For this, we considered the SDO/AIA observations in two UV channels (1600 and 1700 Å) and seven EUV channels (94, 131, 171, 193, 211, 304, and 335 Å) with 12 hr cadence from 2011 to 2017. We developed 170 DL models based on pix2pix: 72 models for single-channel input, 56 models for double-channel input, and 42 models for triple-channel input. All models have a single-channel output. To quantitatively compare the image translations, we evaluated the model results by pixel-to-pixel CCs within the solar disk. The real images and AI-generated ones were also presented to qualitatively compare the image translations.

The major results of this study are as follows. First, the model with 131 Å gives the highest average CC (0.84) among the single-channel models. Second, the model with 131 and 1600 Å shows the best translation (average CC = 0.95) among double-channel models. Third, among the triple-channel models with the highest average CC (0.97), the model with 131, 1600, and 304 Å has the highest minimum CC (0.96). The model with 131, 1600, and 304 Å better generates not only the full disk of the Sun but also local regions (e.g., CHs and ARs) than the model with 131 Å and the model with 131 and 1600 Å. The average CC (0.98) for the 1700 Å translation by the model with 131 and 1600 Å is more significantly improved than that (0.68) by the model with 131 Å. The DL model with 131, 1600, and 304 Å translates the intensity more precisely than the base model using the multiple linear regression with the same combination of channels.

In this study, we compared 170 DL translation models using 24 combinations of input channels. From this, we presented the combination of channels that best translates the other channels. Our results may be used as a secondary perspective in addition to primary scientific purposes in selecting a few channels of an UV/EUV imaging instrument for future solar satellite missions. Our study could help design a DL model for a specific scientific target. We focused on the image translation performance depending on input combinations using one DL method. According to scientific targets, another DL method using multi-channel input may be considered (e.g., a DL method for translation of the coronal loops). Although a large number of models have been developed and compared, we do not consider all input combinations. Thus, future studies of other combinations not dealt with in this study may be needed. As mentioned in Section 3, the current DL model could be improved in the future, and it will show the possibility of quantitative analysis such as differential emission measure.

We thank the reviewer for the helpful comments. This work was supported by the Basic Science Research Program (BSRP) through the National Research Foundation (NRF) funded by the Ministry of Education (NRF-2019R1A2C1002634), the Korea Astronomy and Space Science Institute (KASI) under the R&D program (project No. 2021-1-850-05) supervised by the Ministry of Science and ICT, and Institute for Information & communications Technology Promotion (IITP) grant funded by the Korea government (MSIP) (2018-0-01422, Study on analysis and prediction technique of solar flares). D.L. acknowledges support from the BSRP through the NRF funded by the Ministry of Education (NRF-2021R1I1A1A01040372). E.P. acknowledges support from the BSRP through the NRF funded by the Ministry of Education (NRF-2020R1C1C1003892). J.Y.L. acknowledges support from the BSRP through the NRF funded by the Ministry of Education (NRF-2020R1I1A1A01071814) and the Air Force Office of Scientific Research under award number FA2386-20-1-4031. We thank the numerous team members who have contributed to the success of the SDO mission. We acknowledge the community effort devoted to the development of the following open-source packages that were used in this work: NumPy (numpy.org), Keras (keras.io), and SolarSoftWare.

Facility: SDO/AIA (Lemen et al. 2012).

Software: SolarSoftWare (Freeland & Handy 1998), Keras, NumPy (van der Walt et al. 2011; Harris et al. 2020), solar_euv_generation (Park et al. 2019).