Prediction of Convective Available Potential Energy and Equivalent Potential Temperature using a Coupled WRF and Deep Learning for Typhoon Identification

To predict typhoons in the western North Pacific Ocean, it is required to predict the determinants of typhoon activities. The formation of the typhoon can be controlled by Convective Available Potential Energy (CAPE) and Equivalent Potential Temperature (theta-e). To predict the variables, a mesoscale numerical model of Weather Research and Forecasting (WRF) can be used. However, the output of WRF needs to improve to obtain a more accurate CAPE and theta-e prediction. This study uses a coupled WRF model and Deep Learning (DL) Multilayer Perceptron Regressor approach to increase CAPE and theta-e prediction skills. Simulation with dataset scenarios with WRF outputs as predictors and sounding data as predictors are developed and tested to obtain the most appropriate package of deep learning simulation. The study found that coupled models provide increased mean accuracy of theta-e and CAPE, namely 16.6% and 32.0% higher than using original WRF, respectively. This study also shows the difference of skill scores in the spatial distribution of CAPE and theta-e of WRF result and its coupled model.


Introduction
Typhoon is one of the hydrometeorological hazards [1] threatening life and disrupting economic activity.The impact of typhoons is very detrimental, especially to inhabitants in coastal areas.Especially in the western North Pacific region, there are more than 5 typhoons that occur in a year.Therefore, this area is known as the most active typhoon occurrence area in the world [2], [3].To reduce the risk of this impact, accurate early identification of typhoons as early warning [4] is needed.Typhoon prediction accuracy will greatly determine the strategy and priority of anticipatory steps in potential areas.Several studies have shown that typhoons can be identified using two variables, namely the equivalent potential temperature (theta-e) and convective available potential energy (CAPE) [5]- [10] as stated in [11].Theta-e is the temperature that an air parcel would reach to diagnose atmospheric instability.CAPE measures the fuel potential for a thunderstorm to form by estimating the strength of an upward current within a thunderstorm and describes the instability of the atmosphere.For early identification of typhoon, both variables can be predicted using a numerical prediction.However, due to several limitations of the numerical model, such as extreme conditions which occasionally cannot be represented by physical equations, low resolution, and inaccuracy in determining the initial and boundary values, the accuracy of these two variables still needs to be improved.One possible solution is the use of deep learning (DL) to improve the numerical model.However, the hyperparameter of the deep learning should be determined to get the appropriate model configuration for each variable.
The increasing number of deep learning models are being considered for weather forecasting due to their well capability to address with important and complex nonlinear problems [12].Previous studies have conducted experiments using a combination of DL and numerical models, particularly weather research and forecasting (WRF), to predict the weather.DL-based parameterizations were successfully conducted to WRF model [13].Hybrid downscaling by WRF and Convolutional Neural Network (CNN) model to get the precipitation in higher resolution [14].Wind signal prediction was produced by using a hybrid of WRF and deep learning neural network [15].The use of machine learning has also been carried out to improve the output of the WRF for wind speed values [16].To the best author's knowledge, there is a lack of study in discussing the combination of machine learning and WRF to get the theta-e and CAPE prediction.
This study aims to develop a prediction framework that combines Weather Research and Forecasting (WRF) and Deep Learning (DL) as a coupled WRF+DL model to produce theta-e and CAPE predictions with better accuracy than using the original WRF.Several steps have been taken to achieve this goal, including WRF simulation of past typhoons, calculation of theta-e and CAPE values, deep learning optimization for the coupled WRF+DL model, and evaluation of model performance.
The discussion of the paper is divided as follows.The data and method is described in Section 2. Section 3 discusses the results and discussion.The conclusion is given in Section 4.

Data
The coarser meteorological forecast of the Global Forecast System (GFS) has been used for input of WRF model [17].Typhoon track from Japan Meteorological Agency (JMA) is used to see the position of typhoons and test the identification of typhoons through the position of CAPE and Theta-e values.The typhoon track data can be accessed at Digital Typhoon: Typhoon Images and Information managed by National Institute of Informatics (NII) [18].Sounding data from Wyoming university [19] which is used as a target in deep learning.There are 14 sounding stations spread across in the western North Pacific region.

Method
This study aims to produce theta-e and CAPE predictions that have better accuracy by using coupled WRF+DL in identifying typhoons.Several steps were carried out to achieve this goal, including WRF simulation, calculating theta-e and CAPE values, deep learning optimization for the coupled WRF+DL model, and testing the prediction results.

Configuration and simulation of WRF for CAPE dan Theta-e calculation.
In this study, the regional Weather Research and Forecasting (WRF) model version 4.2 [20] is run for periods of 15 typhoon occurrences with double-nested modeling system.The WRF model is used to 3 downscale GFS data to get the higher resolution based on the double-nested domain.The mother domain with 27 km spatial resolution is centered at 28.686 o N, 127.753 o E, and finer resolution domain (i.e., 9 km) is nested with the same center location (see Figure 1).

Figure 1. Domain for WRF simulation
The physical parameterization in the model input is adjusted with previous studies performed in the western North Pacific region [21].The scheme comprises of Kain-Fritsch convective parameterization scheme [22], WRF single-moment 3-class microphysics (WSM3) scheme [23], the Revised MM5 surface layer scheme [24], NCAR Community Atmosphere Model (CAM 3.0) radiation scheme, Yonsei University (YSU) planetary boundary layer scheme, and Noah land surface model.The same configuration is applied for the mother and the nested domain.
To accommodate data that includes the presence or absence of typhoons, the dataset is compiled from the results of WRF simulations and sounding on 15 past typhoon events that pass through the western North Pacific region.The WRF output is used to calculate the theta-e and CAPE values using equation 1 and equation 2 [11].The two variable values from the WRF are as predictors and sounding observations are as predictand.Therefore, the WRF grid was chosen at a location adjacent to the sounding location.There are several sounding stations that have been passed/approached by the typhoon path and some other areas have not.To calculate CAPE, equation ( 1) is used.
where Zf and Zn are the height of the level of free convection and equilibrium level (neutral buoyancy), Tv,parcel and Tv,env is the virtual temperature of the specific parcel and the virtual temperature of the environment, respectively, R is the gas constant and g is the acceleration due to gravity.
where θe is the equivalent potential temperature, Te is equivalent temperature, p0 is a reference pressure (usually 100 kPa), p is the partial pressure of dry air, T is the temperature, Lv is the latent heat of vaporization, Cpd is the heat capacity at constant pressure of dry air, r is the total water mixing ratio, c is the heat capacity of liquid water and Rd is the gas constant for dry air.For theta-e calculations, several main variables that can be taken from the WRF output include air temperature, relative humidity, air pressure.

Deep learning.
The deep learning algorithm used in this study is the Multilayer Perceptron (MLP) by using scikit-learn as python library [25].MLP is a supervised learning algorithm that can learn a number of input dimensions to produce a number of output dimensions through a feature set X=x1,x2,...,xm and target y which is processed by one or more hidden layers [26].In this study, MLP will generate the improved accuracy of CAPE and theta-e prediction based on predictors generated by WRF.Because the supervised algorithm is used, the target is needed by taking data from the sounding.The target is required since the supervised algorithm uses data from the sounding.Correction weight values are determined automatically during MLP simulation iterations.To get the optimal configuration of the MLP algorithm, hyperparameter tuning is needed as shown in Figure 2.However, since the number of data is large, a python script is needed that can automatically be used by an iterative process to find the optimal hyperparameter configuration for predicting theta-e and CAPE values that have a better accuracy.To prevent indefinite looping, upper and lower threshold values for each parameter have been established.
There are several parameters required to obtain the optimal prediction, namely activation, solver, alpha, hidden layer size, and learning rate.The output of a node is determined by its activation function given an input or set of inputs.In MLP, the available activation function consists of identity (linear), logistic, heaviside (tanh), and relu (rectified linear unit).The SGD method uses the proper smoothness technique to iteratively reduce the value of the loss function.LBFGS, a member of the family of quasi-Newton algorithms, uses a small amount of computer memory to approximate the Broyden-Fletcher-Goldfarb-Shanno algorithm (BFGS).On unconstrained input values, LBFGS minimizes the loss function, which is a differentiable scalar function.The regularization term uses the alpha parameter to choose weights with large magnitudes for preventing overfitting.
The hidden layer size is determined to identify weighted dimensions that will be used in the output value computation procedure.The size of the hidden layer demonstrates the algorithm's backpropagation's time complexity.The step-size in the space search parameter scheduled using the chosen criteria, such as constant, adaptive, or invscaling, is controlled by the learning rate.

Evaluation of CAPE dan Theta-e prediction
The prediction of CAPE and theta-e is evaluated using Pearson's correlation for skill scores.Comparing the visualization of the vertical profile for theta-e among deep learning results, observation, and WRF is another method of evaluation.Additionally, both theta-e and CAPE's temporal variation are used in the comparison.

Result and Discussion
This study examines the ability of coupled WRF+DL to predict theta-e and CAPE which are used to identify typhoons in the future.There are 3 important discussions including testing the prediction results, vertical profile analysis of theta-e values, and temporal analysis for theta-e and CAPE values using the Bavi 2020 typhoon case.After going through an optimization process in using the WRF+DL to predict theta-e and CAPE, the most appropriate configuration is obtained for use, as shown in Table 1.To get the optimal result, the learning rate, solver, and activation parameters for the two variables (theta-e and CAPE) should be the same, namely logistic, lbfgs, and constant, respectively.However, there are also different hidden layer and alpha sizes, including (1) 0.22 and (64,) for theta-e variables, and (2) 0.2 and (32,) for CAPE.Each chosen parameter is examined.Logistics is the proper activation value for both parameters.This indicates that there is a chance that the two variables (theta-e and CAPE) will have data patterns that resemble a logarithmic curve.For weight optimization for both atmospheric variables, lbfgs is the proper solver.The prediction accuracy of CAPE and Theta-e can be increased by fitting the dataset with the quasi-Newton method employed in lbfgs.The "lbfgs" solution does not require the learning rate parameter, hence it was not chosen.
The difference in alpha and batch size values between the two variables is not large.Small modifications, on the other hand, can have considerably different results.Because the alpha value prevents overfitting in the model, also known as L2 regularization, the alpha value of each data series of a variable will vary depending on the nature of the data.
The batch size parameter specifies the amount of training samples gathered from the first to the last sample number.Processing a smaller quantity of data compared to the overall data becomes valuable when memory is limited.A batch size that is excessively small, on the other hand, will result in less reliable predicted results.Because the entire amount of data trained in this study is 3920, the effective batch size for 10 iterations of training is 392.
There are no standard criteria for choosing hidden layer size because it is determined by the complexity of the problem, the amount of data, and the model architecture rather than the number of features.As a result, numerous possibilities were considered in order to reach the best values, namely 64 for theta-e and 32 for CAPE.Overfitting can occur when hidden layer sizes are excessively large.As a result, in addition to calculating the proper hidden layer size, alpha determination is also considered at this stage.For CAPE, each station has one score as seen Figure 4.The highest value is shown at station 47104 with a score of 0.84 after using the coupled model.Previously, WRF could only predict CAPE with a score of 0.70 at the same station.The prediction accuracy score of CAPE is generally higher than the prediction accuracy of theta-e.The lowest score of WRF+DL is 0.23 after improving from 0.14 reached at station 47909.The mean accuracy of WRF+DL use is 0.54, while mean accuracy of WRF use is 0.41.
The stations inside the red box are located on the ocean offshore with lower score compared to the other stations that are located near coastal regions (see Figure 4).The coastal regions benefit from various land surface features that can influence weather patterns, such as mountains, forests, and urban areas.These features can create local variations in temperature, moisture, and wind patterns, which contribute to convective initiation (CI).In contrast, ocean offshore areas lack such significant surface features, leading to a relatively homogeneous environment that may not promote the same level of convective initiation.The ocean offshore contributes to the convection propagation (CP) after the triggering of coastal CI [27].The pattern of convection propagation tends to be more complex than CI activity since CP refers to the movement, growth, and evolution of existing convective systems.Once convective storms have initiated, their behavior becomes more complex and dynamic.This condition can generally cause the model to be difficult to predict the CAPE.Based on data from all stations and a variety of altitude levels, we conclude that the mean accuracy of theta-e prediction using WRF+DL is 16.6% higher than using original WRF, and the mean accuracy of CAPE prediction using WRF+DL is 32.0%higher than using original WRF.For both variables, the mean accuracy improvement is 24.3%.This percentage demonstrates the considerable improvement in accuracy with the use of paired WRF+DL.However, certain accuracy scores at some locations are still extremely low, and it is difficult to enhance using this methodology considerably.This issue arises since we generalize the DL hyperparameter configuration to all sounding locations, which is unable to resolve certain particular cases.
In this study, the typhoon "Bavi 2020" was used to see the results of deep learning in improving WRF prediction results for theta-e and CAPE.This typhoon was developed on 22 August 2020 at 00 UTC and disappeared on 27 August 2020 at 06 UTC.The lifetime of this typhoon is around 126 hours.Minimum pressure, maximum wind, average speed, travel length of this typhoon is 950 hPa, 157.4 km/h, 18.6 km/h, and 2346 km, respectively.The vertical profile of the equivalent potential temperature (theta-e) before, during and after the typhoon passed the sounding station area of 47918 is shown in Figure 6 by comparing the predicted results of WRF, coupled WRF+Deep Learning, and observations.It can be seen from the three charts that the vertical pattern of the coupled WRF+DL results is generally more similar to the observations compared to only using the WRF model.If seen in Figure 6a, the vertical profile of theta-e is undisturbed.However, the vertical pattern changed with the many variations of values as the typhoon "Bavi 2020" passing through the area near the observation station.The vertical pattern of theta-e began to return to normal condition as the typhoon move away from the station area.Based on the comparison shown in Figure 7a, the time series pattern of the predicted theta-e value generated by the coupled WRF+DL is closer to the observation data than the WRF models.The figure shows that as typhoon Bavi begins to approach, the theta-e value shows an increase.Likewise, the comparison for the prediction of the CAPE value (see Figure 7b) shows the same conditions where there is an increase in CAPE when a typhoon approaches the sounding area.
The results of this study will then be used to continue the development of a typhoon prediction model using the ensemble k-NN algorithm to analyze the atmospheric similarity of current and past typhoons.The initial development of the typhoon track prediction model has been carried out previously based on the closeness to the track from past typhoons [28].

Conclusion
The purpose of this study is to compare the prediction performance of coupled WRF+DL to original WRF for theta-e and CAPE values.The two variables are required for predicting typhoon events, particularly those that occur often in the western North Pacific region each year.This study has three primary components that show the results of the proposed model, examination of the vertical profile of theta-e values, and temporal analysis for theta-e and CAPE using the typhoon Bavi 2020 case.The study results reveal that coupled WRF+DL predictions of theta-e and CAPE improve the original WRF predictions.The mean accuracy of WRF+DL is 16.6% higher than using original WRF, and the mean accuracy of CAPE prediction using WRF+DL is 32.0%higher than using original WRF.The vertical profile of theta-e values and temporal patterns of the two variables (theta-e and CAPE) were also evaluated, with the coupled model producing better results.The study is conducted before, during, and after the typhoon passing through the station area.
The simulations are limited to the use of deep learning only.Therefore, there is an opportunity to improve the accuracy of WRF using other machine learning algorithms.The training dataset also only inlvolves typhoon occurrences affecting the Korea region.As a result, if there is a possibility of different hyperparameter tuning that should be optimized.

Figure 3 .
Figure 3. Skill score for theta-e prediction using WRF and coupled WRF-Deep Learning

Figure 4 .
Figure 4. Skill score for CAPE prediction using WRF and coupled WRF-Deep Learning

Figure 5 .Figure 6 .
Figure 5. Sounding station and typhoon track of Bavi 2020.Track and sounding distribution in (a), and soundings impacted by typhoon in (b) and (c)

Figure 7 .
Figure 7. Temporal variation of Theta-e (a) and CAPE (b) near sounding station of 47918