Storm surge forecasting based on physics-informed neural networks in the Bohai Sea

Physics-informed neural networks (PINN), as a new method of integrating artificial neural networks (ANN) and physical laws, have been considered and applied in the fields of ocean forecasting and ocean research. In this paper, the simplified two-dimensional (2D) storm surge governing equation is introduced into an ANN to establish a PINN-based storm surge forecast model. The numerical simulation results of 14 storm surge events in the Bohai Sea are selected as the PINN training set, and 6.3% of the training set data are randomly selected to reconstruct the storm surge field information. The storm surge reconstructed at each tide station is nearly identical to the storm surge curve simulated by the numerical model, with the root mean square error (RMSE) less than 0.12 m and absolute error of maximum storm surge less than 0.2 m. The analysis of the storm surge field at key moments (storm surge height lager than 1 m) shows that the difference in storm surge field between the PINN reconstruction and the numerical model is generally less than 0.4 m. Two storm surge events in the Bohai Sea are selected as forecast cases, and the same network structure, parameters, and storm surge data assimilation scheme are used for predictions by the ANN, PINN, and numerical model. The results show that compared to the ANN and numerical models, the average relative error of the maximum storm surge predicted by the PINN is reduced by approximately 25%, which significantly improves the forecast accuracy, therefore, the PINN is suitable for storm surge forecasting and research due to its advantages in small sample data training and strong physical meaning.


Introduction
Storm surge is an abnormal rise and fall of seawater due to violent atmospheric disturbances such as strong winds and sudden changes in air pressure (usually catastrophic weather systems such as typhoons and temperate cyclones), causing the tide level in the affected area to rise and fall significantly above the normal tide level.According to the multiyear statistics in the Bulletin of China Marine Disaster, the economic losses caused by storm surges have always ranked first in China marine disasters; therefore, timely and accurate forecasting of storm surges is of great significance and practical demand for coastal economic and social development.
Based on physical governing equations, storm surge numerical forecasting is achieved by finding the spatial and temporal solutions to numerical differentiation and integration schemes.However, numerical differential calculations require considerable computing resources and are very timeconsuming.In addition to numerical calculation, another method is based on neural networks.In theory, a two-layer neural network in machine learning can fully approximate any defined continuous function and solve any nonlinear problem.By directly predicting input-output mappings and bypassing numerical integration, deep learning provides an effective alternative for learning high-dimensional spatiotemporal dynamics from massive data [1].
In recent years, the rise of machine learning has provided new possibilities for numerical forecasting.Machine learning has been applied in climate prediction [2], and many scholars have used machine learning to conduct research and find applications in the field of storm surges.Lei [3] used the recurrent neural network (RNN) to forecast storm surges and found that it can obtain better prediction results than the backpropagation (BP) neural network.Liu [4] used long short-term memory (LSTM) to establish a single-station storm surge nowcasting model, and the model using the wind speed and wind direction process data in the previous 2 hours and the tide level process data in the previous 7 hours has the smallest error in terms of predicting the astronomical tide and storm tide level in the next 1 to 3 hours.Xie [5] used the convolutional LSTM (ConvLSTM) network to conduct storm surge floodplain prediction, and the proposed model can essentially reproduce the results of the numerical model simulation during short-term forecasting even if the boundary conditions, topography, surface runoff, and atmospheric signals are unknown.
Despite the considerable progress made in machine learning, data sparsity remains a major challenge when applying machine learning to solve complex scientific and engineering problems, especially in disciplines such as physics, biology, or Earth systems.In this regard, the physicsinformed neural network (PINN) has become a new direction for solving such problems.The PINN has received extensive attention and exploration since its inception [1], [6]; it has the powerful learning capability of a neural network and can combine the related science background to make the model interpretable.Many scholars [7], [8] have conducted experiments on the Navier-Stokes (N-S) equation under ideal conditions.Snaiki [9] first introduced the PINN into the simulation of typhoon wind fields and used storm parameters as input to establish a PINN model to simulate tropical cyclones; they found that the boundary layer wind field of various tropical cyclones can be accurately and effectively predicted.Beucler [10], [11] applied the PINN to the climate simulation process to solve the energy conservation problem of artificial neural networks (ANN), and good results were obtained.Yao [12] used the PINN to construct the N-S flow field under ideal conditions.
In summary, there are two paradigms for marine environmental forecasting.The first paradigm is numerical forecasting methods, which have the advantage of clear physical mechanisms, the disadvantage is that parameterization methods can only be used for sub scale physical processes, the uncertainty of initial field and driving fields makes it difficult to improve the accuracy of forecasting.The second paradigm is a deep learning method based on big data, which has the advantages of fast prediction speed and close integration with historical observation data, but its biggest drawback is the lack of physical law constraints, leading to ineffective prediction of extreme marine environments.The deep learning model requires big data support, but the sample size of storm surge events is small and the distribution is discontinuous, so pure data-driven deep learning used for storm surge modeling may have limitation.An important scientific problem is how to express the physical equation as a separate loss function of the deep neural network and couple the "physics driven" deterministic storm surge prediction model with the "small sample data driven" deep learning model.
Wind is the primary driving force for storm surge in the Bohai Sea, there is a high correlation between wind and storm surge, the meaning of physical equations is clear [13].However, the frequency of storm surge events is low, and the observed storm surge events belong to small sample data, these two important attributes (small sample data and strong physical meanings) are precisely the advantages of PINN.In this paper, a PINN storm surge model is constructed, the PINN model combine the advantages of two paradigms to reflect the complete physical control equations or partial physical laws and historical storm surge data, storm surge reconstruction and forecasting applications are carried out along the coast of the Bohai Sea.Then, PINN forecasting, ANN forecasting, and storm surge numerical model forecasting results are compared and analyzed, and the advantages and possibilities of storm surge forecasting based on the PINN are investigated.This study aims to improve the accuracy of storm surge forecasting, not to replace numerical storm surge forecasting, but to provide new reference or idea for storm surge forecasting, and better alleviate storm surge disaster in coastal area.

Principles of the PINN
The PINN developed from the traditional ANN, and the network structure is shown in figure 1.The ANN network structure shown in the left panel of figure 1 uses the constructed neural network to form a nonlinear mapping between the independent variable and the dependent variable, while the PINN adds a partial differential equation (PDE), i.e., the physical governing equation for the storm surge, to the ANN (the right panel in figure 1).When the neural network's loss function Loss is minimized, the predicted value is constantly approaching the real value so that the physical governing equation is validated, that is, a neural network related to the physical equation, the PINN, can be obtained.The input elements of the ANN and PINN include longitude x, latitude y, water depth z, time t, zonal wind speed u, meridional wind speed v, sea surface pressure p, and other variables.The output elements include the water speed U along the x direction, water speed V along the y direction, and storm surge value h.Loss is the PINN loss function, Loss h is the ANN loss function, and Loss PDE is the loss function for the storm surge governing equation.

Construction of the PINN storm surge model
For the physical PDE used in the storm surge numerical model, the wind stress term, the pressure gradient force term, and the bottom friction term are generally introduced into the two-dimensional (2D) shallow water motion equation, the motion equation and the continuity equation are given as follows(Eq.(1) ~ Eq. ( 3)): In the above equations, u and v are the horizontal velocities of the storm surge; h is the storm surge; H is the total water depth; g is the acceleration of gravity; ρ w is the density of seawater; and ρ a is the sea surface atmospheric pressure.The first term on the left side of the equation of horizontal motion, ∂ is the pressure gradient force term, and τ s and τ b are the wind stress term and the bottom friction term, respectively.Referring to Wang's research [14], the convection term could be removed from the motion equation, which not only simplifies the equation but also does not affect the storm surge numerical simulation.Considering that the storm surge occurs in the ultra-shallow sea along the coast of the Bohai Sea, the convection term and the pressure gradient term make a relatively small contribution [14], [15].The convection term and the pressure gradient term are adopt the neural network compensation method in the study, and D 1 and D 2 are the compensations of the motion equation in the directions of x and y , respectively.Using the output of the neural network, we construct the following equations (Eq.(4) ~ Eq. ( 6)): = When there is a solution such that F_u, F_v, and F_h are always 0, thus satisfying the simplified physical governing equation, the PINN-based storm surge can be constructed.Therefore, the PINN loss function Loss consists of four parts, i.e., the horizontal motion equation along the h, u, and v directions of the loss function and the continuity equation, and it is expressed as follows(Eq.( 7) ~ Eq. ( 11)): (11) In the above equations, h F is the prediction value of the storm surge, h R is the true value of the storm surge, n is the number of training samples, and ω I represents the weights of the four parts of the loss function.This paper adopts the adaptive weighting scheme [16], [17] to assign the weights of different losses, i.e., for iteration during the training process, the weight is proportional to each loss, and the adaptive weighting scheme helps to improve the PINN's training efficiency.

Entropy Weight Method
In this study, the numerical model of the storm surge in the Yellow Sea and the Bohai Sea established is used [18].This numerical model is based on an unstructured triangular grid, and the calculation area covers the Bohai Sea, the Yellow Sea, and a part of the East China Sea.The boundary extends to 128°E in the east and 29.5°N in the south, and the model grid, parameter settings, and distribution of the coastal tide stations (as can be seen in figure 2) are consistent with those in Fu's research [18], similarly hereinafter.
Three storm surge events ("20130923","20151105" and "20160212") were selected, and the experiments of three important parameters (training size, activation function and loss function coefficient) in the PINN model has been carried out to facilitate the following research.
The relative error of tide station's maximum surge in the Bohai Sea based on five different training sizes has been shown in figure 3a.The results show that when the training size is only 0.4%, the average relative error reaches 40%, and the maximum relative error reaches more than 60%, the training size is too small, leading to underfitting of the model.When the training size gradually increases to 4%, the average relative error gradually decreases to 6%, and the maximum relative error also decreases to 12%.However, as the sample size continues to increase, the relative error continues to increase, this is because larger training samples require more iterations and smaller learning rate, 4% of the training size can achieve the minimum error.
Four different loss function coefficients are tested, and the relative error of tide station's maximum surge in the Bohai Sea has been shown in figure 3b.The four digits of the horizontal ordinate in figure 3b represent the values of ω1 ω2 ω3 and ω4 in Eq. ( 12), for example, 3111 represents 3, 1, 1, and 1.The results shows that with the same number of iterations, the adaptive coefficient (also known as Softmax function) results are better.

Figure 2. The distribution of tide stations in the Bohai
Sea used in the study.The activation function is also important to the accuracy of the PINN model, four different activation functions were tested, and the relative error of tide station's maximum surge in the Bohai Sea has been shown in figure 3c.The results showed that the Sigmoid function performed the worst with a maximum error of over 65%, while the Tanh function was relatively optimal with the lowest average relative error of 7% and a maximum relative error of 24%.

Application of the PINN in storm surge reconstruction
Storm surge reconstruction refers to using the storm surge information of some known points to reconstruct other unknown points or the entire storm surge field.Usually, the data obtained from experiments or observations is limited, based on a portion of the existing information, the entire field information is reconstructed to test the training skills and application capabilities of the PINN model.In terms of PINN storm surge reconstruction and the application of PINN and ANN storm surge forecasting, the numerical simulation results of 14 storm surge events in the Bohai Sea from 2013 to 2016 are selected as the training set ("20130318", "20130923", "20131014", "20131109", "20140929", "20150403", "20151105", "20151122", "20160212", "20160503", "20160825", "20161005", "20161022" and "20161105"), the wind field adopts the ERA5 reanalysis wind field data from the European Centre for Medium-Range Weather Forecasts (ECMWF), and the numerical storm surge simulation data are used to create a PINN training dataset.
The PINN network structure adopts 7-100-100-100-100-5, the batch size is 1000, the number of iterations is 150, the learning rate is set to 0.001, the activation function selects Tanh, and the loss function structure of the adaptive coefficient is used.The storm surge numerical prediction model uses the ERA5 wind field to simulate the storm surge event in the Bohai Sea for a total of 4 days from October 28, 2016, to October 31, 2016.The numerical prediction model adopts a cold start, and the data of each grid point in the next 3 days are used to build a training set (there are approximately 1.6 million data samples in total, 100000 data samples are selected by random sampling as the PINN training set, the remaining data samples are used as the test set, and the training samples used account for 6.3% of the total number of samples).
To assess the performance of the PINN model with a given lead time during the training and validating, we use the indices of the root mean square error (RMSE) and the mean relative error (MRE): Where hR is the observed surge level, hF the forecasted surge level, h(R,max) the observed highest surge level and h(F,max) the forecasted highest surge level.
Figure 4 shows the comparison of the storm surge time series at four tide stations (Weifang, Gudong, Huanghua, and Tanggu) in the Bohai Sea.For both the maximum storm surge and the occurrence time of the maximum storm surge, the storm surge curve obtained from the PINN reconstruction is nearly identical to the storm surge curve simulated by the numerical model.The root mean square errors (RMSEs) of the Weifang, Gudong, Tanggu, and Huanghua stations are 0.11 m, 0.09 m, 0.12 m, and 0.12 m, respectively, and they are all less than 0.12 m.Furthermore, the absolute error of the maximum storm surge of a single station is less than 0.2 m.

Application of the PINN in storm surge forecasting in the Bohai Sea
Different from the reconstruction of storm surge, when performing PINN storm surge forecasting and on the basis of the numerical simulation results, the measured data of the above mentioned 14 storm surge events are assimilated by the data of the tide stations along the coast of the Bohai Sea.Then, the numerical simulation data within 2 km of the station are replaced with measured data from coastal stations based on inverse distance weighted (IDW) interpolation, and the same assimilation scheme of the storm surge measured data is used before the ANN storm surge forecasting.
The PINN network structure adopts 7-100-100-100-100-5, the batch size is 1000, the number of iterations is 150, and the learning rate is set to 0.001.Using the same network structure, parameters, and coastal station storm surge data assimilation scheme, the PINN and ANN storm surge forecasts are performed.In addition, storm surge forecasting based on a numerical model is also carried out, and the forecast results of the three are compared and analyzed.Two storm surge events in the Bohai Sea (20141026 and 20151018) are selected for forecast verification, and the tide stations with a measured storm surge lager than 1 m are selected for comparison, as shown in figure 6 and figure 7.   6 and 7 and Table 1 show that the variation curve of the storm surge with the time predicted by the PINN at each tide station is closer to the measured curve than the ANN and the numerical model results, and the average relative errors of the maximum storm surge predicted by the numerical model, ANN, and PINN are 36.2%,37.8% and 10.0%, respectively.For the storm surge time series or the maximum storm surge, the PINN forecast is better than the numerical model and the ANN forecast.Xiong [19] showed that the small maximum storm surge predicted by the storm surge numerical model is caused by the underestimation of the ERA5 wind field, and the ANN forecasts of the storm surge are more volatile than the PINN forecasts (a relatively large decrease in the seawater level often occurs before the maximum storm surge, and since the ANN forecasts depend heavily on the number of samples in the training data set, the ANN is more sensitive to the amount of data).

Results
As a new method of integrating ANN and the physical prior knowledge expressed by mathematical equations, the PINN has considerable flexibility and high application possibility in many fields in the future.This paper uses the PINN to construct the storm surge model and successfully performs storm surge reconstruction and forecasting.
(1) The simplified 2D storm surge governing equation is introduced into the ANN network, a storm surge forecast model based on the PINN is established, and the numerical simulation dataset of 14 storm surge events in the Bohai Sea is selected as the training set of the PINN and ANN.
(2) The PINN is used to reconstruct the storm surge field, and only using 6.3% randomly sampled samples can restore and reconstruct the storm surge field accurately.The PINN-reconstructed storm surge curves at each tide station are nearly identical to the numerical model results, the RMSEs are all less than 0.12 m, and the absolute error of the maximum storm surge is less than 0.2 m.For the key moments when the storm surge height is larger than 1m, the difference in the storm surge between the PINN reconstruction and the numerical model simulation is generally less than 0.4 m.
(3) Two storm surge events in the Bohai Sea are selected as samples, and the same network structure, parameters, and storm surge data assimilation scheme are used for predictions by the ANN, PINN, and numerical model.The results show that the average relative errors of the maximum storm surge for the numerical model, ANN, and PINN forecasts are 36.2%,37.8%, and 10.0%, respectively, and for both the storm surge time series or the maximum storm surge, the PINN forecast is better than both the numerical model and the ANN forecasts.

Discussion
In order to explain the difference between PINN and ANN in the above forecast results, the change curve of each loss function items with the number of iterations based on the above two storm surge forecast processes has been shown in figure 8. Figure 8b shows that in the process of PINN training iterations, when epoch increases, each loss function sub term (Loss __ℎ , Loss __ , Loss __ , Loss _ℎ ) and the total loss function term (Loss) show a downward trend, When epoch reaches 150, the total loss function will eventually converge to less than 0.05. Figure 8a  PINN can be used for small sample training, by training only 14 storm surge processes, PINN can achieve good forecasting results, while ANN may require training for hundreds of storm surge processes to achieve this effect.In addition, due to the constraints of physical equations, even when encountering storm surge events caused by extreme weather processes, PINN can still obtain reasonable forecasting results, while ANN may encounter "black box effects" due to untrained such extreme storm surge events, and making unreasonable or unpredictable forecasting results.
The storm surge model based on the PINN realizes the combination of the physical meaning of storm surges and machine learning, which improves the prediction accuracy.Compared with the numerical model, the PINN occupies fewer computing resources, has faster computing speed, and can achieve a second-level response on a PC.However, there are many factors that affect storm surges.Firstly, Wind is the primary driving force for storm surge in the Bohai Sea, influences of the training data from different weather to the final model deserves further research, ERA5 wind field from ECMWF is widely used in the marine field, especially in the Bohai Sea, which is the reason why it is used in this paper.Of course, including GFS, WRF, or typhoon model wind fields can also be used to train model, evaluating the influences of the training data from different weather to the final model can improve the depth of research.Due to the integration of measured data from tidal stations and the introduction of storm surge control equations into PINN, the trained model is unique and only applicable to specific research area, retrain the samples if you want to use them in other areas, this is a disadvantage of PINN compared to numerical models.
Furthermore, this paper only introduces the simplified storm surge governing equation and does not consider the influence of the couplings of astronomical tides and coastal waves on the storm surge.In addition, the PINN may require a variety of experiments with different network structures and different initial schemes to determine the optimal parameters, which will be studied in depth in future research.

Figure 1 .
Figure 1.PINN structure (ANN structure on the left and PDE of the storm surge numerical model on the right).
time-varying term, the second and third terms, u ∂ ∂x and v ∂ ∂y , are the convection terms, the fourth term fv and fu is the Coriolis force term, and the f planar approximation is used.The right side of the equation of horizontal motion, ∂ ∂x and ∂ ∂y , is the local variation term of the storm surge,

Figure 3 .
Figure 3. Tide station's maximum surge in the Bohai Sea based on different training sizes (a), different loss function coefficients (b) and different activation functions (c) (The black horizontal line is the upper and lower line of the relative error, the blue box is the upper and lower quartile, the green horizontal line is the median, and the green triangle represents the average).

Figure 4 .
Figure 4. Comparison of the storm surge at the Weifang (a), Gudong (b), Huanghua (c), and Tanggu (d) tide stations.To compare the PINN reconstruction results and the model simulation results in detail, figure 5 shows the comparison between the model-simulated and the PINN-reconstructed storm surge fields at the key moments (the 52nd hour and 67th hour) when the storm surge height is larger than 1 m.The results show that the constructed storm surge field is similar to the model-simulated field.At the 52nd hour, the maximum storm surge of approximately 1 m occurs in the Liaodong Bay of the Bohai Sea, and the difference in the overall storm surge field (between the PINN and the model) is less than 0.2 m.At the 67th hour, both Bohai Bay and Laizhou Bay, which are in the Bohai Sea, experience maximum storm surges of 1.0-1.5 m; the overall difference in the storm surge field (between the PINN and the model) is generally less than 0.4 m, and the difference on the southwest coast of the Bohai Sea is generally less than 0.2 m.It should be noted that the PINN can effectively reconstruct the basic physical quantities without initial or boundary conditions and physical boundaries, and only a small amount of training data input into the PINN is sufficient to reconstruct a storm surge field similar to the model-simulated.

Figure 5 .
Figure 5. PINN reconstruction results, model simulation results, and their differences at the 52nd hour (a, b, c) and the 67th hour (d, e, f).

Figure 7 .
Figure 7.Comparison of storm surge forecasts and measurements at the Weifang (a) and Tanggu (b) stations during the 20151018 event.Figures6 and 7and Table1show that the variation curve of the storm surge with the time predicted by the PINN at each tide station is closer to the measured curve than the ANN and the numerical model results, and the average relative errors of the maximum storm surge predicted by the numerical model, ANN, and PINN are 36.2%,37.8% and 10.0%, respectively.For the storm surge time series or the maximum storm surge, the PINN forecast is better than the numerical model and the ANN forecast.Xiong[19] showed that the small maximum storm surge predicted by the storm surge numerical model is caused by the underestimation of the ERA5 wind field, and the ANN forecasts of the storm surge are more volatile than the PINN forecasts (a relatively large decrease in the seawater level often occurs before the maximum storm surge, and since the ANN forecasts depend heavily on the number of samples in the training data set, the ANN is more sensitive to the amount of data).Table1.Relative error of the maximum surge forecast by the numerical model, ANN, and PINN.
shows that in the process of ANN training, with the increase of epoch, each loss function term (Loss __ℎ , Loss __ , Loss __ ) slowly increase.When epoch reaches 150, each loss function terms increase to a stable state, indicating that the ANN training method based on pure data may have large fluctuations in the results of surge forecasting due to the lack of training data and constraints of physical equations, which cannot meet the corresponding physical laws.

Figure 8 .
Figure 8. Change curve of each loss function term during ANN (a) and PINN (b) training (The red solid line is the change curve of the total loss function, the yellow dotted line is the change curve of the data item loss function, and the blue dotted line, the green dotted line,

Table 1 .
Relative error of the maximum surge forecast by the numerical model, ANN, and PINN.