Validation of WRF Rainfall Model during the November 4th, 2021 Flash Flood Event in Batu City – Indonesia

Flash floods in Batu City, Indonesia on November 4th, 2021, caused damage to property, agricultural land, settlements, death of livestock, and loss of human life. One of the important factors triggering this flash flood is heavy rainfall. This study aims to analyse the spatial and temporal dynamics of rainfall as a trigger for the event. Rainfall modelling was conducted using the WRF model with 2 microphysical schemes and 3 cumulus schemes. The data used is GFS data on October 30th, 2021, with a resolution of 0.25 x0.25 degrees which is used to predict rain events until November 4, 2021. Contingency table verification, RMSE value verification, and verification using tolerance limits are used to verify the results of modelling data based on BMKG observation data. The results of this study show that this model can predict rain and non-rain events very well. However, this model is not good enough to predict the thickness of rain until November 4, 2021. The best scheme in modelling rainfall is scheme 5 (Lin and Betts Miller Janjic). Scheme 5 has the smallest RMSE value of 77, the data fit is appropriate and the medium error rate in scheme 5 is 84%. Scheme 5 with GFS data input started from 02 November 2021 can record rain events on 04 November 2021 with extreme rain categories. This extreme rain is influenced by convective clouds. The temperature of the cloud tops changes significantly from 13.00 WIB to 14.00 local time, namely from −4 degrees, and decreases to −76 degrees Celsius.


Introduction
Rain with high intensity occurred in Batu City on Thursday, November 4, 2021.This rain event caused 8 villages in Batu City to be inundated by flash floods.The villages are Giripurno, Sidomulyo, Bulukerto, Sumberbrantas, Bumiaji, Tulungrejo, Sumbergondo, Temas, and Pendem Village.Several villages also experienced water network breaks.This incident also caused 8 people injured, 7 people died, damage to 31 houses and 14 buildings.The flash flood in Batu City occurred in the part of the Sumbergunung Sub-watershed, administratively located in two villages, namely Sumbergondo Village and Bulukerto Village, Bumiaji Sub-district, Batu City.The location is located on the west side of Mount Arjuno with an area covering the peak to the foot slope of the mountain.Sumbergunung 1313 (2024) 012037 IOP Publishing doi:10.1088/1755-1315/1313/1/012037 2 Subwatershed has a centrifugal flow pattern.The condition of the slope is steep upstream and gentle downstream, which can cause flash floods in the future.Land use on the upper slope is used as forest land with mixed trees, while on the middle slope, the majority is used for mixed garden land and some residential areas.The lower slopes are used for settlements [1].Land conversion continues to occur from forest land to agricultural land and settlements.For example, the forest in Bumiaji sub-district in 2008 was 5886.38 ha, while in 2015 it was 5389.7 ha.Settlement areas in 2008 were 482.82Ha and in 2015 were 558.26Ha.Plantations in 2008 were 426.19 and in 2015 were 668.39 Ha [2].The comparison shows the conversion of forest land into settlements and plantations.
It is important to mitigate flash flood disasters in the region.One of the mitigation efforts that can be implemented is by rainfall forecasting, which is one of the main variables that cause flood disasters.Rain forecasts using Numerical Weather Prediction (NWP) are very useful in strengthening the operational capability of weather forecasts.However, the weather prediction capability operationalized by BMKG is only good for predicting the dichotomy of rain and no rain, while in predicting the occurrence of heavy rain it is still in a small reliability and ability [3].The need for weather information, especially weather prediction, is highly demanded both in terms of speed and accuracy.Therefore, a weather model with high reliability is needed, one of which is using Weather Research Forecasting (WRF).The WRF model has been widely used in numerical weather modeling [4].
The WRF model can predict the state of the regional atmosphere, so it is well used to analyze and interpret meteorological phenomena [5].In addition to public weather services, WRF is also used for atmospheric research, namely for analyzing extreme weather events, heavy rains and knowing the factors that cause important phenomena or extreme events [6].Extreme weather is caused by the opportunity for convective cloud growth.This situation is based on regional to local factors such as the effects of Tropical Cyclones, Eddy Circulation, and Shearline areas [7].The WRF-ARW model has many parameterization schemes.Parameterization schemes are used to simulate or predict rain, temperature, wind speed and direction, and other weather parameters.However, not every parameterization scheme will work well under all circumstances and for all regions, therefore it is important to conduct sensitivity experiments so that a suitable scheme can be identified [8].
Several studies have conducted sensitivity testing of different WRF-ARW model options such as Planetary Boundary Layer (PBL), radiation, convection, and microphysics schemes, and land surface models and their influence on the prediction of meteorological variables [9].In NWP models, proper representation of microphysical processes (MP) is an important factor in providing trends in atmospheric heat and moisture and vertical precipitation fluxes.This parameterization is critical in the prediction of precipitation and related variables [10].The cumulus parameterization (CP) scheme was developed to properly estimate the sub-grid scale effects of cumulus clouds in NWP mode.The CP scheme provides trends in cloud fields and precipitation in the column and convective components of surface precipitation [11].
This study aims to model rainfall during a flash flood event with the Kessler and Kain-Fritsch Schemes, namely November 4 th , 2021, in Batu City.The flood event in Batu City requires further investigation from a meteorological perspective and requires a semi-detailed to detailed scale weather simulation method that can simulate actual atmospheric conditions.Verification of the WRF model's performance in making rainfall predictions in the study area is needed to determine how much fit this model has on the region, which is useful in predicting future disaster events.Verification will be done quantitatively by comparing the model prediction results to the nearest BMKG station rainfall data.

Material and Methods
This research uses a rainfall modeling method using the WRF-ARW model.WRF serves to detail or improve global data (GFS) to less than 10 meters [12], by performing a downscaling process.Downscaling is a technique to increase model resolution by reducing the grid scale of the global model to a regional scale in the desired domain.The data used in this study are GFS (Global Forecast System) data from the National Centers for Environmental Prediction (NCEP) with a spatial resolution of 0.25°×0.25°downloaded from the website (https://rda.ucar.edu.).The downloaded data starts from October 30, 2021, to November 4, 2021.WRF-ARW rainfall modeling is used to simulate and predict heavy rainfall events that occur in the Sumbergunung Subwatershed using 2 microphysical schemes and 3 cumulus schemes, which are then cross-fitted to form 6 schemes.BMKG rainfall data at six nearby stations are used as validation data for the WRF-ARW model fit, while 1 ARG rainfall data is used to verify cloud top temperature through Himawari satellite images (Table 1).The microphysics and cumulus schemes were then cross-validated, resulting in 6 modeling schemes.Scheme 1 consists of the Kesseler (KS) microphysics parameterization scheme and the Kain-Fritsch (KF) cumulus scheme.Scheme 2 is a modeling scheme using Kesseler (KS) microphysical parameterization and Betts Miller Janjic (BMJ) cumulus parameterization scheme.Scheme 3 consists of the Kesseler microphysics scheme (KS) and the Grell-Devenyi cumulus scheme (GD).Scheme 4 was run using the Lin microphysics scheme and the Kain Fritsch (KF) cumulus scheme.Scheme 5 consists of Lin's microphysics parameterization scheme and the Betts Miller Janjic (BMJ) cumulus scheme.Scheme 6 was run using Lin's microphysical parameterization scheme and the Grell Devenyi (GD) cumulus scheme.The rainfall data used to verify the modeling data is rainfall data from 6 nearby rain stations.The data are shown in Table 2. Verification is needed to determine the accuracy of the WRF-ARW model in predicting rainfall.This verification is carried out by comparing the WRF-ARW modeling data with BMKG observation data.The verification process will produce the accuracy of the Kesseler and Kain-Fritsch schemes in simulating heavy rains that trigger flash floods in the Sumbergunung Subwatershed, Batu City.The results are then included in the contingency table for calculation.The results of this verification will show the accuracy of the model in predicting rain and non-rain events.Meanwhile, the model's ability to predict the thickness of rain will be verified using verification with tolerance limits.The contingency table is shown in Table 3.Based on the contingency table, verification variables that will be used can be calculated, such as Frequency of bias, False Alarm Ratio (FAR), Probability of Detection (POD), and Percent Correct (PC).Bias frequency is used to calculate the frequency of "yes" events in modeling or prediction data compared to "yes" events in observation data.The data comparison can show whether the model tends to overestimate (if bias>1) or underestimate (if bias<1).FAR is used to calculate the number of "yes" events in the modeling data that are predicted to occur, but not in the observation (false alarm), meaning that the modeling data does not match the observation data.A modeling can be considered accurate if the FAR value is 0, which means there are no false alarms, and the worst model FAR value is 1.POD is used to calculate the number of "yes" events in the modeling data that occur or match the observational data.POD values range from 0 to 1.A value close to one indicates that the experiment has almost perfect performance [13].For example, a POD value of 0.75 means that about 75% of the "yes" events in the modeling data match the observation data.PC is used to calculate the occurrence of "yes" in the modeling according to the observation data.This means that the modeling rainfall data matches the observation data.The PC value ranges from 0 to 1, and 1 is an accurate value.For example, a PC value of 0.85 means that about 85% of the events in the modeling data match the observational data.The PC skill score is also called the Accuracy (ACC) skill score.
Verification with tolerance limits is carried out by providing a certain tolerance limit by including the tolerance limit in a certain level of conformity.Rainfall values from WRF-ARW modeling are compared with BMKG observation data within a certain tolerance limit and level of conformity.The greater the error tolerance, the worse the level of agreement.Meanwhile, the smaller the modeling value within the predetermined tolerance limit, the better the level of agreement with the BMKG observation data.The scheme with the highest level of conformity and the smallest amount of data falling into the non-conformity category is the best.The tolerance limit verification can strengthen other verification methods.The following is the tolerance limit verification table (Table 4).

Contingency Table Verification
The contingency verification results show the level of conformity of the model results or outputs in predicting rain and non-rain events compared with BMKG observation data.The model with a scheme that has a high level of agreement with BMKG observation data is the best scheme.The best scheme can then be used in predicting rain and non-rain events at the research location.Verification is done using a contingency table so that the level of bias of the model and others can be known.Not all models are able to describe rainfall events.Estimation errors in heavy rainfall are most likely related to differences in physical environmental factors that play a role in precipitation formation.Contingency table verification in this study was used to verify the model's ability to predict rain and non-rain events.This verification excludes the value of rain rates, so rain rates are not verified in the contingency table.
Based on the verification results, the best scheme for predicting rainy and non-rainy events from October 31 to November 4, 2021, is scheme 1. Scheme 1 is run by combining the Kesseler microphysical parameterization and the Fritsch Fabric cumulus parameterization scheme.This scheme managed to get the highest PC value of 0.52, POD value of 0.91, FAR 0, and the lowest bias value among the 6 schemes run, namely 0.91.
Based on the modeling verification results, it is known that the PC value is 0.52, this result shows that scheme 1 has an accuracy rate or correct prediction value of 52%.The PC value is the highest compared to other schemes.The POD value in scheme 1 is 0.91, so the value of the "yes" event in the modeling data that occurs or is following the observation data is 91%.Meanwhile, the POD value of schemes 2, 4, 5, and 6 is 1, so the value of the "yes" event in the modeling data that occurs or matches the observation data is 100%.
The FAR value of the scheme 1 model is 0, this result indicates that there are no false alarms in modeling with scheme 1. FAR is used to calculate the number of "yes" events in the modeling data that are predicted to occur but not in the observation (false alarm), meaning that the modeling data does not match the observation data.The higher the false alarm value, the worse the model results.The highest false alarm value is in schemes 5 and 6, which is 0.2.The smallest FAR value is also owned by scheme 3, although scheme 3 has a high bias frequency.However, schemes 1 and 3 successfully described the non-rain event on November 3rd, 2021.The bias frequency result of the scheme 1 model is 0.91.These results indicate that the scheme 1 model tends to underestimate.In schemes 2,3,4,5 and 6, the frequency results of the bias shown above 1, so the model tends to overestimate.
Based on the RMSE verification results (Table 5), the smallest RMSE value is modeling using scheme 5. Scheme 5 has an RMSE value of 77.2.The next smallest RMSE value is modeling using scheme 1 with an RMSE value of 81.Meanwhile, the model with the worst estimated value is modeling using scheme 4 with an RMSE value of 117.4.Based on the RMSE verification results, the best scheme with the closest estimation value to the observation data is modeling using scheme 5. Scheme 5 was run with Lin's microphysical parameterization and the Betts Miller Janjic (BMJ) cumulus scheme.The BMJ scheme has been proven in many studies, as shown in the research methods to successfully model heavy rains in many areas in Indonesia, such as Bawean and Tuban islands.Cumulus parameter sensitivity test conducted by [14], that the BMJ scheme is the best scheme for rain prediction in Lampung on rainy days in March 2015.In addition, another study also revealed that the Purdue Lin scheme is a reliable scheme for simulating real data with high resolution [3].

Tolerance Limit Verification
Verification of tolerance limits is carried out by providing an error tolerance value on the model output data against BMKG observation data.This verification is carried out because the modeling results will tend to experience differences in value with observational data.The classification of tolerance limits is in (Table 6).This verification is carried out to find other schemes that are still possible to be the best scheme, namely by providing an error tolerance limit.The modeled data values were categorized as suitable, medium error, and unsuitable, then summed up and compared with the BMKG observation data.
Based on the verification results using tolerance limits, scheme 4 has the highest conformity value of 60%, but the value of non-conformity is also high at 30%.The percentage of non-conformity is the highest and the same as the percentage of non-conformity of scheme 6.The schemes with the next highest level of conformity are Scheme 3 and Scheme 5.The percentage level of conformity of these schemes is 57%.However, scheme 3 has a medium error rate of 20% and non-conforming data of 23%.Scheme 5 has a percentage of moderate conformity level of 27% and the level of nonconforming data is only 16% which is the lowest data error of the 6 schemes.Based on the verification results, scheme 5 tends to be more stable and has a low level of data mismatch, so scheme 5 is the best.In total, the total amount of data conformity and moderate error rate in scheme 5 is 84%.
Scheme 1 also has a high percentage of matched data and moderate errors, at 73%.The results of scheme 1 (Kessler and Kain Fritsch) are good at thresholds of 0 to 20 mm with a percentage of 73%.This condition is in line with the research conducted by [15], using the Kain-Fritsch cumulus scheme which found that the Kain-Fritsch scheme performed quite well for a threshold of 20 mm.Meanwhile, at a threshold of 50 mm, the model tends to have an overestimated performance.The difference in results between modelling data and BMKG observation data is likely due to the model losing its stability in predicting the thickness of rain over a long period.This occurrence is also relatively reasonable because the longer the modelled time, the accuracy of the model also tends to decrease.For example, the scheme 1 did not successfully predict the rain on November 4, which is indicated by data mismatches at all rain stations.Meanwhile, scheme 5 also has the highest data discrepancy on November 4, 2021.
Based on the contingency table verification, it is known that scheme 1 is the best scheme, and RMSE verification, and tolerance limits show that scheme 5 is the best.Based on the results, the analysis is repeated using GFS input data on 2 days before November 4 th , namely November 2 nd , 2021.The result of rain distribution is shown in (Figure 1).Data from November 2 nd was chosen because the farther the input data was selected, the verification results showed the higher modelling errors.These results show that Scheme 5 is more representative than Scheme 1. Scheme 5 shows the presence of very heavy rain at the study site (Figure 1e).This situation is very likely to occur, and it is not recorded by BMKG observation data due to limited observation equipment on the earth's surface.The Himawari 8 satellite has 16 bands.Information on various key aspects of heavy and extreme rain events can be obtained from weather satellite data, such as the Himawari Satellite.The presence of the Himawari-8 in 2015 with a temporal resolution of 10 minutes and 16 channels as an update generation from MTSAT-2 (Multi Transpose Satellite-2) makes observations of convective cloud growth with satellites more detailed [16].
Rain clouds begin to organize and may persist for several hours or even more within the MSC.Heavy rain can accumulate in large amounts when MSCs persist for a long time in the same location.The development of convective clouds over time can be known through the processing of Himawari 8 satellite data on channel 13.Channel 13 was chosen because its composition is suitable for cloud analysis.The development of convective clouds is presented in (Figure 2), the red circle shows the research location.At 10:00 a.m., convective clouds formed in the Java Sea region (Figure 4a).At that time, the research location, in general, were still clear.At 2:00 p.m., convective clouds were already above the flash flood location (Figure 2e).According to news reports on social media, the flash flood started at 2:00 pm.
Based on the Himawari 8 satellite cloud top temperature graph data, it is known that the cloud top temperature experienced significant changes between 13.00-14.00WIB.At 13.00 WIB, the cloud top temperature is -4°C, while at 14.00 WIB the cloud top temperature has decreased to -78°C.At that time the phase of the formation of mature convective clouds may have formed and the potential for heavy rain.The temperature of the cloud tops, decreased after heavy rainfall events, indicating the release of water or ice masses within the clouds.At 13.00-15.00,rainfall was recorded by ARG data between 27 and 50.2 mm/hour.This indicates a sudden onset of rain.This condition is also in line with the spike in cloud top temperature at 13.00-14.00local time (Figure 3).The sudden increase in cloud top temperature indicates the release of a lot of water mass.The convective cloud persisted for hours afterward and then disappeared from the flash flood scene at 23:00 local time.This interpretation shows that the integration of cloudburst data and atmospheric characteristics is very useful in analysing the formation of heavy or extreme rainfall in detail.

Conclusion
The WRF ARW model can be used to model rainfall as well as rainy and non-rainy events.The best scheme in modelling rainfall is scheme 5 (Lin microphysics and Betts Miller Janjic cumulus).Scheme 5 has the smallest RMSE value of 77, the data fit is good and the medium error rate for scheme 5 is 84%.The best scheme for modelling rain and non-rain events is scheme 1 (Kessler microphysics and Fritsch Fabric cumulus).Scheme 1 gets the highest PC value of 0.52, the FAR value is 0 indicating that there are no false alarms in modelling with scheme 1.Data bias is 0.91 which means the scheme 1 model tends to underestimate.The modelling results show that there was a rain event with very heavy intensity on November 4 th , 2021.The rain event was influenced by convective clouds so the cloud tops changed significantly at 13.00 WIB until 14.00 local time.

Figure 1 .Figure 2 .
Figure 1.spatial distribution of rain-rate from WRF model results in November 4 th , 2021 based on each model scheme (Source: Analysis result, 2023)

Table 1 .
WRF model configuration used in this research (Source: data analysis, 2023)

Table 2 .
Observed rainfall from BMKG Stations in the study area (Source: data analysis, 2023)

Table 4 .
Threshold of the tolerance limit used in this research (Source: data analysis, 2023)

Table 5 .
Verification result of the contingency table and the RMSE (Source: Analysis result, 2023)

Table 6 .
Verification result of the tolerance limits (Source: Analysis result, 2023)