Improving flood impact estimations

For the description of the hazard we use two flood masks giving the flood extent and magnitude. The first flood mask was specifically derived for the flood event 2013 by JBA Risk Management (www.jbarisk.com). This flood mask has a spatial resolution of 10 meter and includes protection measures like dikes. The second flood mask was put together from flood maps provided by the European Joint Research Center (JRC) [36]. This flood mask was put together based on the calculated return periods at the respective river sections which were affected during the flood in 2013 [34]. This flood mask has a spatial resolution of 100 meter and does not include protection measures like dikes. More details are given in section 'Hazard'.

Two damage models estimate the direct economic impacts to private households and companies. The development of both methods is based on the same survey data sets collected in the aftermath of flood events [42–44].

Random Forests (RFs) which utilize a broader set of variables for the impact estimation including additional information on the inundation duration, the contamination and damage reducing factors as e.g. precautionary measures [38]. RF is a machine-learning algorithm which partitions a data set into smaller chunks of data points with similar response values (economic flood impacts in this case) by means of ensembles of regression trees [39]. They are commonly used for flood impact estimation in the scientific literature [21–23, 40]. We used the conditional inference tree algorithm to overcome a typical variable selection bias towards predictors with many possible splits [41].

Further we use SDFs which use only the water level as explanatory variable for the economic impacts. We chose a square-root function for the impact estimation of companies and households as this is a widely used function in Germany [37]:

$$\begin{equation} damage = a + b * \sqrt[2]{water\ level}. \end{equation} \tag{ 1 }$$

Both models are applied to a set of realisations of the exposed objects resulting in distributions of possible impacts [35]. See section 'Vulnerability' for more details.

Two data sets support the identification of flood-affected areas and buildings. The Basic European Asset Map (BEAM) is landuse-based and contains monetary asset values per area unit [45]. OSM is voluntarily collected geographical information of buildings and infrastructure, which is more and more used in geo-risk applications [17, 19]. Buildings with an occupancy related to a commercial, educational, industrial, public and mixed use are counted as affected companies. The term companies is here therefore used as an umbrella term for buildings involved in various economic activities. See table A2 for an extensive list in the appendix. Buildings with an occupancy related to a residential use are counted for the number of affected residential buildings. The data sets differ greatly in their spatial resolution (figure 1). Both data sets are publicly available. More details are given in section 'Exposure'.

Zoom In Zoom Out Reset image size

The BEAM data sets include asset values, but the OSM data sets require a separated asset estimation. For this we use data sets from the national accounts and scale them down to the object level in two different ways. The first approach uses the area taken from the OSM data for whole Germany and distributes the asset values for each economic sector respectively. This approach is further referred to as area scaling. The second approach distributes the asset values based on the number of flats in a residential building or based on the number employees of a company. We refer to this approach as unit scaling.

We use rough damage reports given by federal state for a plausibility check at the federal state level [46]. We conduct a more detailed validation with in-depth reports on damage and affected objects per municipality for the federal state Saxony-Anhalt and Saxony. The continuous ranked probability score (CRPS) is a scoring rule which evaluates entire distributions, in this case of flood impact estimations. The score evaluates the sharpness and calibration of a predictive distribution and generalizes the absolute error [47, 48]. The CRPS of a given distribution for a observation y_i is defined as

$$\begin{equation} CRPS_{i}(F_{i}, y_{i}) = \int_{-\infty}^{\infty} {(F_{i}(x) - \textbf{1}{\{y_{i} \leq x\}})^2dx} \end{equation} \tag{ 2 }$$

where $F_i(x)$ is the cumulative distribution function (CDF) of the predictive distribution $f_i(x)$ and $\textbf{1}{\{\cdot\}}$ is the indicator function. We compute the CRPS with an empirical CDF estimated from samples of $f_i(x)$. See [49] and [50] for details on the numerical implementation. The CRPS is calculated individually for each reported impact in the federal states and municipalities. It has the same unit as the observations, which is Euro in this case. The scores are divided by the observations to ensure their comparability across different federal states and municipalities. This means estimations evaluated with a value of one show an error exactly as high as the observation used for validation. As we use a logarithmic scale for most of the figures, values above zero indicate higher CRPS values than the observations and vice versa. The lower the CRPS values the better does the forecast perform. For the federal states the observation y_i is chosen as the middle (25%) of the range of the reported values (between 15% and 35%). For the municipalities the reported value is used as observation y_i .

The use of different data sets and methods to describe the risk components affects the impact estimation. We compute a relative difference between impact estimates to assess the effect of the data sets and methods on the estimations. The difference is related to the range of the impact estimations of the respective spatial unit. The spatial unit can be a single building up to a federal state. The relative contribution RC of a risk component $h,e,v$ for a spatial unit i is defined as

$$\begin{equation} RC_h^{(i)} = \frac{\underset{e \in \{e_1,e_2\}, v \in \{v_1,v_2\}}{max}|x_{h_1,e,v,}^{(i)} - x_{h_2,e,v}^{(i)}|} {\underset{h \in \{h_1,h_2\}, e \in \{e_1,e_2\}, v \in \{v_1,v_2\}}{max}{x_{h,e,v}^{(i)}} - \underset{h \in \{h_1,h_2\}, e \in \{e_1,e_2\}, v \in \{v_1,v_2\}}{min}{x_{h,e,v}^{(i)}}} \end{equation} \tag{ 3 }$$

where x is the mode of the calculated distribution of the impact estimates. The RC is calculated similarly for all components. Consequently, a high relative contribution means that there is a lower agreement of existing methods and data sets in the respective component compared to the other components.

The impacts computed based on OSM data sets with area scaling show a systematic overestimation (figures 2(a) and (c)). These overstimations are less pronounced for estimates based on the BEAM data set. In contrast, the estimated impacts on companies based on OSM data with unit scaling show a better fit with the observed impacts than the estimates based on the BEAM data set (figure 2(d)). Estimated impacts to assets of residential objects based on OSM and unit scaling show an improved fit as well (figure 2(b)). In most cases the estimates resulting from the JBA flood maps show better agreements with reported values than those resulting from JRC flood maps. Remarkable differences between estimates from different vulnerability models are not apparent.

Zoom In Zoom Out Reset image size

The calculated CRPS values in figure 3 show the performance of the estimates in the federal states. Estimates based on OSM and area asset estimations show the highest scores and therefore the lowest performance. The performance of estimates based on OSM improves when using the unit asset estimation. Estimates based on JBA flood maps perform better than those based on JRC flood maps for residential areas and companies as well as both asset estimation approaches. The use of different vulnerability models shows only marginal effects on the performance.

Zoom In Zoom Out Reset image size

The systematic overstimation of models using the OSM data sets with area asset scaling in combination with JRC water masks can be traced back to two possible reasons. Firstly, the overestimation could be due to a general incompleteness of the OSM data sets [51]. The incompleteness can lead to asset values per area which are too high, since asset values per area are calculated based on areas of OSM objects assigned to any of the economic sectors considered for whole Germany. The overestimation is therefore also present in area with a higher completeness of the OSM data. Allocating the asset values per employee or the number of flats leads to better results, as the data sources are seemingly more reliable. The estimates reflect this increased reliability. The completeness of the OSM data sets, however, tends to increase due to their highly dynamic growth [52]. Detailed studies on the completeness of the OSM data sets and how they evolved over time are missing but would strongly support the confidence in the validation of historic and recent events. Secondly, the larger error when using the JRC water mask in comparison to the JBA water mask (figures 3(b) and (d)) can be explained by the larger area identified (table 1). These errors can be largely reduced by using the unit asset scaling method (figures 3(b) and (d)).

Table 1. The number of residential and commercial objects and their area identified by the water masks of JBA and JRC based on the OSM and BEAM data sets for the flood event 2013. The number of objects are given in hundreds and the area is given in 100 000 sqms. Note that the values from JRC water masks are only displayed for the eight flood-affected federal states considered in this study.

	Companies		Residential
Hazard $|$ Exposure	Objects	Area	Objects	Area
JRC $|$ OSM	360	62	2510	44
JRC $|$ BEAM	48	112	75	111
JBA $|$ OSM	90	18	1020	41
JBA $|$ BEAM	29	50	51	75

Companies show a higher performance with OSM objects and unit scaling compared to the BEAM data set (figure 3(d)). The performance of the estimation of impacts to residential areas based on OSM data with JRC water masks does not improve much with the altered asset scaling method (figures 2(b) and 3(b)). One reason could be that the sum of the assets estimated stay the same with both asset estimation methods. JRC flood masks cover a larger area than those from JBA and thus identify more small buildings, which add up to the a similar area (table 1). Table 1 shows that JRC and JBA identify a comparable total area of residential objects, but JRC identifies much more single objects leading to higher assets when estimated based on buildings. In general, the estimation of economic flood impacts to residential objects at the level of federal states is most accurate with JBA water mask and BEAM data sets.

Overall, the comparison reveals that using different water masks, exposure data sets and asset scaling methods can lead to large differences in estimated economic impacts and the model performance. The asset scaling method for the OSM data sets can make the difference between being inferior or superior to estimations with the BEAM data sets. In most cases BEAM data sets perform a bit better at the spatial scale of federal states than the OSM data sets. Estimations based on water masks which are specifically derived for a certain event are more plausible. Yet, water masks put together from hazard maps with a good approximation of the return periods still result in good estimates at the level of federal states. The use of different damage models does not lead to comparable differences in the estimations.

Here we validate the estimations at the spatial scale of municipalities with detailed damage reports from the federal state Saxony-Anhalt and Saxony. The first section deals with water masks and the identification of flood-affected objects. The second section investigates differences due to the exposure data sets and asset scaling methods.

3.2.1. Water masks and the identification of affected objects

Figure 4 shows the identified objects based on the water masks and the number of objects reported by the municipalities in Saxony and Saxony-Anhalt. Each point reflects the reported and identified objects in a municipality. The reported and identified numbers show a strong mismatch for both residential and commercial objects in Saxony and Saxony-Anhalt. Especially the JRC water mask identifies affected objects in municipalities which have not reported to be affected and vice versa. Both water masks tend to overestimate the number of affected objects, whereby the JRC water masks identify more objects than the JBA water mask.

Zoom In Zoom Out Reset image size

Consequently, the use water masks of JBA and JRC leads to overestimation per federal state and sector by factors between 1.6–8.5 and 3.6–10, respectively (figure 5). In absolute numbers this is an overall overestimation of 1.8 billion and 3.6 billion Euro for both federal states together. The overestimation is higher for residential areas. Model runs based on the reported number of objects tend to be lower than the reported impacts. The range of the reported impacts in Saxony reveals a large variety of possible impacts with a similar amount of affected buildings (figures 5(b) and (d)). Impacts for one building vary therefore between a few thousands up to one million Euro.

Zoom In Zoom Out Reset image size

These shortcomings in the identification of affected buildings on the spatial scale of municipalities affect the impact estimation directly. Consequently, the resulting estimates based on the identified objects have low precision and low accuracy (figure 5). The true overestimation might a bit lower than the one shown here, since the number of reported objects is prone to uncertainties as well. Not every object which has been affected by the flood might have been reported to the officials. Still, it can be assumed that the majority has been reported.

Estimations with the reported numbers have in most cases a high precision and high accuracy. Yet, even if the numbers are known, the models underestimate the reported impacts. This underestimations can hardly be attributed to different models for residential or companies since both deliver results with a lower (figures 5(a) and (d)) and a higher underestimation (figures 5(b) and (c)). Even with the uncertainties in the estimations considered there is a strong underestimation at least for the residential areas in Saxony (figure 5(b)). At the lower spatial scales, i.e. municipalities with a lower number of affected objects, the low accuracy of the estimates stand out (figures 5(b) and (d)). This shows once more the stochastic behaviour of damage processes at individual objects and the following difficulty to capture these processes reliably.

Overall, the use of water masks lead to a overestimation of economic flood impacts. This overestimation is higher for result based on JRC water mask compared to the JBA water masks. Estimations based on the reported number of affected objects per municipality underestimate the reported impacts, but are in general reliable.

3.2.2. Exposure data sets and asset scaling

The performance of the impact estimations based on OSM data is higher for residential and commercial objects in both federal states (figure 6). BEAM data sets show a higher performance for impact estimations of commercial objects than for residential objects. The differences in performance based on water levels from different water masks or resulting from different damage models is comparably low (figure 6).

Zoom In Zoom Out Reset image size

The unit scaling methods leads to lower CRPS values for companies in both federal states and for residential objects in Saxony-Anhalt in comparison to the area asset estimation (figures 7(a), (c) and (d)). CRPS values are getting lower with increasing number of objects. The opposite can be observed for residential objects in Saxony (figure 7(b)). Differences in the water levels from JBA and JRC are very low.

Zoom In Zoom Out Reset image size

Estimates based on OSM data sets show superior performance at the level of municipalities compared to BEAM data sets (figure 6), especially, if a reliable number of affected objects can be obtained. This is more pronounced for residential areas than for commercial objects. This can be lead back primarily to the coarser spatial resolution of the BEAM data sets making them less suitable to be used in combination with reported numbers of affected objects. The influence of water levels taken from different water masks as well as the use of different vulnerability models on the estimates is negligible.

Asset scaling based on units shows mostly a higher performance compared to the area scaling (figure 7), with the exception of the estimations for the residential areas in Saxony (figure 7(b)). Here the models underestimate the impacts severely (figure 5(b)). A big part of this underestimation is probably due to underestimated exposed assets. Using the area scaling increases the asset values (figures 2(a) and (b)) and therefore leads to more accurate estimations (figure 6(b)).

The choice of exposure data sets as well as asset scaling methods has a strong influence on the performance of the estimations. OSM data sets in combination with the unit asset scaling lead generally to the most reliable estimates for municipalities. Different water levels and damage models do not have much influence on the actual impact estimates at the spatial scale of municipalities.

The choice of exposure data sets has the highest influence on the impact estimation for both, private households and companies (figure 8). The choice of the hazard data influences the impact estimation to a slightly lesser extent. Different vulnerability models contribute only a very small part to the difference of the impact estimation. All effects are independent of the spatial scale.

Zoom In Zoom Out Reset image size

The differences in estimations caused by different water levels in different water masks is not very big (figures 6 and 7). Hence the high relative contribution of the risk component hazard can be attributed to the different extents of the water masks (figure 8). Figure 3 shows the same effect. The high contribution shows the importance of flood maps showing the flood extent reliably for the assessment of recent flood events. While the influence of the extent of the floods excel the influence of different water levels for the hazard component, the exposure component is strongly influenced by the asset scaling and the exposure data sets (figures 2, 3, 6 and 7). Both aspects need to be considered, especially for the use of OSM data sets.

The high contributions of the risk components hazard and exposure show that there is high potential for improvement. Using a particular data set can influence the estimations strongly and the implications should be known for risk assessments or impact estimations. The room for improvements seem to be smaller for the component vulnerability. The low contribution also means that there is a higher agreement of existing methods and further improvements are somewhat limited in their potential to significantly improve the overall flood impacts assessments for now.

The extent and the magnitude of the hazard is described by two different water masks. The first water mask is provided by JBA Risk Management (www.jbarisk.com). This water mask was specifically derived for the flood event in 2013. The water mask takes flood defences into account and the extent is cross-checked with aerial imagery. The spatial resolution is 10 meter.

The second water mask is a product of local flood maps in Europe derived with LISFLOOD provided by the European JRC. Flood protection measures are not taken into account [36]. The spatial resolution is 100 meter. The local flood maps are provided for return periods of 10, 20, 50, 100, 200 and 500 years. The return periods of the flood maps are chosen according to the respective return periods calculated for the corresponding river sections during the flood event 2013 [34]. The resulting local flood maps are then merged to a water mask depicting the flood event in Germany in 2013.

The vulnerability is described by two different models. The first approach to model vulnerability is the use of RFs, the second is the estimation by means of SDFs. Both approaches use survey data sets collected after major flood events in Germany [42–44].

The RFs make use of several input variables to estimate the impacts shown in table A1. From the RFs we derive impact distributions conditioned on the input variables following the algorithm described in [54]. These distributions reflect the probabilities of possible flood impacts based on the input data base. This study makes only use of the possible impacts with the highest probability instead of using the whole distribution. The package 'party' (version 1.2) was used to compute the RFs [55]. The models for company flood damage are described in detail in [38] and [35].

Here we additionally implement the estimation of impacts to buildings and contents of private households. The model development for the households is analogous to the company models, however, the input variables differ (table A1). The relevant variables were preselected according to previous studies [20, 56]. Hence, individual damage models are used for the four different economic sector groups (1) manufacturing, (2) commercial, (3) financial and (4) service formed according to the European statistical classification of economic activities in the European Community (NACE Rev. 2; Nomenclature statistique des Activites economiques dans la Communaute Europenne) [58] as well as (5) residential use. In total, ten different RFs were built according to the conditional inference tree algorithm of [41] for the respective sector groups as well as two different types of assets buildings and equipment/contents.

The SDFs use only the water level as explanatory variable for the estimation of the flood damage. We chose a square-root function for the impact estimation of companies and households [37].

$$\begin{equation} damage = a + b * \sqrt[2]{water\ level}. \end{equation} \tag{ A1 }$$

This is a typically used SDF in Germany which was also applied in former studies [20, 21, 59].

Both models are applied to a set of realisations of exposed objects (similar to [35] and [18]). For each exposed residential object 300 realisations are sampled to reflect the missing knowledge about variables like the inundation duration, contamination, precaution and the exact flood space. 500 realisations are sampled per affected company due to the higher heterogeneity of companies compared to private households. The samples are taken from distributions which are derived from surveys of past events and from the official statistics. Note that the water level was not sampled but taken directly from the location of the objects within the water masks. The data sources and the used distributions are listed in table A1.

The estimation of economic impacts is done for companies and private households. Here the term companies serves more as an umbrella term of the following economic activities which we divided into four sectors following NACE Rev. 2 (Nomenclature statistique des Activites economiques dans la Communaute Europenne) according to the European statistical classification of economic activities in the European Community [58]: the manufacturing sector (Mining and Quarrying, Manufacturing, Electricity, Gas, Steam, and Air Conditioning Supply, Water Supply; Sewerage, Waste Management and Remediation Activities, Construction; NACE classes B-F), the commercial sector (Wholesale and Retail Trade; Repair of Motor Vehicles and Motorcycles, Transportation and Storage, Accommodation and Food Service Activities; NACE classes G-I), the financial sector (Information and Communication, Financial and Insurance Activities, Real Estate Activities, Professional, Scientific and Technical Activities, Administrative and Support Service Activities; NACE classes J-N), and the service sector (Public Administration and Defence; Compulsory Social Security, Education, Human Health and Social Work Activities, Arts, Entertainment and Recreation, other Service Activities; NACE classes O-S).

The exposed areas and buildings and their asset values are based on the OSM data set and the BEAM. The OSM data set is composed by individual buildings mapped by citizens. The data set version we used dates back to the year 2018. We assumed this as a sweetspot between more complete mapping status without having to many buildings which have been built in the time between 2013 and 2018. We used two approaches for the estimation and distribution of asset values of companies and residential buildings identified from OSM data. Both approaches take net asset values from the national accounts (VGR des Bundes) from the year 2013. The data is available via the GENESIS online data base (www-genesis.destatis.de/genesis/online/). The first approach distributes the asset values based on the area calculated from the OSM data. That is the net assets from the perspective sector for whole Germany are divided by the squaremeters of all buildings identified as any of the economic sectors. For residential objects the areas reported in GENESIS table 31 231-0001 are taken. Asset values in Euro per sqm of the contents of residential buildings are taken from [57]. Paprotny et al [57] use OSM and publicly available data from e.g. Eurostat and national statistical institutes (e.g. the German Federal Statistical Office) for the estimations. The resulting values are then applied to flood affected buildings to calculate their assets. We refer to this as the area scaling approach. The second approach uses the number of employees following the approach described in [35]. Residential assets are calculated and distributed based on the number of flats in a residential building. Net assets of the respective sector are divided by the living area in whole Germany taken from GENESIS online data base (table 31 231-0001). Based on these statistics the number of flats and consequently square meters affected are sampled. We refer to this as the unit scaling approach. Table A2 shows the resulting asset values per area and per unit for the different economic sectors.

The BEAM data set uses enhanced CORINE land use data [45]. The enhancement uses data from the Urban Atlas (a Copernicus land monitoring service) to increase the spatial resolution in urban areas. The BEAM data set also contains monetary asset values per area unit. The values are derived by socio-economic statistics from the EUROSTAT database. The data set provides values for residential areas as well as for areas of industry, service and trade.

We estimate economic flood impacts with different methods and data sets for a flood event in Germany in 2013. During this event eight federal states in Germany were affected, namely Bavaria, Brandenburg, Lower Saxony, Mecklenburg-Vorpommern, Saxony, Saxony-Anhalt, Schleswig-Holstein, Thuringia. The impacts are estimated at the building level following the method described in [35]. The results are distributions of impact estimates which can be aggregated to any spatial unit (from individual buildings up to countries). These distributions reflect the broad range of individual specifications of characteristics of company and residential buildings (described by the input variables and the asset values), which are usually not fully known. The ranges are covered by sampling possible characteristics of an object from distributions taken from official statistics, survey data and the water masks. The estimations in the figures are modes taken form these distributions.

We validate the results with two different data sets on the level of federal states and the municipal level. Firstly, we use damage reports per federal state as benchmarks [46]. The reports show overall damage jointly for the sectors private households, the industrial and commercial sector, agriculture and forestry as well as state and municipal infrastructure. The share of the individual sectors is only available for the federal states Bavaria and Saxony. The range between the shares of the two federal states is 15% and 35% for the industrial and commercial sector and 20% and 35% for private households. These ranges are applied to the reported overall damage of the other federal states.

Secondly, we use data sets obtained by the ministries of Saxony and Saxony-Anhalt. These data sets contain the number of households and companies which applied for reconstruction assistance and the amount of the overall loss reported per municipality. Note, that the values of both data sets are also prone to uncertainties. Especially the values per federal state can only give an idea of which damage value ranges are plausible. The values per municipality in Saxony and Saxony-Anhalt are likely to be more reliable, but can still deviate from the true flood damage.