Probabilistic ﬂood extent estimates from social media ﬂood observations

. The increasing number and severity of ﬂoods, driven by phenomena such as urbanization, deforestation, subsidence and climate change, create a growing need for accurate and timely ﬂood maps. In this paper we present and evaluate a method to create deterministic and probabilistic ﬂood maps from Twitter messages that mention locations of ﬂooding. A deterministic ﬂood map created for the December 2015 ﬂood in the city of York (UK) showed good performance ( F ( 2 ) = 0.69; a statistic ranging from 0 to 1, with 1 expressing a perfect ﬁt with validation data). The probabilistic ﬂood maps we created showed that, in the York case study, the uncertainty in ﬂood extent was mainly induced by errors in the precise locations of ﬂood observations as derived from Twitter data. Errors in the terrain elevation data or in the parameters of the applied algorithm contributed less to ﬂood extent uncertainty. Although

the Atlantic Ocean in 2009. Probabilistic maps of the location of the wreckage were successfully used to find the wreckage in 2011, while previous attempts, spanning a two-year period, all failed (Stone et al., 2014). More specific to flood mapping, information about uncertainties can be used to direct surveys to areas in which the flood extent is highly uncertain. The information can also be used by rescue workers navigating an affected area, to choose the most optimal route by weighing the length of a route against the probability of it being flooded. 5 In the present manuscript we investigate the applicability of using social media content from Twitter to generate probabilistic flood maps. We explicitly address uncertainties in the data and assess the added value of probabilistic maps over deterministic maps. The analyses presented in this manuscript give insight into the magnitude of errors in flood observations derived from Tweets and improve understanding of how these errors affect the flood extent estimates. Furthermore, we investigate how the 10 uncertainty caused by errors in the Twitter data relates to the uncertainty caused by other sources of error.
This manuscript starts with a description of the case study (Sect. 2). This is followed by an overview of the methods used (Sect. 3). Section 4 subsequently presents the research results and Sect. 5 includes an in depth discussion thereof. Finally, the conclusions of the research are given in Sect. 6. 15

Case study
On the 27 th of December 2015 peak water levels on the river Ouse, caused by large amounts of rainfall over the month December, led to the flooding of a considerable area within the City of York in the North of England. The 2015 floods were widespread and inundated almost 600 homes and businesses (Stott, 2016). 20 The flooding of the City of York in December 2015 was selected as a case study, since both high resolution terrain elevation data, as well as recorded flood extents were available for this event. In Fig. 1 an Environment Agency (EA) digital terrain model (DTM) of the city of York is given (EA, 2014). This manuscript focusses on Central York, delineated by the central administrative areas of York. In most of the study area terrain slopes are moderate, although some higher ridges are found in the south and south-west of the area. North of these ridges the inner-city of York is located. At this location there is a confluence 25 of several rivers, of which the River Ouse is the largest.

Methods
Data from Twitter were used to derive flood information, since these data are openly and freely available. The research was comprised of several phases. First we extracted useful information from flood-related Tweets. This information was subsequently used to create a deterministic flood extent estimate. After this, based on information about the magnitude of 30 Nat. Hazards Earth Syst. Sci. Discuss., doi:10.5194/nhess-2016-376, 2016 Manuscript under review for journal Nat. Hazards Earth Syst. Sci. Published: 25 November 2016 c Author(s) 2016. CC-BY 3.0 License. errors in the data, probabilistic flood extent estimates were derived. In this section we discuss the setup of these three phases along with the methods used to evaluate the accuracy of the deterministic and probabilistic flood maps.

Twitter data extraction
We performed several steps to create a database of Twitter based flood observations (Fig. 2). First of all, we collected all Tweets that contained a number of common flood-related keywords such as 'flood' or 'inundation', using the Twitter streaming 5 API. From this database we selected the Tweets that were sent between the 25 th and 30 th of December 2015. To ensure purely Tweets regarding York were found, only Tweets that mentioned 'York' or '#YorkFloods' were included and messages referring to 'New York' or 'York County' (both in the USA) were excluded. As a last selection step, we only kept Tweets that contained explicit references to locations, such as streets or points of interest (POIs), by looking for common keywords such as 'street', 'lane', 'museum' or 'school'. Some other minor filters were applied to ensure only relevant Tweets were found, for 10 example by excluding Tweets related to flood barriers and flood warnings.
We derived locations from the Tweets in the remaining dataset by manually identifying the section of the Tweet that contained a locational reference. Based on this reference, X and Y coordinates were assigned to the Tweets. To illustrate this process, the following Tweet is used as an example: 15 "Cumberland Street in York -say they're used to flooding here but only 2000 was worse" (By: @jimtaylor1984) The locational reference in this Tweet is "Cumberland Street". These locational references were used directly to search Google Maps. The message above however did not refer to a point location, which would be the case for a POI, but to a line element. 20 We derived exact spatial coordinates from such Tweets by using the location of the street from Google Maps in combination with the DTM of the EA (EA, 2014). If topographical depressions were present along the street, the deepest depression, identified by filling the sinks in the DTM, was used as the location of the observation. In case no depressions were found, the point of lowest elevation was used, which was the case for the Tweet above (See Fig. 3).

25
To review the accuracy of the spatial coordinates derived from the Tweets, we looked at the photographs attached to some of the Tweets and compared them to Google StreetView. If we found the specific location of a photograph, we compared it to the spatial coordinates derived from the Tweets text to determine the locational error (example: Fig. 3).

Flood extent mapping
Flood extents were derived at 20 m resolution using the locations derived from the Twitter messages and a DTM from the EA 30 (EA, 2014). We applied an interpolation method to derive flood maps from the observations, similarly to Fohringer et al. (2015). Before interpolation, two processing steps were applied to guarantee more realistic flood extent maps. Firstly, we Nat. Hazards Earth Syst. Sci. Discuss., doi:10.5194/nhess-2016-376, 2016 Manuscript under review for journal Nat. Hazards Earth Syst. Sci. Published: 25 November 2016 c Author(s) 2016. CC-BY 3.0 License. derived water levels relative to the nearest drainage channel from each observation. Since none of the Tweets about York mentioned water depths, water levels were derived by assuming the same default water depth (DWD) for all observations. We calculated these water levels relative to the nearest drainage channel by using a height above nearest drainage (HAND) elevation model (Rennó et al., 2008;Norbre et al., 2011). Secondly, observations were grouped based on the local drainage directions (LDDs) to interpolate only hydrologically 'connected' observations. Instead of using the bilinear spline interpolation 5 of Fohringer et al. (2015), we used inverse distance weighting (IDW) interpolation to determine the flood extent. Figure 4 gives an overview of this process. These steps are further explained in this section. Norbre et al. (2016) applied the HAND concept to derive inundation extents for fluvial floods. By using a HAND map instead of a DTM, river slopes are filtered from the dataset. This means that HAND values in an area are directly related to river stage 10 and HAND contour lines describe the flood extent at a specific river stage. Since river slopes are filtered, upstream observations are also less likely to cause overestimations of water levels downstream. We constructed a HAND map from the DTM by deriving the LDDs and using these to determine the elevation value of each grid cell in the study area, relative to the nearest drainage channel. Grid cells were identified as being on a drainage channel if they had an upstream area of 6 km 2 or more, which gave the best representation of drainage channels in the study area. Topographical depressions were filtered to derive 15 the LDDs used to construct the HAND map. To also account for pluvial flooding of local topographical depressions, these depressions were reintroduced in the final HAND map. This map was then used to translate the DWD assigned to each observation to a water level with respect to the nearest drainage channel.
In order to only interpolate observations belonging to the same continuously flooded area, we grouped observations prior to 20 interpolating the water levels. This was done by calculating the LDDs in the area using the DTM. These LDDs were used to determine which cells are downstream of an observation. If the location of the observation is flooded, it is assumed its downstream cells are also flooded, since these are located lower than the observation and are directly connected to it. Therefore, all observations of which the flow paths intersect downstream, are located within the same continuously flooded area.

25
The water levels (relative to the nearest drainage channel) of each group of observations were subsequently interpolated using IDW interpolation as given by Eq. (1) and Eq. (2): 30 Nat. Hazards Earth Syst. Sci. Discuss., doi:10.5194/nhess-2016-376, 2016 Manuscript under review for journal Nat. Where Zx,y (m) is the interpolated water level at spatial coordinates x and y, Zi (m) is the observed water level of observation i, n is the total number of observations, Wi is the interpolation weight of observation i, dx,y,i (m) is the distance to observation i measured along the flow paths downstream of observations, s (m) is the smoothing parameter and p is the power parameter.
We used IDW interpolation since it allows for smoothing, which is useful in averaging the errors in social media content. Also 5 the nominator and denominator of Eq. (1) can be updated with new observations, meaning the additional computational time in real-time applications is limited. Water levels of observations were interpolated along their downstream flow paths. The water levels along these flow paths were subsequently given to the grid cells upstream of these flow paths, which produced a grid of water levels. From this we subtracted the HAND map, to create a grids of water depths in the area. Since water depths were calculated in areas which were separated from the observations by small ridges, flooded areas that were not connected to 10 any of the observations were removed. This procedure produced the deterministic flood maps.

Uncertainty analysis
The uncertainties in the flood extent maps were investigated using a Monte Carlo analysis. We evaluated the uncertainty originating from errors in the locations derived from Tweets, errors in the elevation data, uncertainty in the parameters of the IDW equation and uncertainty in the DWD. The characteristics of the error distributions used to simulate these errors are given 15 in Table 1.
The analysis of the locational errors of Tweets indicated that the locations derived from Tweets that refer to point locations contain less error than those derived from Tweets that refer to streets (see Sect. 4.1). The locational errors of both types of Tweets were therefore simulated differently. We simulated the locational errors of Tweets that referred to point locations by 20 adding random errors to the spatial coordinates of the Tweets. The locational errors of Tweets that referred to streets were simulated along these streets. We did this by extracting the streets to which the Tweets referred from OpenStreetMap. The locational error was modelled using a normal distribution. To generate each realization, the observations were moved a distance along the street, which was drawn from this distribution. As some streets are shorter than six standard deviations, effectively reducing the modelled error, the standard deviation used for modelling these errors was modified so that the resulting errors 25 matched the observed locational errors.
Since no reference information was available regarding the accuracy and errors in the EA DTM, these errors were simulated using typical values from literature. Since using independent normally distributed errors does not accurately reflect errors in the elevation data (Heuvelink et al., 2007;Raaflaub & Collins, 2006), spatially auto correlated errors were added using the 30 method described by Dullof & Doucette (2014). Based on typical values of standard deviations and auto correlation distances of errors in LIDAR elevation data found in literature (Leon et al., 2014;Mudron et al., 2013;Li et al., 2011;Livne & Svoray, 2011;Hodgson & Bresnahan, 2004) a standard deviation of 20 cm and correlation distance of 100 m was used. These errors Nat. Hazards Earth Syst. Sci. Discuss., doi:10.5194/nhess-2016-376, 2016 Manuscript under review for journal Nat. Hazards Earth Syst. Sci. The uncertainty caused by the input parameters, being the power and smoothing parameters (Eq. 2) and the DWD, was also evaluated. Since no specific information is available about the distribution of errors in these parameters, we used a uniform 5 distribution. The power parameter was varied between 2 and 5, the smoothing between 0 and 2000 m and the DWD between 20 and 80 cm.
To determine the number of Monte Carlo simulations required to produce the probabilistic flood maps, multiple maps were created using the same input uncertainties. It was found that using 1000 Monte Carlo simulations, two probabilistic flood maps 10 generated using the same input error distributions were nearly identical.

Evaluation of results
We evaluated the accuracy of both the deterministic and probabilistic flood maps using recorded flood extents. A draft version of the fluvial flood extents for the City of York was supplied by the EA. These flood extents however only identified areas that were directly affected by flooding from the rivers. Areas separated from the river around Knavemire Road, Water Lane 15 and Shipton Road were also known to be flooded based on photographical evidence in some Tweets. This was confirmed by news articles about these three locations. The flood extents around these locations were approximated by using the EA dataset of historic flood extents (EA, 2015) from between the years 1991 -2012. The flood extent specific to fluvial flooding during the December 2015 event and the historical flood extents were merged into one flood extent map for validation. Using these data, we evaluated the accuracy of the deterministic flood maps by calculating the F (2) statistic of Aronica et al. (2002). This 20 statistic gives the percentage of correctly modelled flooded area, relative to the total area that is either in the modelled or observed flood extents.
We evaluated the probabilistic flood maps using reliability diagrams. These diagrams offer a comparison between the modelled probability (on the horizontal axis) and the observed probability (vertical axis) of flooding (Wilks, 2006). If the probabilistic 25 flood map is correct, it is expected that of all cells on the map that have for example a 10 % probability of being flooded, only 1 out of 10 cells is actually flooded in reality. To create the reliability diagrams, the modelled probabilities were first binned in 10% intervals. For each probability bin, the cells on the probabilistic map that fell within that bin range were compared to the map with actual flood extents (the validation map). The number of times the selected cells were indeed flooded in reality were counted and divided by the total number of cells in this bin. For each binned probability interval, the central value was 30 taken as the modelled probability value that should be compared to the observed probability. Note that the first probability bin ranged from 0.01% to 10%, and the 0% probability of flooding value in the diagram included all cells having less than 0.01% probability of flooding. In case the line that is constructed this way is exactly on the diagonal of the graph, the probabilistic Nat. Hazards Earth Syst. Sci. Discuss., doi:10.5194/nhess-2016-376, 2016 Manuscript under review for journal Nat. Hazards Earth Syst. Sci.

Locational errors
We compared the locations that were derived from the text in the Tweets and used to create the flood maps to locations derived from attached photographs, in order to evaluate the magnitude of locational errors. Figure 5a gives the result of this comparison.
The magnitude of locational errors depends heavily on the type of locational references in the Tweets. If point locations such as POIs or intersections are mentioned in the Tweets, the locational error is limited (Fig. 5c). If streets are mentioned in the 20 Tweets however, locational errors are considerably larger (Fig. 5b). The difference is likely caused by the fact that the locations of point references were directly extracted from Google Maps, whereas an additional procedure was necessary to derive exact spatial coordinates from Tweets referring to streets. The large outlier in Fig. 5b is for example a Tweet that refers to Huntington Road, which is a long street that is located alongside the River Foss (see Fig. 1). The photograph was made at the northern end of Huntington Road, whereas the Tweet was pinpointed to the deepest depression along the road, in the south. There were 25 however photographs attached to other Tweets which indicated that this southern location was also flooded. Without this outlier, the standard deviation in locational errors of Tweets referring to streets reduced to 118 m.

Flood Extent mapping
We created deterministic flood extent estimates by interpolating the locations and water levels derived from the Tweets. The parameters of the IDW interpolation were calibrated using the F (2) value based on the EA recorded flood extents. A power parameter of 4 in combination with a smoothing of 600 m (Eq. 2) and a DWD of 50 cm gave the best results. Figure 6 gives a comparison between the flood extent generated using information harvested from the Tweets and the validation data, by 5 classifying all grid cells based on the four quadrants of a confusion matrix. An F (2) value of 0.69 was found, indicating that the modelled area of the flood extent that is correct, makes up 69% of the total flood extent either in the modelled or observed data. dataset. This underestimation is therefore a result of the lack of data rather than an error of the interpolation method.

Uncertainty Analysis
We created probabilistic flood extent estimates by varying the input parameters as well as simulating the locational errors and errors in the DTM in a Monte Carlo analysis. Based on the results in Sect. 4.1 the error distance along streets was modelled using a normal distribution with a standard deviation of 200 m. This modelled error effectively translates into a standard 20 deviation in spatial coordinates of 100 m as some streets were too short to reproduce the full error distribution. Given the results from Fig. 5c errors in point locations were simulated using a normal distribution with a standard deviation of 50 m. The uncertainty resulting from simulating these locational errors along with errors in the DTM and parameters is given in figure   7a.

25
The uncertainty in flood extent is considerable (i.e.: the flood probability is around 50%). However, near the inner-city, at location [1], the uncertainty is limited. This is only partly caused by the high density of observations in this area and is mostly a result of the fact that the inner-city of York is situated lower than its surroundings, effectively limiting flood extents. For the areas within York that are more flat, the uncertainty in flood extent was generally larger. The density of observations is not well represented in the uncertainty estimates. Generally speaking, one would expect an area that has a high probability of We assessed the performance of the probabilistic flood extent map in Fig. 7a by comparing it to the validation data and constructing a reliability diagram (Fig. 8a). Probabilities between roughly 15 to 85% are mostly overestimated, although the 5 most important probabilities close to 0 and 100% are accurately represented in the map. Comparing this map of all uncertainties to a map created by simulating only the errors in the elevation data and the parameters (Fig. 7b) indicates that locational errors are likely responsible for a considerable amount of uncertainty in the flood extent estimates. This is further confirmed by the results in Fig. 9, which shows the empirical cumulative distribution functions of the F (2) measure of accuracy. This was calculated by using the result of each random simulation of the Monte Carlo Analyses of different types of errors separately. 10 It can be clearly seen that locational errors cause most variation in the accuracy of the maps.
However, the reliability diagram that was constructed using the map generated without simulating errors in location (Fig. 8b), shows that by omitting the simulation of locational error, the uncertainty calculated using the Monte Carlo analysis more accurately describes the real uncertainty in flood extent. This indicates that either the probability distributions used to simulate 15 these errors or the way these errors are propagated cause the flood probability to be overestimated.

Discussion
This study gives insight into the potential of using inherently uncertain social media content to create flood maps. The deterministic flood map created using social media content already gave a reliable estimate of flood extent. However, the magnitude of locational errors was considerable, and the analyses presented in this manuscript indicate that these locational 20 errors cause considerable uncertainty in flood extent.
This uncertainty in flood extent seems to be strongly related to the topography of the study area. Especially at locations with moderate terrain slopes, flood extent uncertainty was limited. Flat areas tend to have a larger uncertainty in flood extent, since the differences in water levels caused by the different sources of error, cause larger horizontal changes in flood extent at these 25 locations.
By simulating all sources of errors however, the uncertainty in flood extent was overestimated, which can have a variety of reasons. Firstly, the magnitude of locational errors may have been overestimated by calculating the errors based on attached photographs. For example, photographs can be taken at a different location, because the location that is mentioned in the text 30 is too severely flooded. Also in places where locations of flooding were omitted, locational errors were overestimated. The outlier, mentioned in Sect. 4.1, illustrates this. This Tweet referred to a street being flooded. The location that was derived Nat. Hazards Earth Syst. Sci. Discuss., doi:10.5194/nhess-2016-376, 2016 Manuscript under review for journal Nat. Hazards Earth Syst. Sci. Published: 25 November 2016 c Author(s) 2016. CC-BY 3.0 License. from the Tweet, was known to have flooded based on photographs attached to other Tweets. Nevertheless, a large locational error was calculated, because the photograph attached to the Tweet showed that a second location along the street had also flooded. The procedure used to derive locations from Tweets referring to streets was only able to identify one location along the street, causing the second location of flooding to be omitted. Since this is an error of omission, rather than a locational error, the exclusion of the outlier is believed to have given a better estimate of the standard deviation in locational errors. 5 Another cause of the overestimation of uncertainty might be found in the empirical probability distribution used to simulate locational errors. The normal distribution used does not reflect the sharp peaks seen at 0m in the graphs of Fig. 6. Also, using a conventional error distribution to simulate errors might not give a correct representation of the actual errors in location. In reality it is more likely that an observation originated from a lower location or a topographical depression, whereas purely 10 using random errors can place observations on top of hills, which are unlikely to be flooded.
We expect however that the main reason for the overestimation of total uncertainty is caused by an overestimation of the errors in either the DTM or parameters. This can both be a result of the quantification of these sources of errors or the methods used to assess their impact. It is likely the quantification of parameter errors contributed most to this overestimation of uncertainty, 15 since these were quantified conservatively in absence of accurate information about their error distribution.
In addition to improving on the quantification and simulation of the different types of errors, the uncertainty maps can be further improved by including information about the density of observations. By default, the errors of all observations are drawn from the same error distribution in the Monte Carlo simulation, although observations that belong to large clusters are 20 likely more certain than observations that are completely isolated. That the density of observations was not accurately represented caused large areas to have a high probability of flooding, although they contained only very few observations. The probabilistic maps proved to be a useful addition to the deterministic map. Firstly, the uncertainty estimates are a source of information in itself. For example, where the deterministic map contained an underestimation of flooded area at location [4] 25 ( Fig. 5), the uncertainty estimate showed that flooding was highly uncertain at this location. This information can be used to send personnel into the area, to find out if the area is really flooded and thereby reduce the uncertainty at that location.
Similarly, the probabilistic map confirmed the accuracy of the deterministic map near the inner city of York. Furthermore, the probabilistic maps provide information about the flood extent without the need for prior calibration of the model parameters.
Therefore, these maps can potentially provide real-time flood extent information, without having to calibrate the method to 30 that particular event or location first. However, to understand how the modelled uncertainty relates to the observed flood extent for a particular area or event, some validation might still be required.
Further research is necessary to explore the full potential of the uncertainty estimates of flood extents derived from social media. First of all, the consistency of the results for other events or locations should be further reviewed. By reviewing more case studies, the effect of topography on flood extent uncertainty and the (in)dependency of the model parameters for a specific event can also be further investigated.

5
Such additional studies can also guide further optimization of the uncertainty estimates. If the results of further case studies indicate that the probability of flooding is consistently overestimated by the same amount, post-processing of the maps, by using the information from the reliability diagram, can be an easy way to improve the uncertainty estimates. If this is not the case, improving the maps by improving the process of simulating locational uncertainties might lead to better results. The overestimation of uncertainty caused by errors in the elevation data and parameters should then also be further reviewed. One 10 element that must be improved, regardless of the results of executing multiple case studies, is the inclusion of the density of observations in the uncertainty estimates. Also methods to include important barriers in the area in the coarse resolution DTM should be investigated, since it was observed that some areas were erroneously assigned a high probability of flooding.
Although such improvements can also be achieved by using actual higher resolution data, this will seriously affect computational time and thereby the potential of real-time application of the maps. 15 Where current methods for flood extent mapping such as hydraulic models and remote sensing have shortcomings in real-time application, this is where the real potential of using social media content lies. The methods used in this report can potentially be applied in real-time. Random simulations for the York case were generated at a pace of about 100 simulations per minute, and the fact that calculations for single observations can just be added to the nominator and denominator of Eq. 1 makes that 20 time can be gained if only the effect of an additional observation has to be added. To further reduce computational time, also the use of different sampling techniques, that can potentially reduce the number of random simulations necessary, can be reviewed. Besides computational time, a further look into the gathering of observations is required for a real-time implementation. Especially in real-time application, a high number of observations is necessary to ensure an up to date map can be made at any point in time. Using the selection techniques applied in this paper, only a small number of Tweets was 25 found. It should be reviewed if using different search techniques or additional sources of data can improve the number of observations, or if techniques such as crowd interaction have more potential in increasing the number of observations available for creating the maps.

Conclusions
This study illustrates that social media content has real potential in generating flood extent estimates. Although errors in 30 locations derived from the Tweets were considerable, the deterministic flood extent map presented in this manuscript showed Nat. Hazards Earth Syst. Sci. Discuss., doi:10.5194/nhess-2016-376, 2016 Manuscript under review for journal Nat. Hazards Earth Syst. Sci. Using information about the errors in the Tweets, DTM and parameter settings, we managed to construct a probabilistic flood extent map. The uncertainty in flood extent mainly originated from the locational errors of Tweets, whereas DTM and 5 parameter errors contributed less to flood extent uncertainty. A comparison of the probabilistic map to validation data showed that by simulating errors in the Tweets, DTM and parameters, a reasonable estimate of flood extent uncertainty is generated, which provides users with additional information on top of the deterministic flood map.
These results illustrate that social media content can be used to derive information about floods, regardless of the uncertainties 10 in this content. If further improvements are made, so that the methods used in this report can be applied in real-time, these maps have the potential of filling in the gap where hydraulic models and remote sensing are lacking.

Code Availability
The analyses in this manuscript were performed using Python 2 scripts. The code used for the different analyses is publically available on GitHub and published in the Zenodo research data repository (doi: 10.5281/zenodo.165818). 15

Data Availability
In the study data from downloaded from the Twitter API as well as data from the Environment Agency was used. The filtered subset of Tweets used in the research, the information about streets extracted from OpenStreetMap as well as the 20m resolution DTM and HAND map can be found in the aforementioned GitHub project (doi: 10.5281/zenodo.165818 Nat. Hazards Earth Syst. Sci. Discuss., doi:10.5194/nhess-2016-376, 2016 Manuscript under review for journal Nat. Hazards Earth Syst. Sci.   Nat. Hazards Earth Syst. Sci. Discuss., doi:10.5194/nhess-2016-376, 2016 Manuscript under review for journal Nat. Hazards Earth Syst. Sci.   Nat. Hazards Earth Syst. Sci. Discuss., doi:10.5194/nhess-2016-376, 2016 Manuscript under review for journal Nat. Hazards Earth Syst. Sci.  Nat. Hazards Earth Syst. Sci. Discuss., doi:10.5194/nhess-2016-376, 2016 Manuscript under review for journal Nat. Hazards Earth Syst. Sci.   Nat. Hazards Earth Syst. Sci. Discuss., doi:10.5194/nhess-2016-376, 2016 Manuscript under review for journal Nat. Hazards Earth Syst. Sci. Published: 25 November 2016 c Author(s) 2016. CC-BY 3.0 License. Figure 9. Empirical cumulative distribution functions of the F (2) performance statistics derived by simulating purely errors in the elevation data (blue), errors in the parameters (red), locational errors (black) and the combination of these errors (dashed green).