Interactive comment on “Quantifying the effectiveness of early warning systems for natural hazards” by M. Sättele et al

Comment #2, general: Then, is it really justified that the forecasting system is considered as an early warning system? Indeed, as for monitoring systems, this system does not actively deliver warning information (Schmidt 2002, Glantz 2003) since the experts have to manually and periodically check data from the monitoring network. In addition, the Swiss snow avalanche forecasting system, that is used to illustrate this type of system, could be considered as well as a part of an early warning system, providing additional information for local experts (at municipality or ski resorts scales) to manage locally the associated risk. This limit to the classification should be at least addressed in the discussion.

Comment #1, general: First, the accumulation of acronyms (POC, POD, CPT, MTBF, etc.) makes the reading less fluent and thus the manuscript sometime difficult to follow. I would strongly advice to reduce the number of acronyms in the text, and to provide a table of all cited acronyms.
Reply #2: We reduced the number of acronyms and wrote out less often used acronyms: ROC, CPT, MTTF/ MTBF, ID. We would like to stick to EWS, BN, POD, PFA, C2367 POC which are basic new elements of our new framework concept and frequently used throughout the paper. We did not add a table for 5 acronyms.
Comment #2, general: Then, is it really justified that the forecasting system is considered as an early warning system? Indeed, as for monitoring systems, this system does not actively deliver warning information (Schmidt 2002, Glantz 2003 since the experts have to manually and periodically check data from the monitoring network. In addition, the Swiss snow avalanche forecasting system, that is used to illustrate this type of system, could be considered as well as a part of an early warning system, providing additional information for local experts (at municipality or ski resorts scales) to manage locally the associated risk. This limit to the classification should be at least addressed in the discussion.
Reply #2: Yes, forecasting systems are EWS because they generate active information in case of a present hazard on a regular base and enable municipalities, ski resorts or individuals to act. However, we agree that individual actions related to different outspoken warning levels can be versatile and the final success depends on human acting and decision making. We included this aspect in our system description and example (Section 2.3). By doing so, a forecasting system is not only a part of an EWS anymore.
Comment #3, general: Last but not least, the paper presents theoretical concepts to compute the EWS effectiveness; nevertheless, there are no practical and concrete keys to estimate and compute parameters of the effectiveness equation, such as PFA, RF(PFA) or sensor failure probability. As a consequence, it would be really difficult to apply the exposed concept for another natural hazard somewhere else, since it misses explanations to concretely estimate them. Quantified examples of Illgraben or Preonzo are given, but without no details on their calculations (as for example results of table 2). I assume that further explanations are provided in Sättele 2015a and b, but this manuscript should be understood by itself.

C2368
Reply #3: We added necessary equations and details from previous papers and case studies to make the paper understandable itself. We improved the example in the 3rd step of Illgraben case study, added a detailed equation and description on quantifying the sensor failure probability in the 4th step (Section 3.1.1). For the Preonzo case study (Section 3.2.1) we also enhanced the text in several steps. To clarify the effectiveness analysis conducted for the Illgraben (3.3.1) we added a figure and details to illustrate how RF(PFA) was quantified.
Comment #4, equation 1: Ew is not defined, and details objectives (results of an effective EWS as close as 1?) Reply #4: We agree and defined the acronym EW in the text. Moreover, we clarify the term objects. In many fields optimal systems achieve effectiveness close to one, but in the case of EWS such a generalization is critical because e.g. in the case of earthquakes even smaller numbers for EW can be a success.
Reply #5: A nonparametric descriptive model, a parametric descriptive model and a parametric normative model. We thought that those expression my confuse the reader and named only the last relevant model (decision tree). However, we extended the paragraph and included the different models.
Comment #6, Sections 2.1, 2.2, 2.3: italic words are not enough highlighted. Try bold fonts or summarize them in a table or in figure 2.
Reply #6: Originally, we submitted these highlights in bold font. To avoid further problems we added Table 1.
Comment #7, Section 2.1: you only describe the system once it is set up. But you do not describe pre-investigation works on the natural hazard (monitoring network design, threshold settings, etc.) and the procedure in case of alarm, as you do in section 2.2.
Reply #7: The level of details should be similar for 2.1 and 2.2 (and 2.3). We did C2369 not include pre-investigations in any of the descriptions of the three different system classes. However, we described typical dissemination action for all three types. For alarm systems red lights, for warning systems evacuations and bulletins for forecasting systems.
Comment #8, Section 2.1 line 13 "automated alarm call is activated": to whom, experts or population?
Reply #8: We adapted the text. It is initiated and activates the alarm stations.
Comment #9, Section 3.1 "3rd determine conditional probabilities": would it be possible to design a decision tree or include it in the BN in order to clarify actions according to AND or OR? This remark highlights as well that no flowcharts defining actions and chain of command seem to be required in you process, while they are recommended by other authors ( Reply #9: We demonstrate possible actions related to AND and OR relations in two conditional probability tables for two nodes of the warning system. Following also your comment 16 we replaced on of the existing tables with a more illustrative example. Please see comment #11.
Reply #10: We added more details and an equation, which we used in the Illgraben case study to calculate the failure probabilities.
Comment #11, Section 3.2 "5th evaluate the reliability" What about the reliability, efficiency of the chain of control/command required from warning checks to real actions?
Reply #11: We agree that the reliability of non-automated system depends not only on the warning decision, but also on the actions required to set up intervention measures successfully. Thus, we adapted the text in Section 3.2. The reliability is not only a result C2370 of the POD; it requires that intervention measures are set up.
Reply #12: As described in the text. It is estimated by fitting a function to the number of observed failures before the event in May 2012.
Comment 13#, Section 3.2 line 2 page 4496: evacuation from 7 May to when (only one day, or evacuation until rock mass failure)?
Reply #13: They evacuated on the 7th for 1 day and then again before the event occurred. We adapted the text.
Comment #14, Section 3.2 last sentence: please provide more details on how the human-decision makers have a significant influence on EWS reliability. Table 3 does not provide valuable information to better understand.
Reply #14: We agree and adapted the text.
Comment #15, Section 4 line 11 page 4502: Please develop or illustrate with an example what tools the software would need to improve your process.
Reply #15: We illustrated more detailed ideas on how such a software tool could look like.
Comment #16, Reply #18: No, in both cases the initial number of the sensors is the same. However we assessed the results for varying numbers of initial sensors (between 5 and 50) in the Preonzo study. We clarified this issue in the text.
Comment #19, - Figure 2: it would be interesting to add requirements of each system, in term of lead time and expressiveness of available precursors.
Reply #19: It is difficult to add this aspect in the figure without overloading it. It is already very complex. However, information on lead time and possible monitoring parameters are given in the text (Section 2.1-2.3) and lately in Table 1.
Comment #20, Figure 13: is the fit really representative? If I do not refer to Sättele et al. 2015a to understand the caption, I can doubt about that. Please clarify.
Reply #20: Yes the fit is representative. Although the failure probability is underestimated up to 7 days before the event, it is then underestimated. Please see arguments in Sättele et al. 2015b. However, to keep the complexity of this paper low we decided to avoid adding details and would like to keep that.