Smartphone applications for communicating avalanche risk information a study on how they are developed and evaluated by their providers

. Every year, people are victims of avalanches. It is commonly assumed that one way to decrease those losses is to inform about danger levels. This paper presents a study on current practices in the development and evaluation of smartphones applications that are dedicated to avalanche risk communication. The analysis based on semi-structured interviews with developers of six smartphone apps highlights the context of their development, how choices of content and visualization were made and how their effectiveness is evaluated by the developers themselves. It appears that all these communicators agree on the message to disseminate and the general representation concepts (i.e., use of the international avalanche danger scale and of a tiered approach). However, the speciﬁc ways this message is presented (e.g., maps, icons) is not uniform. Moreover, only simple evaluation processes (e.g., usage monitoring) are conducted by the developers. However, they are well aware that further efforts need to be made in order to thoroughly analyze the effectiveness of the smartphone apps in terms of their real impact (e.g., increase in awareness or change in behavior). This work also highlighted that the smartphone applications are in transition from being one-way communication tools to be-coming two-way communication platforms, with the possibility for non-experts users to report on snow and avalanche conditions. This paper indicates challenges that avalanche risk communication is facing, although it is indisputably the most advanced and standardized practice compared to communication tools for other natural hazards. In addition to being relevant for the avalanche risk communication community, this research is therefore of interest for scientists and practitioners working on risk communication related to natural hazards.


Introduction
The practice of recreational mountaineering activities, such as backcountry and off-piste skiing, has increased significantly (Jamieson and Stethem, 2002;Tase, 2004;Harvey and Zweifel, 2008;Burkelijca, 2013).Unfortunately, every year people die in avalanches practicing these sports.The appropriate way to reduce the number of fatalities lies in forecasting and education (Harvey et al., 2013).However, the best forecast is worthless if it is not communicated and fully understood by the users (Burkelijca, 2013).Consequently, the question arises whether the current ways of informing recreationists about the dangers levels and the mitigation behaviors are effective.A literature review highlights that numerous papers presented in the proceedings of the regular International Snow Science Workshops deal with this topic in terms of form, content, use, suitability of avalanches bulletins and tools to disseminate them (Dennis and Moore, 1996;Conger, 2004;Tremper and Conway, 2006;Statham et al., 2010;Burkelijca, 2013;Johnsen, 2013;Klassen et al., 2013;Landrø et al., 2013;Valt and Berbenni, 2013).It shows that the avalanche experts' community is highly concerned with providing effective avalanche risk communication and that discussions of the best practices to adopt are still ongoing.
In the last years, several smartphone applications were developed to communicate avalanche risk.This is not surprising as the smartphone market is growing (IDC, 2015), and accessibility to the wireless mobile technology is increasing around the world.This makes smartphones interesting for disaster risk reduction communication.
Doubts are sometimes cast upon the effectiveness of avalanche education because changes in behavior are not achieved by providing information only (McCammon, 2004a).This argument can also be made concerning avalanche risk communication.Accidents continue to happen although major communication efforts have been undertaken by the European and North American avalanche centers.However, the appearance of these smartphone apps in the last years shows that the development of communication is considered useful and valuable.
Developing risk communication campaigns is resourceconsuming and risk communicators want to make their communication efforts effective.Proceeding to a systematic evaluation of the effectiveness is therefore necessary.In the case of avalanche risk communication, and in particular using smartphone applications, no scientific research has been published.Before conducting an evaluation research of the smartphone applications dedicated to avalanche risk communication, it is important to assess how current practices are developed, what and how choices were made, what questions and challenges avalanche risk communicators face and how the apps' effectiveness is currently evaluated.This first step is needed for future research that would evaluate the effectiveness of the avalanche risk communication effort by smartphone apps.Therefore, this study aims to analyze how these apps are developed and evaluated by the persons and organizations providing them, based on semi-structured interviews with the developers of the smartphone applications for avalanche risk communication.
The interest of this work reaches beyond avalanche risk communication.It is interesting to focus on communication related to this particular hazard, as it is more advanced than communication related to other hazards.Avalanches are the only natural hazard for which, after long debates, an international standard for the dissemination of risk information was developed, i.e., the public avalanche danger scale.Consequently, the findings, lessons learnt limitations and recommendations derived from this work could be taken into account in future practices of risk communication covering other natural hazards.

Methodology
In order to describe the way smartphone applications disseminating avalanche danger information are developed and consecutively evaluated, semi-structured interviews were conducted via Skype during fall 2014 with the developers of six of the seven available smartphone apps which focus on avalanche risk (Table 1, Fig. 1).All these apps are free to download and use.Those are the apps that provide avalanche forecasts and warning but that are not specifically developed for searching for victims or as an aid to risk assessment.
Interviewees were identified through the web pages of the smartphone applications.Snowball effect facilitated the process of access to the interviewees.The qualitative analysis presented here is based on the interviews' reports whose con- Observations derived from the use of the apps by the authors complete the interviews.
To address the way the smartphone apps were developed and evaluated, several parameters were taken into account in the analysis.They were chosen according to the pillars of risk communication (Höppner et al., 2010): (1) actors, (2) purposes, (3) modes, channels and tools that we combine into means and (4) message, as well as to risk communication evaluation research (e.g., Rohrmann, 1998).Consequently, we produced descriptions of the apps in terms of developers, content and mean; their development in terms of purpose, target audience, choices of content, visualization approach and tools as well as the place of the apps in a larger communication plan; the evaluation strategies implemented by the developers, i.e., users' feedback, usage, understanding, effectiveness.
Additional information about the apps were retrieved from the interviews and can be found in Table S1 in the Supplement.

The smartphone applications
This chapter describes the six studied smartphone applications in terms of who developed them, what their content is and how it is presented in maps, icons and drawing, texts and terminology.Note that this description corresponds to the apps as they could be accessed during the winter season 2014-2015, unless specified otherwise.

Description of the communicators
Apps 1 and 2 were commissioned by warnings services of North America and apps 4 and 5 by European ones.All the corresponding interviewees are avalanche experts.Apps 3  and 6 were created by smartphone apps developers by profession.They are not avalanche experts but are familiar with the topic as they are both recreational mountaineers and work for or in collaboration with the avalanche centers that are producing the data used in the apps they developed.

Description of the content of the apps
The smartphone apps contain several types of information (Tables 2-5) but the main content is the avalanche bulletin with the avalanche danger level.The international standard danger scale with five levels (low, moderate, considerable, high, extreme) is used and displayed.Apps 3, 4 and 5 provide an explanation of the danger scale.While app 3 provide links towards the websites of each considered forecast regions in order to get further information, the latter is included directly in the other apps.Apps 1, 2, 4 and 5 give more detailed information using the avalanche problems "concept", i.e., the types that can occur given a set of conditions (Landrø et al., 2013) (Table 3).For apps 1 and 5, even though the danger level is the first information to be presented, the current avalanche problems get a central position in the bulletin as their characteristics are systematically reported next to icons (see Fig. 2h for app 1 and Fig. 1 for app 5).In apps 2 and 4, the avalanche problems are described in the text of the bulletin.In White Risk, their typology can also be accessed in the "about the bulletin" explanation tab.Moreover, in this last app, current danger patterns (avalanche-prone location in terms of slope aspects and elevation) are described similarly in app 6.Additional information such as weather condition and snowpack information are standard in all apps.In one case (app 2), it is completed with information on road conditions, the emergency contacts, the users' observations as well as the terms of use.

Ways of presenting the information
Maps are often used to present hazard and risk information (Dransch et al., 2010) and much research has been conducted on maps' design for risk communication (e.g., Fuchs et al., 2009;Meyer et al., 2012).In relation to avalanche risk communication, online GIS and maps are used extensively (Conger, 2004).However, in the case of the six apps, the use of maps is not standardized.App 2 does not use this type of visual mean.App 1 uses maps for localization purposes and access to the regional bulletins.Apps 3, 4 and 5 display danger levels with colored polygons on a base map, while app 6 shows the icons of the international danger scale rather than the color on the polygons.In addition, to represent danger levels or to help for localization, additional use of maps is present in app 4. They are used to display snow related observation.
Several icons appear in the apps (Fig. 2).The symbols of the avalanche danger scale (A) are used as a legend banner (app 3) to display the highest danger rating on the map (app 6) or in the bulletin (app 1, while icon G is used here on the map).A single icon (B) is used in app 2 to represent the danger ratings according to elevation and slope aspects.In app 5, icon C indicates the slopes prone to avalanches, while prone elevations are indicated next to it by mean of text (in the 2014-2015 version of the app, the icon also integrated elevation).Two separate icons (D) display this information in app 4. E and F icons are combined in app 6 to provide avalanche-prone slopes and danger level according to elevation.Finally, app 1 uses one icon (G) to show this latter information on the main page and various drawings (H) to display the characteristics of the current avalanche problems, i.e., the elevation, the aspects, the chances and the expected size.
Even though the smartphone applications have a major visual component, text is used quite extensively.Typically text is used as followed: (1) one sentence, placed at the top of the main page, describes the danger situation; (2) a few words are used in support of icons for the danger level (e.g., "moderate"), elevation/aspects repartition (e.g., "in all aspects above approximately 1800 m") or avalanche problem (e.g., "naturally released"); and (3) extensive and elaborated text is used to explain detailed information on the current danger situation, the recent activity, the avalanche problems, the snowpack stability, the weather and/or the forecast tendency.
The term "danger" is used, whereas no occurrence of "hazard" was noticed.Reference to the "risk" term was only found in app 6, in the expression "risky expositions".Note that in the app 1, there is no mention of any of those words.The bulletin and forecast are expressed only with the words linked to the different levels of the danger scale, e.g "considerable" or "moderate".The likelihood of avalanche problems is expressed as "chances" with the terms "unlikely", "possible", "likely", "very likely" or "certain" in app 1.In app 5, terms related to probability are used, for example "probable" or "low probability".

Development of the apps
This chapter highlights the underlying processes and elements that determined the development of the apps: the purpose, the audience, the choices of content, visualization approach and tools as well as the place of the apps in a larger communication plan.

Purpose of developing the apps
The general purpose of developing those apps is to inform about avalanche risk by making use of the smartphone tech-nology.In the opinion of the interviewees, its main advantage is the ease of access of information in terms of timing and location (e.g., when people do not carry their computer or when people are on recreational sites).This general purpose is common to all apps, but more specific purposes were mentioned as well by the developers that were interviewed, such as increase awareness and reduce the loss of lives (app 5), help users plan their trips (app 1) and retrieve users' observation (app 2).Moreover, in the case of the two apps developed by avalanches non-experts, the more specific reason for their development was that the developers wanted to fulfill their own needs.Being themselves recreational mountaineers, they wanted quick access to avalanche risk information using their smartphone.

Targeted and actual users
From the interviews, it appeared that the Swiss app (app 4) targets the general public in its totality, while the other apps were developed for recreational mountaineers (snowmobilers, off-track skiers, backcountry skiers) independent of their knowledge and skills.An additional user group was targeted by the app 5, i.e., road managers.App 6 was primarily developed for a young audience as they were the main users of smartphones at the time of development.The developer therefore chose a cartoonish look for the app, i.e., colorful with a little animal-like mascot.However, it is stated on the website that some parts of the apps were designed for "advanced users".The actual users of the considered smartphone applications are not well known.None of the developers has a direct way to find out.One reason that was given is the lack of resources and expertise to carry out such a survey.However, it was sometimes mentioned that the actual users are probably the targeted ones.Moreover, some developers have clues about who are using their products.For example, the developer of app 6 knows that mountain guides are using it.A survey on the avalanche bulletin, which is displayed in app 4 but also on their website, showed that people accessing the bulletin are active backcountry tourers or free-riders (Winkler and Techel, 2014).Interest in gathering users' statistics was expressed by most interviewees.One action that was proposed is to analyze where the users come from and correlate this information with forecasting regions in order to get insights in the differences (e.g., of use, of opinion) between people living in those and the persons that do not.

Basis for choice
When asked how the content of the apps was chosen, it was most of the time implicitly answered that the information displayed is "useful" for the users.Common sense was stated as one basis for the choices.In addition, requirements from the smartphones' operating systems were mentioned to have an influence (apps 2 and 4) as well as the opinions of the warning services (app 6).
Except for app 3, which only provides the danger level with links to avalanche warnings services' web pages, all the other applications were constructed around a pyramidal approach.When this was explicitly stated by the interviewees, the reason behind using this approach is that the most important information, i.e., the danger level, has to be presented first.The rest of the information is presented by going more and more into detail as tabs are accessed or as users scroll down.The term "tiered approach" was used by the interviewee of app 1.The associated reason is the need to address all potential users (with potentially a wide range of abilities and knowledge) rather than the importance of the information.This logic was also expressed by the app 5 interviewee.A perfect bulletin should address non-expert users with headlines, dangers levels, exposed area and avalanche problems only, while trained users need more detailed information in order to take decision about the "trip" they will take.
One given reason to display the danger level on a map is the fact that people do not want to read text and therefore using a visual is the best way to present the most important information.Moreover, it is believed that in this way a quick overview of the situation on a whole area is possible and can help for the planning of a trip.This perspective is not shared by all interviewees.In app 1, the map is only used for localization purposes and access to the regional bulletins.The reason given for not displaying the danger level by coloring the full forecast regions is that it would be a too serious simplification to make.Nonetheless, the danger levels appear on the map by the display of an elevation icon (G, see Fig. 2).App 6 makes use of a map in a similar way: it is used to demarcate the forecast regions and display the overall danger with one of the icons of the avalanche danger scale, in order for users to get an overview and choose a region for their trip.Similar concerns linked to the difficulty and the danger associated to the aggregation of local information in a larger area resulted in the fact that app 2 does not present any maps.

Place in a larger communication plan
The apps created by avalanche warning centers were not the only communication tools they used.They all have a website to communicate the bulletins, which was sometimes viewed as the most important communication tool that exists, the smartphone application only coming to support it.Other means of communication are social media, blogs, telephone and newspapers.
It is interesting to note that the Norwegian application (app 5) was built in a multi-hazard framework of risk communication.In addition to present avalanche bulletins, it shows the bulletins related to floods and landslides.

Evaluation of the apps
After concentrating on the ways the apps were developed, the interviews focused on how the developers proceeded to evaluate their apps.It appears that several types of evaluation were conducted and other ones were discussed with the interviewees.

Users' feedback
Possibilities for users to send a general feedback through the app are limited.App 6 has a form included in the app, which is said to be mostly used to report on technical problems or to ask whether the app is available for other regions.App 2 provides a direct link to send an email.However, other feedback possibilities exist.On the associated web page of app 5, it is possible to report when the bulletin was useful using a like/dislike button.In addition, it appears that the Avalanche Canada receives feedback by emails or phone.
Although opportunities for general feedback are not very extensive, the importance of another type of feedback, i.e., giving the users the possibility to share their observations on snow and avalanches conditions, is put in practice or acknowledged by most of the apps.The best example is in app 1.In addition to date, time, location and the possibility to attach a picture, people can report on skiing, snow, avalanches and weather conditions and they can add comments (details in the Supplement).Currently, the observations are not moderated as no inappropriate content has been posted so far.In future updates, incident reports will be possible as well as more detailed observations concerning avalanches, snowpack and weather.
Although the reporting of observation from the users is stated as one of the goals to develop the app, app 2 does not propose a similar form but provides a link to send observations by emails.There is also the possibility to take a picture with the app and to report it.However, the possibility to send more structured reports will be given in the future.At the time of the interviews (winter 2014-2015) observation forms were not available in apps 4 and 6, but they were planned.During the 2015/2016 season, they were introduced in app 4.There is no direct way to provide feedback using the Varsom application.However, observation feedback can be given using its twin app, regObs.

Usage monitoring
Almost all applications have a technical monitoring of the usage.The metrics used, varying from app to app, are for example the number of downloads, number of people using the app and number of times specific features of the app were accessed.One interviewee stated that the latter is useful, for example, to assess whether there is a need to move or remove some features.At the time of the interview, the usage of app 5 was not yet monitored because it was launched for the first time that winter but its future inclusion was mentioned.Monitoring of the usage of both the apps and the website, which also provides avalanche warning information, will be compared to see whether the use of each of the tools is influenced by variables such as a given danger level or some specific weather conditions.

Understanding of content and visuals
Two of the six apps had been evaluated for content and presentation.App 1 had been evaluated during the design phase.Basis surveys had been conducted to assess what people understand/think when they see the information.It appeared that participants understood the different icons that were used and the representation of variation of danger level depending on the elevation.Moreover, risk communication experts were consulted on the ways to display the forecasts as well as on the use of icons and text.
The SLF (app 4) had performed a quality and usability evaluation by an internet survey in 2008 (Winkler and Techel, 2014).Note that this evaluation did not focus on the app in particular but on the bulletin that was displayed in both the website and app.Nevertheless, it induced a revision of the bulletin for both tools in 2012.This evaluation resulted mainly in the modification of the display of the bulletins according to the pyramidal approach favored by the European Avalanche Warning Services (EAWS, 2009).Danger pattern information (avalanche-prone location in terms of slope aspects and elevation) was therefore removed from the danger level information and placed in a separate "tab".Moreover, the interviewee reported that the way regions were described in the text of the bulletin was not always understood.Therefore, they were removed.Currently, the extent of the different danger levels is shown independently of any definition of regions.In 2014, a second survey allowed to assess the results of these modifications.Interesting results, in light of this work, are that the new way to inform about danger patterns is an improvement and that the large majority of the participants find the bulletin very important.
App 2 interviewee declared that a process that would allow to test the effectiveness of different ways to present the same information has started.Tests are planned to be conducted in collaboration with experts in people surveying.The use of a game environment in which people could choose, between different formats (3-D vs. 2-D, separate icons vs. combined icons), the ones they prefer or understand the best is considered.
App 5 had not been evaluated as such but previous users' surveys conducted for the website had an impact on the way information is displayed in the app.It appeared that users did not understand the complex drawings that were used to illustrate avalanche problems.Consequently, it has been decided that visuals would only be used for elevation and aspect information.At first, on the website, two distinct icons were used in order to ensure that users, mainly the Norwegians, would understand.The combination in one single icon (2014-2015 version of the app) was introduced because it is the way that most warning centers present such information and non-Norwegians are already used to this "standard" visual.

Effectiveness
Generally, the need to evaluate the effectiveness of the apps was acknowledged.Several goals for an evaluation, which was sometimes in the process of development, were proposed: satisfaction of the users, understanding of the information provided, remembering of the information, change in risk and danger perception, increase of awareness as well as change in behaviors.An indirect evaluation using the users' comments that can be written on the downloading websites (Apple Store, Android Store) was also mentioned.
Resources-related and methodological reasons were given to explain why such evaluations were not yet performed.Lack of expertise, funds and time constitutes the first type of reasons.Related to the second type, the increasing difficulty of truly evaluating the effectiveness from a satisfaction survey to an analysis of the change in behavior was mentioned.In this line of thought, one interviewee mentioned the need to conduct longitudinal surveys during several years in order to assess the changes in behavior.

Discussion
The combination of elements and results presented in the preceding three chapters is discussed here to provide considerations of the information chain that is taking place through the apps, of the appropriateness of the chosen content, of the fact that visuals are not used uniformly, of the reasons why the apps were developed, of the target audience and the associated representation approach as well as of the way the developers assess the effectiveness of their product.

The information chain
The communication chain of the considered smartphone applications takes place either between warning services and users (apps 1, 2, 4 and 5), i.e., via direct information flow, or between application developers, who use the information from the warning services to feed their app (apps 3 and 6), and the users, i.e., via indirect information flow.In the first case, the apps were created to use the intrinsic benefits of this technology and thus as an extension of the existing websites.Concerns about the way avalanche information is best communicated did not start with the development of this mean of communication nor did they disappear with its use.The interviews did not reveal that the development of these apps is part of a clearly defined communication strategy.However, being multi-hazards (avalanches, floods and landslides), the Varsom app is taking part in a larger communication plan that aims at informing the public on all the major natural hazards occurring in Norway.
The indirect flow of information (apps 3 and 6) is due to the need to fulfill personal needs as well as having expertise in smartphone technology.The fact that the developers are not the creator of the information could theoretically be seen as a threat to correctness of information and an open door to the dissemination of false messages.However, this is prevented by the use of information directly from the source, i.e., the warning services that collaborate with the developers.Added to the fact that the information provided by the apps is relatively basic, this type of communication chain, with involvement of external parties, is thus reliable.This in-direct information flow is possible because the data are openaccess (in one case under signed agreement) and because no legal constraints exist on the way avalanche danger information should be communicated (see Supplement).Nevertheless, most developers protect themselves from any legal action from users by adding a disclaimer at the start page of their app.
The willingness of the warning center to share their data as well as the unconstrained legal context are favorable conditions for the involvement of external parties with risk communication expertise.Even though the latter were sometimes consulted, the interviews did not reveal that communication experts were directly involved in the development or evaluation of the apps.However, following the opinions of some of the interviewees, we believe that a systematic involvement of risk communication specialists could increase the effectiveness of such communication tools.

Appropriateness of content
The central content of all described apps is the avalanche danger level.For all apps, this information is disseminated using the avalanche danger scale.This instrument, the purpose of which is risk communication (Statham et al., 2010), is now, after years of debates and development (Dennis and Moore, 1996), the standard to communicate avalanche conditions and forecasts.This shows that the development of the smartphone apps is basically a logical continuation of the existing avalanche risk communication framework.The use of smartphone technology did not trigger a major change in the information that was already communicated using other communication tools.This means that the information at stake is easily transferrable from one platform to another and that the apps are not seen as a really different communication tool.It seems to be perceived as another type of "computer screen" on which the same danger information can be displayed.However, differences can exist in the effectiveness of each type of communication tools.For example, in relation to the accessibility, the use of a mobile website compared to an app is more inclusive and therefore maybe more suitable to target as many people as possible, as there is no issue related to the operating system or type of device.Therefore, evaluation and comparison studies are a must to verify whether smartphone applications are as effective to communicate information as other communication tools and, if not so, what content adjustments should be made.
The fact that the use of smartphone app is the logical continuation of how the avalanche centers communicate about avalanche risk -mostly using websites -can explain why answers to the question of how the choice of content was made were hard to obtain.The interviewees seemed puzzled by this question.It seems therefore that the reasoning behind this choice is somehow implicit or following common sense, as said by one of the interviewees.This, as well as the fact that the content of the apps is largely similar to what is pre-sented in the websites, suggests that there is no debate about what is the most important information to disseminate in effective avalanche prevention communication.It is interesting to note that the avalanche communicators' community has a very strong opinion on what the most important information to disseminate is for prevention while it is not always clear what is the most effective information to disseminate in order to achieve disaster risk reduction of other natural hazards.However, avalanche communicators should not forget that the message they provide might be new to some users and that some explanation is required.Indeed, only three applications provide a description of the avalanche danger scale.However, whether the absence of explanation has an impact on the understanding of the bulletin by various users is still unexplored.
As a matter of fact, previous knowledge, ability to understand and needs of potential users are elements that must be considered to ensure effective risk communication.This is especially the case when the information is ample.In addition to the avalanche danger scale, two-thirds of the apps present "avalanche problems" (see Sect. 3.2).Those are considered to assist in decision-making (Atkins, 2004from Klassen et al., 2013;Landrø et al., 2013) as they can help recreationists choose the area to go to and techniques to avoid danger (Klassen et al., 2013).Avalanche problems can help understand local conditions while danger levels give information on the extent of the issue (Landrø et al., 2103).In other words, danger levels help raise awareness while avalanche problems are risk mitigation information (Klassen et al., 2013).Even though risk mitigation was not specifically stated as one of the purposes of the apps which do include avalanche problems, it is implicit that they were designed in this line of thought.Note that risk mitigation can be addressed using means other than avalanche problems.App 4 proposes a wide range of tools (e.g., situation analyzer, risk reduction method) to help decision-making.Consequently, there is a need to pursue the effort started by Landrø et al. (2103) of evaluating the use of avalanche problems as a risk mitigation tool for different types of audiences (e.g., experts and lay persons).

Non-uniformity in the use of visualization tools
While the use of the avalanche danger scale is not under discussion, not all its components are uniformly used.Its icons (Fig. 2a) are only used in 3 of the 6 apps (ID 1, 3 and 6).App 4 uses a different color scheme for level 5 (black/red checked pattern instead of black).In addition, travel advices, which are one of the components of the avalanche danger scale, can be found in the textual explanation of the danger situation in all apps (except app 3).They are only systematically presented in avalanche problem sections in apps 1 and 5. Finally, non-uniformity in the use of maps or aspect/elevation icons is an illustration that the current debate among avalanche experts focusses on the representation of the forecast and related information rather than on the content to disseminate or the terminology to use.
Uniformity in terminology is taking place.The term "danger" is used in all apps, while "risk" and "hazard" terms are not used.Similarly, the level terms of the avalanche scale (e.g., considerable) are the same in all apps.As explained by Dennis and Moore (1996), the debate about which terminology to use took place in the 1990s and the observed uniformity of terminology used in the smartphone applications shows that avalanche experts have reached an agreement on that point.

Reasons to develop an app
The primary purpose of creating these apps is to take advantages of the smartphone technology, e.g., popularity and mobile network spatial coverage.These are good reasons as using a support that is popular can favor access to information.Moreover, the portability of smartphones tackles the issue of overlooking some details or forgetting the bulletin that was checked in the morning while being out in the field, a problem that even seems to happen to the most educated professional (Tremper, 2006).However, this purpose is not one on which a communication effort can be assessed to be effective or not in terms of disaster risk reduction.The effectiveness of a given risk communication effort, similarly as for an educational program, depends on the goal for which it has been developed (Covello et al., 1991;McCammon, 2004a).Such types of goal, like raising awareness or helping users to plan trips, were mentioned by interviewees, although generally after reference to the technical goal of using the smartphone technology.Note that only once was the decrease of loss of lives stated as the goal for creating such apps.It is startling as this goal can be expected to be the ultimate one for avalanche risk communication.One reason that could explain why this purpose is not mentioned by all communicators might be in that it is now known that the reason for being caught in an avalanche is most of the time not lack of awareness, knowledge or expertise but rather heuristics (McCammon, 2004b).

Target audience and tiered approach
This analysis shows that the smartphone applications are targeting a more or less defined audience, from general public to a more precise group, i.e., backcountry mountaineers.There is clearly a need to target the latter as most accidents involve them or off-track skiers (Harvey et al., 2013).However, the targeted audience is perceived to be heterogeneous in terms of several variables, e.g., level of skills and knowledge or demographical characteristics such as age.Differences between experienced/trained and unexperienced/untrained users are acknowledged and taken into account by the way the information is presented, i.e., pyramidal or tiered approach.Using this approach allows to avoid simplifying the message/content too much and meet the needs of such a broad audience, i.e., provide the most important information first for lay users and at the same time give useful details for more advanced users.Demographical characteristics are taken into account in one of the app by using an intuitively appropriate design, i.e., cartoon type in order to target a young audience.All these considerations about the audience seem sound.However, there is no verification, as the risk communication agencies, other than the SLF, do not have data on who the actual users of the smartphone apps are.
The pyramidal approach as well as the use of some icons is recommended by the European Avalanche Warning Services (EAWS, 2009).None of the interviewees stated that they created their apps according to this specific advice except for the SLF, which acknowledged it in a publication (Winkler and Techel, 2014).Therefore, the detailed process of how the avalanche risk communication community reached the agreement of using the tiered approach for their avalanche bulletins is not known in detail.

Evaluation types
Evaluations of the apps that were performed fall in the three goal-related types of evaluation of the effectiveness of risk communication described by Rohrmann (1998): content, process and outcome.
The degree of information distribution is (will be) partially performed by all developers.Monitoring the usage of their apps falls within the outcome type of evaluation.Conducting this is an obvious precondition as the apps can in fact only be effective if they are used.However, although the usage is monitored, the characteristics of the users are basically unknown.Therefore, no validation of the choice of target audience and display approach is available to the communicators.The need to obtain information on the users, essential for effective risk communication, is shared by the interviewees.In addition, they are very conscious that deeper outcome evaluations are needed to assess the effectiveness of smartphone apps in terms of understanding change in risk perception and behavior.The fact that it is not done appears to be due to a lack of resources and expertise and not to a lack of interest or willingness.
However, other types of evaluation of the apps are performed.First, the evaluation of the comprehensibility of the icons (app 1) and message (app 4), which is essential for effective risk communication, relates to a content evaluation.These evaluations were useful as they confirmed the adequate use of icons in the first case and resulted in an effective modification of how the message is displayed in the second case.This type of evaluation is the most cited by the interviewees when asked what evaluation are needed or will be implemented.This shows that the communicators acknowledge that efforts are needed to make the representation of the information understandable, as suggested by Burkelijca (2013).Second, requests for feedback are implemented.Those relate to a process evaluation.Although not conducted directly in the concerned app but in a linked website, satisfaction with the bulletin is monitored using a like/dislike button.It might be useful to allow this feedback directly in the apps in order to increase the amount of data collected for this evaluation criterion.
Another kind of process evaluation is the (future) possibility for users to send observations of the current situation to the providers via the apps.Therefore, there is an exchange of information between the risk communication agencies and the information receiver.The potential of this feedback is important.It goes towards citizen science, volunteered geographic information or community-based monitoring (e.g., Buytaert et al., 2014;Haklay, 2013;Stone et al., 2014), approaches that are increasingly used for disaster risk reduction (Maskey, 2011).In the context of important local heterogeneity of the processes, or in case of data scarcity (Storm, 2012), the collection of observation by the users can help to improve the forecast.Moreover, observations and incidents' feedback bring a social media component to the smartphone apps where users can exchange information not only with warning services but also among themselves.If these feedback features develop further, moderation will be needed by the warning services in order to avoid the dissemination of erroneous information.Moreover, it will require to decide whether feedback becomes a real dialogue-oriented two-way risk communication practice, which has been proven to be effective in terms of awareness raising and willingness to learn risk mitigation (Kuhlicke et al., 2011).

Concluding remarks
Based on semi-structured interviews with developers of smartphone applications disseminating avalanche information for risk prevention, this work analyzed the context, the reasons and the ways the apps were developed.Moreover, we investigated how those developers evaluate their products in terms of effectiveness.We were able to highlight how choices were made and what are the remaining challenges that avalanche risk communication faces.Two main results came forward.First, it appears that the debate is currently focusing on the way information is presented rather than on what is the most important content, a debate that seems to be over.Second, the effectiveness of the apps, including the choices of information display, is unknown and urgently need to be evaluated.
The avalanche experts' community is a tight one.This was shown by several observations.The way a snowball effect facilitated the access to the interviewees is a clear example of this.Moreover, it was mentioned, most of the time implicitly but explicitly as well in some cases, that each app developer knows about the other apps, gets inspiration and adopts perceived good practices from each other.This is not only true for the development of the apps.There were long debates among the avalanche forecasters on the ways to disseminate Nat.Hazards Earth Syst.Sci., 16,[1175][1176][1177][1178][1179][1180][1181][1182][1183][1184][1185][1186][1187][1188]2016 www.nat-hazards-earth-syst-sci.net/16/1175/2016/M. K. M. Charrière and T. A. Bogaard: Smartphone applications for communicating avalanche risk information 1187 danger information.A result of these discussions was the development of the standard avalanche scale.The fact that this tool is used in all apps shows that avalanche risk communication has reached a high level of uniformity and a consistency that is beneficial to users that are traveling worldwide to enjoy mountaineering.This uniformity is also seen in the fact that the content is presented using a tiered approach and that information helping for decision-making and thus risk mitigation is existent in the apps.However, the specific ways this type of information is presented is not standard.Therefore, the developers are facing an exploration phase in terms of how to display, visualize and explain the message that they want to bring to their users.The need to evaluate the quality and the effectiveness of the apps is widely acknowledged.Efforts in this sense have been made and further evaluation processes are envisaged.However, several issues are hindering them.Practically, lack of resources and expertise prevents evaluation.Moreover, there is a need to define more precisely the purposes of the apps.The effectiveness of a communication tool should mainly be assessed by an output evaluation that can only be performed if the goal is specified accurately.Many valid purposes are attributed to the apps, from raising awareness to help for decision-making and planning.Ultimately, it is legitimate to ask whether these smartphone applications contribute to the change in behavior and therefore to a reduction of losses, which is the ultimate goal of any prevention campaign.A sound, scientific, assessment is demanding as it requires longitudinal studies that are complex to operationalize.Note that information is not the sole contributing to decision-making (McCammon, 2004b) and as such could be considered of limited use.However, not enough knowledge is currently available to confirm or deny this position.Therefore, risk communicators should pursue their intention to assess whether the message they disseminate with the apps is appropriate, understandable and useful.This need for further evaluations can and should be supported by the contribution of experts in risk communication as well as researchers.Moreover, the impacts of technical issues such as network coverage, on/offline mode, extreme weather conditions and usability on the use of those smartphone applications directly in the field should be addressed to complete an exhaustive evaluation of their effectiveness.
No matter how, the potential of those smartphone applications is important.In particular, in relation to the tendency of these tools to be medium for a two-way risk communication process.The planned upgrade to develop further the possibility for users to report observations and incidents opens the door to adapt these applications for community-based monitoring that can help forecasters or/and sharing information platforms between users.
This study presented the way risk communication tools for avalanche prevention was developed, evaluated and modified.The wealth of expertise and experience available in snow avalanche risk communication should be analyzed and used to build and improve risk communication tools related to other types of natural hazards.An increasing number of disaster risk reduction agencies are developing smartphone apps that are dedicated to informing about danger such as the Disaster Alert App of the Pacific Disaster Center, the Hurricane Flood or Earthquake by American Red Cross Apps or the Wetter-Alarm developed by the Swiss public and private insurance companies.However, there is an evident need to evaluate such products in order to ensure their effectiveness in terms of damage reduction.

Figure 2 .
Figure 2. Icons used in the smartphone applications during the 2015-2016 winter season.(a) Avalanche danger scale used in apps 1, 3 and 6.(a) One icon to display danger levels according to elevation and slope aspects in app 2. (c) Slopes prone to avalanche in app 5. (d) The two icons used to inform on slope and elevation prone to avalanche in app 4. (e) Slope prone to avalanche icon in app 6. (f) Danger levels according to elevation in app 6. (g) Danger levels according to elevation in app 1. (h) Drawings used to characterize avalanche problems in app. 1.

Table 2 .
Content available in smartphone applications: danger level and related information.√indicatesthat the information is present in the given smartphone application.

Table 3 .
Content available in smartphone applications: avalanche related information.

Table 4 .
Content available in smartphone applications: snowpack and weather information.√indicatesthat the information is present in the given smartphone application.

Table 5 .
Content available in smartphone applications: Additional information.√indicatesthat the information is present in the given smartphone application.