Multi-criteria decision-making for flood risk management : a survey of the current state of the art

This paper provides a review of multi-criteria decision-making (MCDM) applications to flood risk management, seeking to highlight trends and identify research gaps. A total of 128 peer-reviewed papers published from 1995 to June 2015 were systematically analysed. Results showed that the number of flood MCDM publications has exponentially grown during this period, with over 82 % of all papers published since 2009. A wide range of applications were identified, with most papers focusing on ranking alternatives for flood mitigation, followed by risk, hazard, and vulnerability assessment. The analytical hierarchy process (AHP) was the most popular method, followed by Technique for Order Preference by Similarity to an Ideal Solution (TOPSIS), and Simple Additive Weighting (SAW). Although there is greater interest in MCDM, uncertainty analysis remains an issue and was seldom applied in flood-related studies. In addition, participation of multiple stakeholders has been generally fragmented, focusing on particular stages of the decision-making process, especially on the definition of criteria weights. Therefore, addressing the uncertainties around stakeholders’ judgments and endorsing an active participation in all steps of the decision-making process should be explored in future applications. This could help to increase the quality of decisions and the implementation of chosen measures.


Introduction
Floods can be regarded as one of the most costly natural hazard both in developing and developed countries all over the world (Balica et al., 2013;Uddin et al., 2013).According to the Emergency Events Database (EM-DAT), these pro-cesses were the most frequent natural disaster worldwide between 2000 and 2014, causing at least 85 000 fatalities and affecting about 1.4 billion people.Apart from the loss of lives and physical damage, floods have resulted in approximately USD 400 billion in damage since 2000 (CRED and OFDA, 2015).
In order to mitigate these impacts, a set of flood reduction measures need to be taken.The decision-making process related to flood risk management, especially in the prevention and emergency phases, tends to be rather complex and uncertain (Akter and Simonovic, 2005;Kenyon, 2007).Part of this complexity arises from the involvement of multiple stakeholders, each one with different views, background knowledge, interests, and frequently with competing objectives (Evers, 2008).In addition, the exact flood magnitude and damage are generally unknown and surrounded by considerable uncertainties (de Kort and Booij, 2007).As a consequence, making these decisions can rarely be solved with intuition alone.Thus, flood risk management requires the use of decision support tools, which can consider multiple stakeholders' views, objectives, trade-offs, feasible alternatives, and evaluation criteria.
Flood risk management can benefit from the use of multicriteria decision-making (MCDM) tools.MCDM is an umbrella term used to describe a set of methods for structuring and evaluating alternatives on the basis of multiple criteria and objectives (Voogd, 1983).These methods provide targeted decisions, as they can handle the inherent complexity and uncertainty of such problems as well as the knowledge arising from the participation of several actors (Yan et al., 2011;Zagonari and Rossi, 2013).
Published by Copernicus Publications on behalf of the European Geosciences Union.
MCDM can enhance the quality of decisions, by making the process more explicit, rational, and efficient, leading to justifiable and explainable choices (Mateo, 2012a).Furthermore, MCDM promotes the role of participants in the decision process, facilitates compromise and group decisions, and provides an adequate platform for stakeholders to communicate their personal preferences (Pohekar and Ramachandran, 2004).The combination of these characteristics enables the development of real participatory processes, which are crucial for the implementation of successful and long-lasting flood management programs (Affeletranger, 2001).
Therefore, MCDM provides a powerful tool for flood management and has received a great deal of attention in solving such problems, not only from researchers but also decision makers and practitioners outside the scientific community.Since the mid-1990s, MCDM has been successfully applied to select the best strategies for flood risk mitigation, helping to optimize the allocation of available resources (e.g.Tkach and Simonovic, 1997;Ghanbarpour et al., 2013;Malekian and Azarnivand, 2015).In recent years, MCDM has also been used to access the flood risk and coping capacity (e.g.Guo et al., 2014;Roy and Blaschke, 2015;Yang et al., 2013).
Several authors have reviewed MCDM techniques in various fields of study.For example, Stewart (1992) conducted a theoretical review by identifying potential advantages and pitfalls in the usage of various MCDM methods.Hajkowicz and Collins (2007) analysed over 134 papers in the field of water resource planning and management, focusing on problems such as water policy evaluation, strategic planning, and infrastructure selection.More recently, Estévez and Gelcich (2015) presented a concise literature survey, exploring the challenges behind participatory MCDM in marine conservation.However, despite practical experiences and methodological advances, there is no comprehensive literature review that explores the use of MCDM for flood risk management.
Hence, we believe that there is a need for a systematic survey to consolidate and synthesize recent research conducted in this area.Therefore, this paper aims to provide a literature review of the state of the art regarding the application of MCDM as a decision support tool for flood risk management, seeking to assess emerging trends and identify issues for future investigation.In addition, it attempts to provide a better understanding of the current status of how participatory MCDM is being conducted and the way uncertainties are considered in the decision-making process.With this review, we attempt to answer the following questions.
1. Which flood risk management problem has used MCDM approaches most?
2. Where was the research undertaken?
3. Which MCDM method was most commonly applied?
4. Were multiple stakeholders explicitly included in the decision-making process? 5. To which extent did these studies apply uncertainty and sensitivity analysis?
For reader's convenience, the remainder of this paper is structured as follows.In Sect.2, the basic features of the MCDM methods are briefly described.Section 3 outlines the search strategy and the procedure used to classify the literature.Section 4 covers the discussion of the outcomes and provides answers to the research questions.In Sect.5, limitations of this study and recommendations for further research are provided.Finally, Sect.6 presents concluding remarks.We hope that this review will serve as a useful and ready source of information for scholars and practitioners working with MCDM and flood risk management.

Overview of multi-criteria decision-making methods
MCDM is a broad term used to describe a set of methods that can be applied to support the decision-making process by taking into account multiple and often conflicting criteria through a structured framework (Cinelli et al., 2014).Since the 1960s, dozens of MCDM techniques have been developed (Mendoza and Martins, 2006).Generally, they can be classified into the following groups (Hajkowicz and Collins, 2007).
1. Multi-attribute utility and value functions: the goal of these methods is to define an expression for the decision maker's preferences through the use of utility/value functions.Based on this, all criteria are transformed into a common dimensionless scale (Linkov et al., 2004).Popular methods include MAUT (Multi-attribute utility theory) and MAVT (Multi-attribute value theory), which have a compensatory nature.This implies that the poor performance of one criterion (e.g.high loss of lives) can be compensated by the better performance of another (e.g.financial cost).Although MAUT and MAVT have well-established theoretical foundations, the preference elicitation can be cognitively challenging and time-consuming (Schuwirth et al., 2012).
2. Pairwise comparisons: this approach involves comparing pairs of criteria by asking how much more important one is than the other according to a predefined scale.Pairwise comparisons are particularly useful when it is not possible to define utility functions; otherwise MAUT is recommended (Ishizaka and Nemery, 2013).
Common techniques include AHP (analytical hierarchy process), ANP (analytical network process) and MAC-BETH (measuring attractiveness by a categorical based evaluation technique).Due to its simplicity and flexibility, AHP is the most applied MCDM tool.Nevertheless, AHP has a limitation when dealing with interde-pendence among the criteria as it assumes that they are independent (Li et al., 2011).In addition, only a limited number of alternatives can be considered at the same time.
3. Outranking approaches: unlike MAUT, MAVT, and AHP, outranking methods are based on the principle that one alternative may have a degree of dominance over another (Kangas et al., 2001), rather than assuming that a single optimal solution exists.).An advantage of outranking approaches is that they avoid compensation between criteria and any normalization process, which alters the original data (Ishizaka and Nemery, 2013).Therefore, they are appropriate when criteria metrics are not easily aggregated, measurement scales vary over wide ranges, and units are incommensurate or incomparable (Linkov et al., 2004).
4. Distance to ideal point methods: the alternatives are evaluated and ordered based on their distance from the ideal point, which represents a hypothetical alternative that best suits the decision makers' goals.Hence, the alternative that is closest to the ideal point is the best solution (Malczewski, 1999).Well-known methods include TOPSIS (Technique for Order Preference by Similarity to an Ideal Solution), CP (Compromise programming) and VIKOR (VlseKriterijumska Optimizacija I Kompromisno Resenje).The main characteristic and advantage of this family of approaches is the ability to consider a non-limited number of alternatives and criteria.
5. Other methods: there are a large number of miscellaneous techniques that cannot be placed under any of the described categories.These include, for example, tailored methods which usually extend or adapt a fundamental method to a particular application, as well as fuzzy and hybrid approaches.
Despite the large number of MCDM methods, none is perfect and applicable to all decision problems.Therefore, the selection of an appropriate tool will depend on the problem type and decision makers' objectives.Guidelines such as the one proposed by Guitouni and Martel (1998) can be followed to choose from available MCDM techniques.Table 1 provides an outline of the fundamental properties of the MCDM methods that have been cited throughout the paper.A comprehensive and detailed description of the theoretical foundations of these techniques alongside with their main strengths and weaknesses can be found in Triantaphyllou (2000), Tzeng andHuang (2011), andIshizaka andNemery (2013).
3 Framework for systematic literature review

Search strategy
A comprehensive literature review was undertaken, aiming to identify peer-reviewed papers that apply MCDM to floodrelated problems.With this scope in mind, the systematic quantitative approach outlined in Pickering and Byrne (2014) was used since this method is explicit, reproducible, and has fewer biases when compared to traditional narrative reviews.Distinct combinations of these terms were used, taking into consideration the syntax requirements of each search engine.When possible, only the abstract, title, and keywords were searched.This narrowed the search space substantially and enabled to exclude papers that mention the keywords only in the references or literature review sections.
These queries elicited over 1350 references published between September 1989 and June 2015.In order to have a 2 decades review, which is considered to be long enough to arrive at consistent conclusions (Jato-Espino et al., 2014), 1995 was chosen as a starting date for this survey.At first, the title, abstract, and keywords were screened manually to exclude irrelevant references.After this preselection, the full text of 207 selected papers was revised in detail.Of this total, 74 papers were found to be beyond the scope of the inquiry and five were not available through the library system.In the end, 128 papers met all inclusion criteria and were included in the analysis.
The review covers articles published in 72 different journals, in several areas of knowledge, suggesting that a diversity of publishers share an interest in flood risk management.search and Risk Assessment, with 16, 11, 10, and 6 articles, respectively.The remaining journals account mainly for one or two papers each.

Classification scheme
Following the selection, all included papers were classified according to some key domains: publication year; area of application; country of application; MCDM method; whether or not it was carried out in a participatory process; participatory techniques applied; and if uncertainty and sensitivity analysis were performed.With regard to the MCDM method, only techniques that were used thrice or more have their own category, whilst the rest were grouped in "others".In terms of research area, the papers were classified based on the overall emphasis of the application discussed.A total of eight types of MCDM applications were identified as follows.
1. Ranking of alternatives for flood mitigation: this comprises the selection of the best combination of structural and/or non-structural mitigation solutions from a set of potential alternatives to reduce flood impacts and magnitude.
2. Reservoir flood control: this consists in selecting operational options among a range of alternatives to ensure safe operation of reservoirs during high inflow events, aiming to reduce the floods intensity to acceptable levels.
3. Susceptibility assessment: this expresses the likelihood that a flood will occur in an area on the basis of local terrain conditions (e.g.slope, elevation, lithology).It does not consider the flood temporal probability or return period (i.e. when or how frequently floods may occur) (Santangelo et al., 2011).
4. Hazard assessment: this comprehends a qualitative or quantitative assessment of the spatial and temporal probability of the occurrence of potentially damaging floods of a certain magnitude in a given area within a specific period of time (Dang et al., 2011).

5.
Coping capacity assessment: this comprises the evaluation of the ability of people, organizations, and systems, using available skills and resources, to face and manage adverse conditions and emergencies resulting from floods (UNISDR, 2009).
6. Vulnerability assessment: this refers to articles that assess the propensity of exposed elements such as human beings, their livelihoods, and assets to suffer adverse effects when impacted by floods (UNISDR, 2009).
7. Risk assessment: this consists in analysing potential flood hazards combined with existing conditions of vulnerability that together could potentially harm exposed people, property, services, livelihoods, and the environment (UNISDR, 2009).
8. Emergency management papers: these are concerned with the organization and management of resources and responsibilities for addressing all aspects of emergencies, in particular, preparedness and response steps (UNISDR, 2009).

Results and discussion
This section presents a systematic analysis of 128 peerreviewed papers published between 1995 and June 2015.To help readers extract quick and meaningful information, the results are summarized in various charts and tables.A complete list of the reviewed papers, including their classification scheme, is provided in the Supplement.

Trends by year of publication
In an attempt to model the evolution of MCDM in time, the data gathered were organized by year of publication.As can be seen from Fig. 1, there has been a continuing growth in the number of flood MCDM studies from 1995 to June 2015.In fact, over 82 % of the compiled papers have been published since 2009.Until 2004, the number of publications was equal to or less than one per year.Surprisingly, from 2010 to 2013, the use of MCDM dramatically increased, from 5 to 22 papers.Accordingly, it can be estimated that in the coming years, these numbers will keep growing.This indicates that MCDM has a good vitality and acceptance for flood risk management.
A reason for the increasing number of publications could be a reflection of a growing awareness of natural disaster prevention and reduction policies.Secondly, the availability of easy-to-use and inexpensive MCDM software packages may also be an influencing factor.Alternatively, this increase may just match a general rise in published papers related to floods as a whole.
To correctly measure the interest in MCDM for flood risk management, an increase of MCDM papers in relative terms needs to be calculated.Thus, a normalization was made according to the number of flood publications in the Web of Science and Science Direct databases, found through searches using only "flood" as keyword.Figure 2 shows that the increase of flood MCDM publications is significantly greater than the increase of flood publications, especially after 2011.This confirms the hypothesis that the use of MCDM to solve flood-related problems has been growing considerably since 1995.

Trends by area of application
During the last 2 decades, ranking alternatives for flood mitigation was the most widespread flood management topic, with more than 22 % of all applications (Table 2).These studies focus mainly on selecting traditional engineering measures to reduce flood risk (e.g.Azibi and Vanderpooten, 2003;Tkach and Simonovic, 1997).Nevertheless, in recent years, they emphasize not only the so-called structural measures, which are still relevant, but also incorporate a wide range of non-structural options such as the development of evacuation plans, enforcement of building codes, and insurance schemes (e.g.Zagonari and Rossi, 2013;Chitsaz and Banihabib, 2015).The second most common theme was risk assessment (21.11 %), followed by vulnerability and hazard analysis, both with 15.00 % of all applications.In this regard, it is worth noting that several papers evaluate the vulnerability, hazard, and risk simultaneously (e.g. Lee and Chung, 2007;Zou et al., 2013;Wu et al., 2015).Few papers used MCDM as a decision support tool in reservoir flood control and emergency management problems.This is probably because managing emergencies, both in rivers and reservoirs, is a complex task, requiring effective coordination and communication among teams involved as well as reliable information regarding the current situation of emergency (Shan et al., 2012).
In order to have a complete overview of works published through time, Fig. 3 presents a temporal breakdown of the different flood topics.As can be seen, flood risk management has recently shifted its main focus from ranking alternatives for flood mitigation towards a risk-based perspective, which includes the assessment of risk and its components.This finding is in agreement with a worldwide trend, where disaster prevention is emphasized over assistance or relief, and evaluating the risk becomes a key element (World Bank, 2006).Another interesting result is that coping capacity studies are quite new in comparison to other topics, with the first paper published in 2009.In addition, the graph reveals  that since 2010, the trend in the other flood problems has remained fairly stable.This diversity of applications shows MCDM flexibility to support decision-making in all stages of the flood management cycle.

Trends by country of application
A total of 37 countries on all populated continents have contributed to this survey (Table 3), showing that the spread of MCDM is truly global.China accounts for 19.40 % of all applications, which is not too surprising.Indeed, similar results were obtained by other MCDM review papers (e.g.Jato-Espino et al., 2014).In contrast to previous surveys (e.g.Govindan and Jepsen, 2016), Germany and South Korea were found to be prolific users of MCDM tools.Surprisingly, South American countries such as Brazil, Colombia, and Venezuela, which are severely affected by floods (CRED and OFDA, 2015), were not represented in the literature.The limited use of MCDM in these countries could be explained by restrictions, such as lack of expertise, resources, and technology.On the other hand, it could be that the existing studies are published in non-English journals (e.g.Drozino et al., 2015;Magalhães et al., 2011).Unlike other MCDM review papers (e.g.Behzadian et al., 2010;Mosadeghi et al., 2013), MCDM tools were rarely applied to solve flood-related problems in Australia.The reason could be that potentially relevant studies are published in conference papers, government reports, non-indexed journals, or in other grey literature.
Half of the MCDM studies were conducted in Asia, followed by Europe (35.07 %), North America (8.21 %), Africa (3.73 %), and finally by Australia and South America, each with 1.49 % of all applications.Therefore, it is clear that when we analyse the findings of the present study, we are providing a predominantly Asiatic and European view of flood risk management.
Furthermore, only three papers report cross-country investigations (e.g.Ceccato et al., 2011;Evers et al., 2012;Almoradie et al., 2015).For example, Ceccato et al. (2011) analysed five case studies in Austria, Germany, India, Bhutan, and China.The authors found out that although the studied watersheds were characterized by distinct ecological, social, and economic dimensions, the criteria selected by the stakeholders were rather similar.In this regard, multiple-case studies allow findings to be compared, parallels to be drawn, and differences across diverse cultural, environmental, and governmental contexts to be examined.

Trends by MCDM method
Results showed that AHP and its family of methods were by far the most used MCDM approach (Table 4).One reason for this might be that its structure is straightforward, flexible, and easily understandable (Cinelli et al., 2014).Thanks to these characteristics, it can be adapted to different problems without requiring previous knowledge from the analyst.Moreover, several software packages incorporate AHP (e.g.DECERNS, ExpertChoice, MakeItRational, DECERNS, EasyMind and Super decisions), including GIS (Geographic Information System) software (e.g.ArcGIS, Idrisi, and ILWIS).The second most employed method was TOPSIS, closely followed by SAW.These results, with a few differences and similarities, were confirmed by other MCDM review papers such as Jato-Espino et al. ( 2014) and Broekhuizen et al. (2015) that ranked AHP as the first and TOPSIS as the second method with more applications.
Note that the sum of the applications (165 items) in Table 4 does not match the number of papers (128 items) since some articles used several MCDM techniques to analyse differences in scoring and ranking.For example, Chitsaz and Banihabib (2015) compared seven MCDM tools and concluded that ELECTRE III stood superior to select flood management options.On the other hand, Chung and Lee (2009) employed The survey also showed that MCDM techniques are not used only in a stand-alone mode, but are commonly extended and combined with soft computing technologies, including fuzzy set theory (e.g.Chen and Hou, 2004;Guo et al., 2014), artificial neural network (e.g.Radmehr and Araghinejad, 2014;Liu et al., 2014), and tools such as SWOT (Strengths, Weaknesses, Opportunities, and Threats) analysis (e.g.Vafaei and Harati, 2010;Miyamoto et al., 2014).Furthermore, there are also numerous hybrid methods, developed to address gaps in classical techniques (e.g.Yang et al., 2013;Shams et al., 2014).This suggests that MCDM is versatile, enabling researchers to combine it effectively with different tools according to the requirements of the decision to be taken.
Overall, AHP is the most prominent MCDM method in all application areas, except for reservoir flood control (Table 5).The primary reason for the popularity of AHP for mapping the risk and its components is that the implementation of this technique within the GIS environment is straightforward, enabling the users to quickly derive the weights associated with criteria map layers (Malczewski, 2007).For reservoir flood control, miscellaneous methods such as fuzzy hybrid approaches were the preferred techniques.This is probably because reservoir operations involve a large number of uncertain factors that can be properly addressed by fuzzy set theory.Additionally, TOPSIS is highly popular for ranking alternatives for flood mitigation, which emphasizes the effectiveness of this technique to deal simultaneously with conflicting objectives.
Although the most widespread MCDM methods were used at least once, no study has used DEMATEL (Decision- Making Trial and Evaluation Laboratory) or ORESTE (Organization, Rangement Et Synthese De Donnes Relationnelles).A likely explanation is that these methods are cognitively demanding when compared to classical approaches, especially when numerous criteria are involved (Dou et al., 2014;Moffett and Sarkar, 2006).For instance, DEMATEL needs to be coupled with other MCDM tools, such as ANP to generate criteria weights, which makes its application difficult.In addition, there is a limited amount of software available, and most of it costs (e.g.Decision Era).However, DEMATEL was specifically developed to address limitations of traditional techniques regarding interdependence between criteria.Likewise, ORESTE is suitable for problems with limited information and with incommensurable criteria (Moffett and Sarkar, 2006).

Trends regarding stakeholders' involvement
Flood risk management decisions may be designed without the direct participation of multiple stakeholders.However, they cannot be implemented without them (Affeletranger, 2001).Therefore, flood management decisionmaking should be ideally carried out in a participatory process, where the knowledge and preferences of interested actors are integrated into the process from the beginning.According to Evers et al. (2014), this creates trust among decision makers and stakeholders, which often lead to a successful implementation of the chosen measures.
The survey revealed that 65 (50.78 %) studies have explicitly acknowledged the involvement of multiple actors in the decision-making process.Policy makers and experts were the stakeholders that participated most.This was expected since they are often responsible for the selection and implementation of chosen measures and have a broad knowledge of the problem of interest.Additionally, some papers mentioned the involvement of local community members (e.g.Kandilioti and Makropoulos, 2012;Sahin et al., 2013;Roy and Blaschke, 2015).According to Affeletranger (2001), the consideration of community members' opinion may improve their resilience as well as their response capacity when confronting natural disasters.
Nevertheless, participation was generally fragmented and restricted to consultation at specific stages, such as the selection of evaluation criteria (e.g.Haque et al., 2012) and the definition of criteria weights (e.g.Kienberger et al., 2009;Sahin et al., 2013).This segmentation may be related to methodological and time constraints since participatory decision-making is time-consuming and costly, particularly when the decisions are made in a group where proper facilitation is required.
Crucial aspects of the decision-making process like the definition of objectives, identification of the alternatives, and estimation of its consequences were usually constrained to analysts and experts, which inhibit the achievement of genuine participation.Only in exceptional cases was the input from the stakeholders a critical element in the entire process (e.g.Ceccato et al., 2011;Evers et al., 2012).For example, Ceccato et al. (2011) developed a methodological proposal aimed at strengthening the communication and collaboration within the scientific community and local actors for flood management decision-making.The authors applied the Net-SyMoD (Network Analysis -Creative System Modelling) framework (Giupponi et al., 2008), where the identification of relevant stakeholders, definition of the problem, establishment of objectives and criteria, and the selection of alternatives are conducted in a participatory process.
Another interesting result is that only four studies seek to obtain consensus (e.g.Haque et al., 2012;Lee et al., 2013Lee et al., , 2014Lee et al., , 2015)), in which participants make decisions by agreement rather than by majority vote or averaging approaches.Nevertheless, enhancing mutual understanding for consensus building is essential for a long-lasting and successful flood management program, especially for selecting alternatives for flood mitigation and emergency management.It allows decision makers to derive meaningful solutions that fulfil their own needs while at the same time satisfying the requirements of other actors, legitimating the participation as a learning process to solve complex problems.
A total of 43 out of 65 studies provided unambiguous descriptions of the participatory decision-making techniques applied.Figure 4 shows that questionnaires (e.g.Giupponi et al., 2013;Taib et al., 2016) and face-to-face interviews (e.g.Deshmukh et al., 2011;Jun et al., 2011) were the most applied tools.These methods allow for opinions to be conveyed without influence from dominant participants and are simple and fast to realize.On the other hand, the participants are not able to share and hear different perspectives through open dialogue, which is essential for achieving common agreement.
In this sense, Mendoza and Martins (2006) argue that group elicitation methods involving open discussion offer several advantages, including the consistency in the information obtained, and a better definition of the preferences.On the other hand, the results can be influenced by dominant stakeholders and noise in the responses (Hsu and Sandford, 2007).Group elicitation methods such as workshops (e.g.Kenyon, 2007;Porthin et al., 2013), group meetings (e.g.Azibi and Vanderpooten, 2003;Marttunen et al., 2013), and focus group discussions (e.g.Rahman and Saha, 2007;Haque et al., 2012) were less applied in the reviewed papers.
Recently, researchers have used the Delphi technique to overcome shortcomings of conventional group elicitation methods regarding dominant individuals and time constraints (e.g.Chung et al., 2014;Lee et al., 2014).This method provides anonymity to respondents, a structured feedback process, and is suited for consensus building (Hsu and Sandford, 2007).Additionally, it is advantageous when the stakeholders live some distance apart, and it is prohibitive to bring them together for a workshop or group meeting (Lee et al., 2013).
It is interesting to highlight that two studies reported the use of collaborative web-based platforms in which stakeholders select and rank alternatives interactively (e.g.Evers et al., 2012;Almoradie et al., 2015).These platforms have the potential to overcome hindrances in participatory MCDM such as the limitation of financial resources and stakeholders' spatial distribution, providing full transparency of information and results.By taking this approach, the confidence in the decision-making process is increased as well as the level of acceptance of negotiated measures, which are crucial conditions for successful participatory flood risk management.

Trends regarding sensitivity and uncertainty analysis
Flood decision-making is subjected to multiple sources of uncertainty, including the assessment of criteria weights, the parameters' uncertainties, and structural uncertainty (Broekhuizen et al., 2015).In addition, there are uncertainties associated with the inherent randomness of flood events (Merz et al., 2008), which, in principle, cannot be reduced.Thus, in order to improve the quality of decisions and verify the robustness of the model outputs, flood risk management should be based on a comprehensive assessment of the sensitivity combined with a thorough investigation of the uncertainties involved.
In this review, 93 (72.65 %) papers do not report any kind of sensitivity analysis, thereby ignoring the impact of changes in input weights on model results.The remaining articles (35 or 27.34 %) applied mainly one-way sensitivity analysis, where one criteria weight or performance score is modified at a time and the variation of the alternatives' ranking is observed.If the induced variation does not change the rank order of alternatives, the decision is considered robust.This technique is intuitively appealing and requires little time, making it a practical way to assess the sensitivity.Even though this method is sufficient for most flood applications, the range over which weights are varied is normally arbitrarily defined, and the commutative impact of uncertainty is not considered.Hence, these drawbacks may lead to a biased view of the influence of uncertainty on the final decision (Broekhuizen et al., 2015).
Two papers performed global sensitivity analysis (GSA) by applying the FAST (Fourier Amplitude Sensitivity Test) procedure, where two or more evaluation criteria are varied at the same time (e.g.Fernández and Lutz, 2010;Chen et al., 2015).Although GSA allows for the full uncertainty range of the criteria to be explored and analysed, it can become an extremely time-consuming task, as a large number of criteria are included in the analysis.Additionally, four papers elaborated best-and worst-case scenarios to incorporate decision makers' attitude to risk (e.g.Kandilioti and Makropoulos, 2012;Penning-Rowsell et al., 2013;Ghanbarpour et al., 2013;Alipour, 2015).Finally, two studies used a probabilis-tic approach (e.g.Yazdandoost and Bozorgy, 2008;Fernández and Lutz, 2010), which is the most rigorous form of sensitivity analysis.This approach requires the estimation of a maximum percentage that the actual criteria weight may differ from the estimated value.
Several authors have listed the uncertainty as a major drawback (e.g.Bana e Costa et al., 2004;Edjossan-Sossou et al., 2014;Godfrey et al., 2015;Almoradie et al., 2015).However, only eight papers (6.25 %) perform uncertainty analysis in an attempt to describe the entire set of possible outcomes, together with their associated probabilities of occurrence.In situations where uncertainty is mainly due to randomness, the methods used were probability-based.This is the case of Qi et al. (2013) and Li (2013) who used Monte Carlo simulation to convert uncertainties in input criteria into probability distributions.Another approach applied was the Taylor's series error propagation method (e.g.Fernández and Lutz, 2010), which analyses how the uncertainty in input data propagates through the model and affects its outputs.In addition, three papers assessed the uncertainty qualitatively, by describing its main sources (e.g.Cozannet et al., 2013) or by analysing the degree of confidence related to stakeholders' opinion (e.g.Ceccato et al., 2011;Penning-Rowsell et al., 2013).
Apart from uncertainty and sensitivity analysis, fuzzy set theory is widely combined with AHP, TOPSIS, and CP to handle uncertainty and incomplete information about the decision situation.For instance, Lee et al. (2013) integrated TOPSIS and fuzzy set theory to fuzzify the weighting values and all criteria maps.In the same sense, the approach proposed by Yang et al. (2012) combines AHP and triangular fuzzy number to assess the flood risk and its components.Fuzzy set theory is widespread in MCDM due to its intuitiveness and computational requirements.Nevertheless, some studies have shown that fuzzy AHP do not provide better results than regular AHP since the judgments in AHP are already fuzzy (Saaty, 2006).Therefore, the additional complexity of utilizing fuzzy numbers may be unnecessary in some cases.
Finally, it is relevant to note that some MCDM methods explicitly account for uncertain input criteria scores.For instance, ELECTRE and PROMETHEE adopt the pseudocriterion model that introduces indifference and preference thresholds.Likewise, MAUT considers imprecise data input with probabilistic approaches (Cinelli et al., 2014).Also, AHP allows the generation of an inconsistency index, which can be considered as an indirect measure of the uncertainty in the criteria weighting step.

Research limitations and recommendations for future research 5.1 Limitations
There are some caveats that should be considered when interpreting the results obtained in this review.One of the main shortcomings is that the papers' quality was not evaluated since they were all published in peer-reviewed journals.Thus, some applications were superficial, while others were detailed, including intensive stakeholder participation, validation of results, and probabilistic-based uncertainty and sensitivity analysis.Some studies were carried out with real data, involving real decision makers and stakeholders, while others discussed hypothetical applications or were secondary studies that re-examined empirical work.A future review paper can address this limitation by applying heuristic checklists (e.g.Beecham et al., 2008) to assess the overall quality of the study.
In addition, defining the flood application area turned out to be a subjective process, especially when it came to distinguishing between susceptibility, hazard, and risk assessment.There is a misunderstanding about these terms in the literature, which are used in slightly different ways by researchers with different backgrounds.Thus, in some cases, it was difficult to define a clear line for when it was susceptibility, hazard, or risk.Whenever possible, the term used by the authors was respected.
The exclusion of non-English literature can also be understood as a limitation (Behzadian et al., 2010).The results of our preliminary searches showed that several MCDM French school authors have published in French language journals.Furthermore, there are a significant number of research papers published in German, Chinese, and Korean.Thus, it should be emphasized that, when feasible, searches using multiple languages are advantageous (Pickering and Byrne, 2014).
Nevertheless, despite these potential limitations, this paper is the first to present a literature review of the state of the art of the use of MCDM for flood-related problems.The sample of papers analysed provides sufficient information to stimulate discussion and research that addresses challenges in this area of knowledge.

Recommendations for future research
This review enabled us to identify gaps in the knowledge of MCDM for flood risk management regarding several aspects.First, classical MCDM methods such as MAUT, MAVT, PROMETHEE, and DEMATEL were overlooked.Almost half of reviewed applications used AHP to elicit criteria weights, which is a relatively easy and flexible method, requiring fewer skills than other tools.In this sense, exploring the implications of methodological differences in existing MCDM methods for flood risk management is an interesting research challenge.Similarly, future research can focus on understanding advantages and limitations of each method for handling different sources of uncertainty.
Secondly, there were surprisingly few studies that effectively considered stakeholders' participation throughout the entire decision-making process.Therefore, greater rigour in endorsing an active participation in all stages of the decisionmaking process should be undertaken, in order to increase the feasibility and subsequent implementation of chosen measures.Future research could be directed towards developing web platforms to elicit stakeholders' preferences, aiming to reach consensus in a simpler and easily accessible way.In addition, this course of action can be combined with other participatory techniques such as cognitive mapping, Delphi technique, and voting theory.Conversely, it should be noted that intensive participation is time-consuming.Thus, in reallife applications, trade-offs have to be made between the available resources and the expected outcomes of the MCDM process.
The third challenge, and perhaps the most relevant research gap, refers to fully considering the uncertainties around decision makers' judgments.Although uncertainty in MCDM is not a new problem and significant improvements have been made over the last decades, it remains a major open issue.Previous studies suggest that properly addressing the uncertainties can substantially improve MCDM applications, assisting stakeholders to make better decisions.Potential exists to apply Bayesian framework methods (e.g.Bayesian networks and Dempster-Shafers' theory), possibility theory, and evidence theory.Additionally, innovative approaches may be required to account for special characteristics in developing countries, where limitations in resources and technology exist.However, regardless of the uncertainty method applied, considering all sources of uncertainty in the decision-making process might not be a feasible task (Mowrer, 2000).Nevertheless, it is essential to identify as many sources of uncertainty as possible, and attempt to reduce or handle them.
Lastly, a significant gain can be made if flood MCDM applications are able to consider climate and socioeconomical changes, which have potential to aggravate existing risks.This has been tackled in a recent study by Giupponi et al. (2013) that assessed the flood vulnerability within the broad context of climate change adaptation.

Conclusions
This study has presented a systematic review of 128 papers that apply MCDM to flood-related problems, aiming to provide an overall picture of what has motivated researchers and practitioners in 37 different countries over the past 2 decades.Our findings suggest an increasing interest in flood MCDM since 2009, as compared to the previous 14 years.A wide range of applications were identified, with most papers fo-cusing on ranking alternatives for flood mitigation, followed by risk, hazard, and vulnerability assessment.This highlights the utility of MCDM as a decision support tool in all stages of the flood management process.
Nearly 85 % of the applications were conducted in Asian and European countries, mainly in China, Germany, and South Korea.Hence, potential exists to develop crosscountry investigations, especially in South America and Australia.Overall, AHP was the most widespread method, indicating that other methods may be overlooked.The review also shows that fuzzy and hybrid approaches (e.g.triangular fuzzy AHP, hybrid fuzzy AHP-TOPSIS, AHP-SWOT, modified TOPSIS) are being increasingly applied to overcome limitations of classical methods.
About half of the studies have acknowledged the involvement of multiple stakeholders.However, participation was fragmented and focused on particular stages of the decisionmaking process.Most of the reviewed studies rely on the use of questionnaires and interviews to capture stakeholders' perspectives, with few applications seeking to obtain consensus.In addition, shortcomings remain in handling the uncertainty.Thus, greater rigour in considering the uncertainties around stakeholders' preferences and endorsing an active participation are important research gaps.Additionally, sensitivity analysis should be conducted as a primary method to check the stability of the results and identify the most critical criteria.This could help to increase the quality of decisions as well as the transparency and credibility of the MCDM outcomes.
It is clear from the literature that the challenge for further research is to foster the development of true collaborative MCDM applications that take the uncertainty around stakeholders' judgments into account.We believe that this paper can provide valuable information for guiding future research and that it can serve as a ready reference for researchers and practitioners working with flood risk management and MCDM.

Figure 1 .
Figure 1.Number of MCDM flood papers published over the period 1995-June 2015.

Figure 2 .
Figure 2. Normalized number of MCDM and flood papers published between 1995-June 2015, based on data from the Web of Science and Science Direct.

Figure 3 .
Figure 3. Distribution of MCDM papers by application area between 1995 and June 2015.

Figure 4 .
Figure 4. Methods used to incorporate multiple stakeholders' views in the decision-making process.

Table 1 .
Description of different MCDM methods cited in the reviewed papers.

Table 2 .
Distribution of applications by flood risk management topic.

Table 3 .
Distribution of applications by country of application.

Table 4 .
Distribution of applications by MCDM method.

Table 5 .
Distribution of applications by MCDM method and area of application.It is important to highlight that some papers analysed two or more flood problems simultaneously by using the same MCDM method.Thus, the number of applications in Table5is higher than in Table4.