Cahier d’Études et de Recherche / Research Report A MULTI … · 2010-12-10 · (International...
Transcript of Cahier d’Études et de Recherche / Research Report A MULTI … · 2010-12-10 · (International...
Ecole Centrale Paris
Laboratoire Génie Industriel
Cahier d’Études et de Recherche / Research Report A MULTI-CRITERIA ASSISTANCE TO THE CHOICE OF RISK MANAGEMENT METHODS IN PROJECTS Franck MARLE et Thierry GIDEL CER 10– 24 Décembre 2010
1
A MULTI-CRITERIA ASSISTANCE TO THE CHOICE OF RISK MANAGEMENT METHODS IN PROJECTS
Franck MARLE et Thierry GIDEL (UTC)
Abstract: As projects are facing tight constraints, uncertainty and change, risk management is a very
important issue in project management. Our goal is to provide a project office manager or a project
manager with one or more adequate Project Risk Management (PRM) methods. In order to achieve
this goal, we propose a typology of PRM methods and a list of criteria that should be considered when
choosing the methods, by screening and ranking. Finally, we propose a Multi Criteria Decision
Making (MCDM) model that could be used to select the methods. An application on an industrial case
study is presented and some conclusions and perspectives are drawn.
Keywords: multi-criteria decision-making; fuzzy numbers; method choice; project management; risk
identification; risk analysis
2
1. Introduction
It is admitted today that Project Risk Management (PRM) has an important role to play on Project
Management and project success rate. This is particularly true in the development of a new product or
system, which is a wide process that includes both project and product lifecycle aspects. In this paper,
we focus on the choice of a relevant PRM method in new product/system development projects. Our
goal is to provide project office managers, project managers or any decision–maker (like risk
managers) with a framework and its associated tools to make the adequate choice. The new product
development project is extremely complex and involves the implementation, not only of product
development practices, but also of project management activities. Standards like ISO 10006
(International Standards Organization), PMI (Project Management Institute) or IPMA (International
Project Management Association) establish that project management consists in the planning,
organization, monitoring, control and reporting of all the aspects of a project, and in the motivation of
all people involved in reaching the project objectives (ISO 2003; PMI 2004; IPMA 2006). According
to PMBOK (PMI 2004) and AFNOR NFX50-117 standard (AFNOR 2003), project risk is defined as
“an uncertain event or condition that, if it occurs, has a positive or negative effect on at least one of the
project objectives”. If these risks are not managed in a pro-active way using a structured approach,
then they can result in serious consequences for the project, as said in ISO 10006 (ISO 2003). PMI
describes the PRM purpose as “the increase of probability and impact of positive events, and the
decrease of probability and impact of negative events” (PMI 2004). As a consequence, various risk
management methodologies have been developed : some standards have proposed risk management
methodologies, which are specific to project context or generic (IEC 1995; APM 1996; AFNOR 1997;
AFNOR 1999; IEEE 2001; BSI 2002; PMI 2004; IPMA 2006). They may have been introduced in
different fields, like project management, systems analysis, design, insurance, food industry,
information systems, chemical systems, industrial safety. Yet, when looking at companies practices,
once can observe that PRM methods are not so widely used (Coppendale 1995; Lyons and Skitmore
2004). When a method is implemented on a project, it is often either imposed by a corporate standard
3
or chosen by the project leader, because he already has tested it. Of course, leading companies and
organisations are implementing various PRM methods. For instance, according to (Tumer, Barrientos
et al. 2005) and (Kurtoglu and Tumer 2007), NASA implements various PRM methods like Failure
Modes and Effects Analysis, Fault Tree Analysis, Probabilistic Risk Analysis, etc. which will be
detailed in the following part 2.
The first issue addressed in this paper is that the PRM method used in a company may have been
chosen for wrong reasons or for historical reasons which are obsolete in the current context. The
second issue is that a given company may have several methods fulfilling the same needs, often for
historical reasons of local use of each method. This makes it difficult to combine these methods in an
efficient and effective way.
To tackle these issues, we define a Multi-Criteria Decision-Making (MCDM) process which will
evaluate, eliminate and rank PRM methods alternatives. An industrial case study is presented in order
to test a first implementation of the methodology. For defining a MCDM process, several questions
have to be addressed, like the goal (choosing, sorting, ranking, screening), the possible alternatives
(the PRM methods), the selected criteria, the mode of evaluation of each alternative among each
criterion, the mode of aggregation for evaluating alternatives using their local evaluations and the
mode of selection regarding the final evaluation and ranking of the alternatives. In this paper, we
chose to use a combination of screening and ranking, and a combination of qualitative and fuzzy
evaluations. These evaluations are aggregated using a geometric weighted average.
The paper is then organized as follows. The application field of Multi Criteria Decision-Making is
presented, with a focus on the use of fuzzy set theory as some judgements being subjective and not
precise enough (part 2). A literature review of PRM methods enables us to propose a typology of these
methods and a list of criteria to select them (part 3). Then, our PRM method choice process is
introduced (part 4). The decision-making process is detailed in three steps, some of them having a
variable depth degree depending on the ambition of the decision-maker and on the context of the
4
company. Finally, an industrial case study is presented (part 5) and some discussions and conclusions
are drawn (part 6).
2. Background on Multi-Criteria Decision-Making
The expertise required to choose a method may be very deep, because of the use of some advanced
concepts, like Markov, Monte-Carlo or logical gates, etc… Usually, only experts of the field are able
to take full advantage of a PRM method. The issue of PRM methods choice (RIM/RAM) becomes
even more important when it is applied on a decentralized way, sometimes with local or web-based
software, without any technical support, from a project office for instance, or a risk manager or an
expert. Both the characteristics of PRM methodologies and preferences of the decision-maker are to be
modelled. The goal is to use a Multi Criteria Decision Making (MCDM) model that enables to match
the decision maker’s preferences. Preferences are often uncertain and expressed by linguistic terms,
such as « good », « very much », « I prefer », which may require the use of fuzzy set theory in addition
of the use of a MCDM method. Classical MCDM methods suppose to conduct an evaluation of some
alternatives regarding some criteria, by using qualitative or quantitative scales, crisp or fuzzy values,
and direct or comparison-based evaluations. Our goal in this paper is to propose one or more adequate
PRM methods to the decision-maker, which supposes to combine screening and ranking. Some
evaluations are based on facts and are quantitative, and some are based on human judgement and may
be qualitative or even sometimes fuzzy. This paragraph introduces general MCDM problems and
methods. It consists in describing the possible objectives, the classical steps of decision-making
process, and the classical principles and approaches. We will explain why we decided to use in our
research method a combination of screening and ranking, and a combination of qualitative and fuzzy
evaluations.
2.1. Possible objectives for a MCDM problem
Three main issues exist in MCDM:
5
• Choice: choose the best alternative. For a basic decision-making problem of choosing one or
several best alternatives, it is useful to begin by eliminating those alternatives that do not
appear to warrant further attention, which is called screening (Hobbs and Meier 2000).
Screening is the process that reduces a set of alternatives to a smaller set of alternatives that
(most likely) contains the best one. It supposes to have some elimination thresholds, or
intervals, on evaluation scales.
• Ranking: rank all alternatives from best to worst. It supposes to have a global evaluation
model for each alternative, taking into account all the considered criteria and their different
scales.
• Sorting: sort all alternatives into different pre-ordered groups
In this work, we consider only screening and ranking categories.
2.2. Classical steps for a MCDM process
A global MCDM problem follows the serial process of:
• defining decision objectives,
• identifying and arranging criteria, with potential interdependencies,
• identifying and arranging alternatives, with potential interdependencies,
• evaluating criteria , with weights and thresholds,
• evaluating alternatives for each criteria and with a global model,
• screening out alternatives which do not fit to criteria thresholds
• ranking remaining alternatives according to their individual evaluations and criteria weights
• making the decision.
For a set of criteria C={Ci}, i=1 to NC (number of criteria) and a set of alternatives A={Aj},j=1 to NA
(number of alternatives), the evaluation of Aj regarding Ci is called Eij.
7
2.3. Classical principles for the decision-maker’s preference expression
Two kinds of evaluations do exist: the values (preferences on consequences) and the weights
(preferences on criteria). Basically, they can be evaluated whether directly or indirectly via relative
comparison (often pair wise comparison). Moreover, they can be quantitative, or qualitative, or even
fuzzy when the degree of precision and reliability is not enough to get crisp qualitative evaluations.
For instance, linguistic decision-making uses linguistic expressions on criteria as constraints, and is
mainly used for screening. We distinguish lexicographic constraints and disjunctive/conjunctive
constraints. In the first case, criteria are ranked in order of relative importance, and all alternatives are
examined to assess whether the first criterion is satisfied. For those alternatives which are not screened
out, the process goes on to the second criterion, and so on until the last. Disjunctive or conjunctive
constraints express conditions involving more than one criterion, and are characterized by the use of
« and » and « or » operators.
For the values, the most known and used models are utility theory (KEENEY and RAIFFA 1976) and
outranking (Vincke 1992). It consists in a transformation of raw consequence information into
preference information which is useful for the decision-maker. Pareto optimality is a well-known
concept introduced by the famous scientist (Pareto 1971) which takes into account multiple criteria for
overall optimality. It is based on the domination concept, and is also called efficiency of an alternative.
A1 dominates A2, denoted A1 > A2, iff ∀ �??1.. ??, 1 ≥ 2 , with at least one strict inequality. Following this definition, an efficient alternative, also called Pareto
optimal alternative, is a non-dominated alternative:
8
A1 is a Pareto optimal alternative iff ∄ � > 1. For the criteria, it refers to expression of the relative importance of criteria. Usually, for each criterion
Ci, the associated weight Wi is strictly positive, and:
Two kinds of weights, trade-off weights (TW) and non trade-off weights (NTW) are defined in
(Belton and Stewart 2002). For TW, preferences are compared as they are aggregated into a single
expression, a phenomenon called compensation. Trade-off weights are essentially used for classical
aggregation models, like additive or average-based methods. It enables to study some phenomena like
sensitivity analysis (Rios Insua 1990), dominance and potential optimality (Hazen 1986;
Athanassopoulos and Podinovski 1997), which consist of changing inputs (values or weights) to look
at consequences on outputs. Non Trade-off weights, also called outranking, was introduced in (Roy
1996). An outranking relation is a binary relation S defined on A, with the interpretation that ASB if
there are enough arguments to decide that A is at least as good as B, while there is no essential reason
to refute that statement (Vincke 1992).
In this paper, we do not consider outranking methodologies but our evaluation will be done according
to single synthesis criteria principle, using multi-criteria aggregation and Pareto domination principle.
We chose to use a single weighted geometric mean. This is simple enough to avoid problems due to
methodology pre-requisites for our users. The choice of a geometric weighted average is made to
screen out methods which do not correspond to minimum thresholds. Finally, it permits to give a
better global assessment to balanced solutions, which is not the case with arithmetic mean.
For instance, (0,5*0,5)½ = 0,5 > (0,9*0,1)½ = 0,3, whereas (0,5+0,5)/2=(0,9+0,1)/2=0,5
2.4. Fuzzy set theory
9
Most of the preferences of the decision-makers are expressed by using linguistic expression, such as
« very likely », « highly preferable », « do not like ». They involve the use of fuzzy set theory, first
introduced by (Zadeh 1965; Zadeh 1975), and applied to decision-making by (Bellman and Zadeh
1970). Some definitions are given for usage in this paper, extracted from several references, like
(Kaufmann and Gupta 1991; Zimmermann 1991; Klir and Yuan 1995).
Definition 1 : a fuzzy set ñ in a universe of discourse X is characterized by a membership function
μñ(x), which associates with each element x in X a real number in the interval [0,1]. The function
value μñ(x) is termed the grade of membership of x in ñ (Kaufmann and Gupta 1991).
Definition 2 : a fuzzy set ñ is convex iff (Klir and Yuan 1995):
for all (x1, x2) in X and λ in [0,1]
Definition 3 : a fuzzy set ñ is normalized if its height is equal to 1 (Klir and Yuan 1995). The height is
the largest membership grade attained for any x in X.
Definition 4 : a fuzzy number is a fuzzy set that is both convex and normal (Kaufmann and Gupta
1991).
Definition 5 : a positive triangular fuzzy number ñ can be defined as (n1, n2, n3), with the following
membership function μñ(x) :
The fuzzy number is symmetrical iff n2=(n1+n3)/2.
10
Definition 6 : a linguistic variable is a variable whose values are linguistic terms, like very low, low,
medium, high and very high. Fuzzy numbers can represent these linguistic values in situations where
quantitative expressions or reliable qualitative expressions are not possible. In this paper, positive
triangular fuzzy numbers are used to express linguistic variables.
Definition 7 : in the case where several decision-makers give their opinion in terms of fuzzy numbers
ñk, k=1 to NDM (Number of Decision-Makers), then the aggregated fuzzy number ñ can be defined as
explained in (Amiri, Zandieh et al. 2009) :
Some methodologies have been developed thanks to the use of fuzzy set theory, like fuzzy Standard
Additive Weighting model, fuzzy weighted product model, fuzzy AHP, revised fuzzy AHP and fuzzy
TOPSIS, studied and compared in (Triantaphyllou and Lin 1996). A combination of principles of AHP
and fuzzy set theory gives the fuzzy AHP method, which has been applied by (Shamsuzzaman, Sharif
Ullah et al. 2003). In our case, we are going to use fuzzy weights, for expression of preferences of the
decision-maker on the criteria importance.
3. Background on Project Risk Management methods and proposal of a choice criteria list
We identify the main PRM methods and their main characteristics according to a literature review
focused on a local analysis (a specific method, a specific company, a specific journal or conference)
and on a global analysis (research by keywords, by application fields and knowledge areas). A list of
criteria that will be used for the choice is introduced as a synthesis of this part.
3.1. Description of the methods
The Project Management Institute (PMI 2004) presents project management in nine knowledge areas:
integration, scope, time, cost, quality, human resources, communications, procurement and risk
management. Risk management consists in the treatment of the project uncertainties through a
structured, four steps generic approach (PMI 2004):
11
1. Risk identification: describes the identifiable risks, that is to say the potential events that could
occur and lead to negative or positive impact on the project,
2. Risk analysis: analyzes causes and consequences of the identified risks, in order to evaluate
their criticality, mainly by assessing probability and impact.
3. Risk treatment (or response planning): decides tasks, budgets and responsibilities in order to
avoid, mitigate or transfer some risks, often the most critical in priority.
4. Risk monitoring and control: follow-up, by the identified responsible persons, of the planned
actions and of their impact on the criticality of the risks.
As detailed in (Marle and Gidel 2010), we propose to present the methods according to a typology
based on the nature of the identification: analogical, heuristic and analytical (Table 1).
Table 1: classification of PRM methods according to three types of risk identification approach
Finally, we have considered 32 RIM and 19 RAM, briefly described below in tables 2 and 3.
Particularly, table 3 indicates to which type each RIM belongs. A method may be used for more than
types, meaning that it can be used with or without experience.
12
Table 2: list of Risk Identification Methods (RIM) and their classification according to
experience/expertise
14
3.2. Building a list of criteria
As there are plenty of PRM methods, it is hard for a decision-maker who is involved at the project
level or at the multi-project level, to select the most suitable methods for his particular context. We
propose a three level decomposition in order to identify the criteria that could be used to choose a
PRM method: the organization level, the project level and the decision-maker level. The description of
each criterion is detailed in (Marle and Gidel 2010). We then propose the following criteria for
selecting PRM methods, as described in Table 4.
The first four criteria (Organization level) are mandatory, because it would be risky for the company to
implement a method if it is not mature enough. They are measured for a company and are compared to
a type of methods (analogical, heuristics, analytic). The last criteria (Project and Decision-maker
levels) are preferences, as the company prefers a method which corresponds to its specific needs. Of
course, as these criteria are deducted from a literature review, they could be discussed, improved and
further tested to confirm their pertinence.
16
Finally, the table 5 shows the evaluation of RIM and RAM among considered criteria.
Table 5: evaluation of each RIM and RAM according to considered choice criteria
17
4. Development of a multi-criteria decision-making process for Project Risk Management
method choice
The different PRM methods introduced in previous section are potential candidates for the company.
The choice is in three steps, which consist in gathering and treating data about methods, criteria and
company (see figure 1):
• A selection of the RIM which correspond to company maturity,
• A selection and ranking of the RIM/RAM methods among the company preferences,
• A final choice of a combination {RIM+RAM} of methods depending on their ranking, their
links and their implementation effort.
The output of these three steps is the decision, followed by the implementation of the choice.
1
2
3
4
DATA ABOUT THE METHODS AND
CRITERIA
DATA ABOUT THE COMPANY
DATA TREATMENT IN DECISION-MAKING PROCESS
1
2
3
IMPLEMENTATIONDECISIONStep 1Screen out
Step 2Screen out and rank
Company maturity Company preferences Company maturity
Assessment of methodsregarding criteria
Implementation issue
Compatibility
Step 3
RiskIdentification Methods
RiskAnalysisMethods
Figure 1: description of the decision-making process by screening and ranking
18
4.1. Step 1 : first selection of RIM regarding company maturity
Four criteria are introduced to define what company maturity in project risk management is. They are
called Company Maturity Criteria (CMCj), j=1 to 4. For each of these criteria, it is asked to the
decision-maker to assess his company, in order to get evaluations called CMj for Company Maturity
for the jth criteria. As seen in part 2, three types of RIM do exist: analogical, heuristic and analytic.
They are called Tk, k=1 to 3. For each type, a minimal maturity is mandatory in order to be able to
implement correctly a method of this type. We then define a threshold for each type Tk and each
criterion CMCj, called MTjk (for Maturity Threshold). We have the following constraint:
For each k [1..3], if there is j [1..4] such as CMj < MTjk, then the type Tk is not adequate for the
company and is screened out.
Then, we evaluate whether the RIMi belong or not to each type Tk. This binary matrix presented in
part 2 enables us to know whether each RIMi can be applied or not regarding company maturity. There
are two possibilities:
• RIMi belongs to only one Tk : if Tk is screened out, then RIMi is screened out
• RIMi belongs to more than one Tk (2 or 3) : if one Tk is not screened out, then RIMi is not
screened out. That means, that even if RIMi cannot be applied in a certain way (analytic for
instance), it can be applied in another way (analogical for instance).
From the initial list of RIM, this step gives a shorter list of potential candidates for next step.
4.2. Step 2 : selection and ranking of RIM/RAM regarding company preferences
This step is detailed for RIM; the same principles are applied for RAM. Only the terms and the values
change. It can be expressed as step 2a and step 2b, where step 2a is detailed below for RIM and step
2b is identical for RAM. The goal of this step 2a is to study the fitness of remaining RIM alternatives
to company’s preferences. These preferences are expressed among some criteria, called Company
Preferences Criteria CPCj. These criteria are expressed with a weight and a minimal threshold,
19
respectively called RIwj and RImtj. We then have to evaluate each RIMi among each CPCj, called Eij
and give a global evaluation Ei (Equation 1):
(1)
With the constraint: RIMi is screened out (RIMi=0) iff there exists j such as Eij< RImtj
We reformulate the constraint as following (Equation 2):
Eij [1..5] except if Eij<RImtj where Eij=0 (2)
In the case where NDM decision-makers give their opinion, then we have NDM expressions for each
threshold, called MTik, k =1 to NDM and the final threshold MTi is obtained as the maximum of the
individual thresholds. This enables to satisfy all the decision-makers, even if it is harder in terms of
selection.
The particularity of evaluation of company’s preferences is the use of fuzzy weights. They are
expressed on a fuzzy scale which transforms linguistic preference judgements into numerical intervals
on a scale [1..10]. The Figure 2 shows this scale.
0
0,2
0,4
0,6
0,8
1
1,2
1 2 3 4 5 6 7 8 9 10
N
VL
L
A
H
VH
Figure 2: description of the fuzzy scale used in our approach
20
It is to be noticed that some levels are particular:
• The level Negligible is not exactly triangular as n1=n2=1.
• The level High is larger than the others. It enables to make a bigger difference in scores when
preference is High or Very High. The goal is to emphasize the most important criteria for the
decision-maker, and try some avoid compensation phenomena.
As we have fuzzy weights, we obtain a fuzzy score by calculation of three scores for each
corresponding weight. A defuzzification formula is applied in order to obtain a final global score for
each method. The methods are ranked according to their global score. A multi-criteria Pareto
optimality analysis is done. The top 5 of the non-dominated methods is presented to the decision-
maker with individual and global scores. This step permits to screen out RIM which do not correspond
to company requirements, then to rank the remaining RIM in order to make the final choice described
in step 3.
4.3. Step 3: the final choice of a combination of (RIM+RAM)
This step may be done very quickly by choosing the first method in each ranking. We give hereafter
three ideas for refining this brutal choice:
• The significance of the gaps in the ranking
The reliability of the outputs depends on the reliability of the inputs. The mistakes and imprecision are
amplified or eliminated by the aggregated calculation. So, we argue that there should be a large
enough gap between two solutions in order to decide to choose one and to reject the other one. This is
why we used geometric aggregation and fuzzy weights. If maximal fuzzy score of solution A is
inferior to minimal fuzzy score of solution B, then we can be more confident on the choice of B and
the elimination of A. The decision-maker can decide at the beginning of the process of a minimal gap
MG between two solutions (Equation 3):
If E(RIMi1) > E(RIMi2)+MG, then the choice of RIMi1 is considered as reliable enough. (3)
21
If some solutions remain very close for instance for two RIM, a difference can be made with the
compatibility parameter described below.
• The choice of a combination of (RIM+RAM) instead of two independent choices
We introduce CR as the Compatibility Ratio between two RIM and RAM. We define the score of a
couple of RIM and RAM as following (Equation 4):
E(RIMj0, RAMj1)=E(RIMj0)*E(RAMj1)*CR(RIMj0, RAMj1) (4)
CR is equal to 1 when methods are independent or neutral, and is superior to 1 if methods do fit easily.
For instance, FMEA can be used both as a RIM or a RAM, then its CR is good. Brainstorming is a
standard RIM that does not have positive or negative influence on the use of RAM, so the CR is equal
to 1. On the contrary, the identification of risks thanks to cause trees is not adequate to the analysis of
the global risk exposure. These are different ways of thinking and different ways of using data, some
CR is inferior to 1. If the methods remain very close with the two first refinements, a third one is
possible by considering the change that the organization will have to do to implement each method.
• The organizational cost of implementation of a method can be a parameter of the final choice
We introduce IE as the Implementation Effort (see list of criteria) of a RIM/RAM in the company.
This ratio depends on both the method and the company. It is not a ratio which is independent of the
company. Then, we can apply a penalty/bonus to the initial score of a RIM/RAM by multiplying by
this index (Equation 5):
E’(RIMj0)=E(RIMj0)*IE(RIMj0) (5)
22
5. Industrial case study
An application of the choice method has been done on a company which delivers tramway
infrastructure for cities. The company was historically on product development and had recently
extended its scope by delivering to a city the product and its environment, that is to say the civil
engineering, the signalling material, the maintenance and storage depots, etc. As this type of project is
new for the company, the question of Project Risk Management method is pertinent, as risk
management for product development project is not the same for other areas.
The first action consisted in interviewing a person accountable for PRM method choice. He was a
decision-maker involved in several running projects, not a decision-maker from a project office. There
was in this case study no question about standardization of the method to the whole projects of this
type. The goal was only to test which RIM and RAM could best fit to these particular five projects.
The smallest project was about 5 years and 200 M€. This interview gave us information about
company maturity and company preferences, respectively CMj, RIwj, RImtj, RAwj and RAmtj. We
obtained the following results.
5.1. Step 1: first selection of RIM regarding the company maturity
Due to the team maturity level in risk management and due to the innovative level of the product, it
was difficult for the company to implement analytic methods (see Table 6).
Table 6: adequation of each type of PRM method to the company’s maturity
RIM which were only analytic were therefore screened out (RIM 16 and RIM 21).
23
5.2. Step 2 : selection and ranking of RIM/RAM regarding company preferences
The decision-maker expressed preferences in terms of weights with a linguistic scale [Negligible, Very
Low, Low, Average, High, Very high]. These linguistic variables were transformed in fuzzy numbers
with the intervals described in Figure 2. He expressed also minimum thresholds, the value 1 meaning
that there was no minimum threshold. The three criteria with a minimum threshold of 1 were not
significant, neither for screening out or for ranking as we used a geometric weighted product for
scoring methods. The other criteria did have an importance as they helped to reduce the number of
possible alternatives. The Table 7 below shows the expression of company’s preferences and methods
evaluation, and the scoring of the methods with their ranking.
This step enabled to screen out many methods. On the final ranking, only 6 RIM and 2 RAM were still
candidates.
25
5.3. Step 3 : the final choice of a combination of (RI+RA) methods
Three additional parameters may be included in order to refine the choice :
• The significance of the gaps in the ranking
In this case, 8 criteria were significant in the differentiation of methods, 2 for RIM and 6 for RAM.
This means that the minimal gap should be different for RIM and RAM. We obtain from the previous
table the fuzzy intervals for logarithmic scores. No obvious elimination can be done at this stage.
• The choice of a combination of (RIM+RAM) methods instead of two independent choices
The final combination in this case was a choice between local and global cause-effect analysis.
Namely, both methods were of the same category (tree-based or cause-effect methods). The difference
was on the scope of the analysis. For instance, root-cause analysis (RIM 17) is a deeper cause-effect
analysis looking for several cause-effect relationships. Chain reactions and depth analysis of causes
are the focus. In more local methods, like FMEA (RIM 6), the causes and effects are analyzed at only
one level, which means direct causes and direct effects.
As the decision-maker decided to proceed to deeper analysis, the choice of Ishikawa was made for
RIM (RIM 20 with experience) and the compatibility was good with RAM (RAM 5 with experience).
He decided to implement also RIM 17 on specific past problems, and to locally implement RAM 3
where there was some novelty on the current projects. Finally, he made a mix between global
analogical methods based on experience and local heuristic methods based on expertise, on some
points only. This is possible as the whole methods are based on the cause-effect modelling principle,
which makes them very compatible.
• The organizational cost of implementation of a method can be a parameter of the final choice
All the candidates are easy to understand and to use. Only FMEA could be judged as more difficult
than Ishikawa and root-cause analysis, but it conducts to the same choice. In this case, the
organizational cost of implementation does not change the final choice, it is not necessary to make a
trade-off.
26
5.4. Result of the case study
The growing complexity of projects involves a huge number of cause-effect relationships between
parameters, resources and events. These relationships can cause surprises like reaction chains or loops,
which are quite impossible to detect in the current situation. The current PRM method implemented in
the company is mainly based on independent analysis and treatment of risks. The decision-maker
noticed the gap between his preferences, which involve the choice of interactions-based methods, and
the current implemented method. He was truly confident that the proposed ranking corresponds to his
inputs and to the company’s needs (for this type of projects). Particularly, the fact that the current
method is eliminated in our approach has been a surprise for him, but it has been well accepted. He
was also surprised by the huge number of methods which were eliminated with his requirements of
minimum thresholds.
Even if the decision-maker agreed with the methodological recommendation, he predicted a tough
change from the current situation to the desired one. Namely, the methods in themselves are not
difficult to implement, but the team maturity in risk management is very low. This can involve
difficulties to understand the benefit of including cause-effect relationships between risks inside the
global process.
The decision-making process took 1 hour to introduce the approach and the criteria, and to assess
company’s maturity and preferences. Then, the scoring of the methods is done instantaneously and the
results can be analyzed immediately.
6. Conclusions
6.1. Synthesis of the approach
This study points out that there are plenty of methods for Risk Identification and Risk Analysis, and no
guidelines to help users and decision-makers to select the most suitable ones for their particular
projects. In this paper, we propose an approach to help choosing the right project risk management
method considering maturity level, innovation degree, effort needed to implement the method, etc. Of
27
course, this could help project manager or project office manager in their decision making process.
The proposal is a three-step decision-making process, where the first step screens out methods which
are too ambitious compared to the company’s maturity, the second one screens out and ranks
remaining methods according to company’s preferences, and the third one refines this choice thanks to
some additional parameters, like gap analysis, compatibility between methods and organizational cost.
As there are several screening phases, we used a geometric weighted product for global scoring of
alternatives. As some judgements are given in linguistic terms, fuzzy numbers are applied in this study
to determinate the weights.
6.2. Discussion and perspectives for future work
Some points can be discussed about the sensitivity and robustness of the final result:
• About the methods evaluation:
Different choices have been made, like threshold definition, discrete scales for evaluations and
classification of methods into three types. It is obvious that final results are sensitive to these inputs,
but :
o The thresholds have been assessed by interviews and literature review,
o Lots of methods are classified into more than one type, which reduces the risk to
screen out a method abusively if a type is eliminated,
o It is necessary to use qualitative scales as no obvious quantitative parameters do exist,
• About the Decision-Maker’s (DM) preferences and evaluation of the company:
As we wanted to test those criteria, we developed a simple Multi Criteria Decision Making (MCDM)
model that could be used to select the methods and apply it to an industrial case study. This is a first
step toward a more robust model, but at this stage, it raises some questions about DM: what is the
sensitivity of this model to DM unreliability? Who should use it and when? Should it be used at
corporate level (project office) or by the project manager before starting its project?
28
In the case where NDM DM give their opinion, then we have NDM expressions for each parameter. The
final parameter is obtained by combination of the individual ones. In the case of crisp numbers
(company maturity assessment or company preference thresholds), the global number is the minimum
of individual values. In the case of fuzzy numbers (company preference weights), it follows the
principles given previously in definition 7 of section 2.2.
A complementary work is also ongoing on the use of fuzzy AHP for weighting the preferences. This
will give a more precise evaluation of the weights, since criteria will be pair wise compared.
• About the completeness of the PRM lists:
The list of methods is quite representative of what exists in literature and what is really applied in
companies. We can be confident that no important method is forgotten. For the list of criteria, it is
different as it is our own creation. But the validity of this list has been tested by studying and
analysing the characteristics of the methods in the literature; this could become choice criteria. Are
there other relevant criteria to take into account when choosing PRM methods? Is there any need for
new methods or for a combination of existing methods? Would it be interesting to define a standard to
evaluate PRM methods and notably their conditions of application and validity domain?
6.3. Added value of our approach
Finally, we argue that this decision-making process has an added value for the Project Risk
Management process of the company, and then for the Project Management process. Namely, a more
suitable RIM/RAM will enable to reduce the impact of the risks and to reduce the probability for these
risks to occur. So, both the rate of success and performance level of projects could be improved. A
perspective of development could be to use this decision-making process as a functional requirement
definition of a good RIM/RAM, and then to develop a specific and more suitable method. This method
could be built by compilation of existing methods (the most frequent) or by specific development.
Finally, we think that, classifying the existing methods might help to identify a lack in some aspects of
these methods. For instance, we find very few project management methods that can handle properly
the interaction between risks. Finally, we think that this study will permit managers to be aware of all
29
available PRM methods and that it will lead them to consider the choice of the method as a strategic
decision that could impact the project success.
7. References
AFNOR (1997). ISO 10006. Management de la qualité: lignes directrices pour la qualité en management de projet, Editions AFNOR.
AFNOR (1999). RG AERO 000 40, dans NF X50 410. Recommandation générale pour la spécification du management de programme, Editions AFNOR.
AFNOR (2003). FD X 50-117: Management de projet, Gestion du risque, Management des risques d’un projet, AFNOR.
Amiri, M., M. Zandieh, et al. (2009). "A hybrid multi-criteria decision-making model for firms competence evaluation." Expert Systems with Applications: doi:10.1016/j.eswa.2009.04.045.
APM (1996). Project Risk Analysis & Management (PRAM) Guide. High Wycombe, ASSOCIATION FOR PROJECT MANAGEMENT.
Athanassopoulos, A. and V. Podinovski (1997). "Dominance and potential optimality in multiple criteria decision analysis with imprecise information." Journal of the Operational Research Society 48: 142-150.
Bellman, R. and L. Zadeh (1970). "Decision-making in a fuzzy environment." Management Science 17: 141-164.
Belton, V. and T. Stewart (2002). Multiple Criteria Decision Analysis: an integrated approach. Dordrecht, Kluwer.
BSI (2002). ISO/IEC Guide 73:2002. Risk Management – Vocabulary – Guidelines for use in standards. London, BRITISH STANDARD INSTITUTE.
Coppendale, J. (1995). "Manage risk in product and process development and avoid unpleasant surprises." Engineering Management Journal.
Hazen, G. (1986). "Partial information, dominance and potential optimality in multi-attribute utility theory." Operations Research 34: 296-310.
Hobbs, B. and P. Meier (2000). Energy Decisions and the Environment: a guide to the use of multicriteria methods. Massachusetts, KLUWER.
IEC (1995). CEI/IEC 300-3-9:1995 Risk Management: part 3 – guide to risk analysis of technological systems. Geneva, INTERNATIONAL ELECTROTECHNICAL COMMISSION.
IEEE (2001). IEEE Standard 1540-2001: standard for software life cycle processes – risk management. New York, INSTITUTE OF ELECTRICAL AND ELECTRONIC ENGINEERS.
IPMA (2006). IPMA Competence Baseline (ICB), Version 3.0, March 2006 International Project Management Association.
ISO (2003). ISO 10006 - Quality Management Systems - Guidelines for quality management in projects. Switzerland, International Organization for Standardization.
Kaufmann, A. and M. Gupta (1991). Introduction to fuzzy arithmetic: theory and application. New York, Van Nostrand Reinhold.
Klir, G. and B. Yuan (1995). Fuzzy sets and fuzzy logic: theory and applications. USA, Prentice Hall Inc. Kurtoglu, T. and I. Tumer (2007). A graph based framework for early assessment of functional failures
in complex systems. DETC 2007., Las Vegas, . Lyons, T. and M. Skitmore (2004). "Project risk management in the Queensland engineering
construction industry: a survey." International Journal of Project Management 22(1): 51- 61. Marle, F. and T. Gidel (2010). Analysis of existing Project Risk Management methods, Ecole Centrale
Paris / Université Technologique de Compiègne.
30
Pareto, V. (1971). Manual of Political Economy (English translation). New York, AM Kelley Publishers. PMI, S. C. (2004). A guide to the project management body of knowledge (PMBOK) (2004 ed.).
Newton Square, PA, USA. , Project Management Institute. Rios Insua, D. (1990). Sensitivity Analysis in Multiple Objective Decision Making. Berlin, Springer-
Verlag. Roy, B. (1996). Multicriteria methodology for decision aiding. Dordrecht, Kluwer. Shamsuzzaman, M., A. Sharif Ullah, et al. (2003). "Applying linguistic criteria in FMS selection: Fuzzy-
set AHP approach." Integrated Manufacturing Systems 14(3): 247-254. Triantaphyllou, E. and C. Lin (1996). "Development and evaluation of five fuzzy multiattribute
decision-making methods." International Journal of Approximate Reasoning 14: 281-310. Tumer, I., F. Barrientos, et al. (2005). Towards risk based design (RBD) of space exploration missions:
A review of RBD practice and research trends at NASA", , . ASME/DETC 2005, Long Beach, California.
Vincke, P. (1992). Multi-criteria decision-aid. New York, Wiley. Zadeh, L. (1965). "Fuzzy sets." Information and Control 8: 338-353. Zadeh, L. (1975). "The concept of a linguistic variable and its application to approximate reasoning."
Information Sciences 8(1): 199-249. Zimmermann, H. (1991). Fuzzy set theory - And its applications Springer.