RISK MANAGEMENT AND TACIT KNOWLEDGE IN IT PROJECTS:

MAKING THE IMPLICIT EXPLICIT

by

Hazel Ann Taylor BSc, MSc

A thesis submitted in fulfilment of the requirements for the degree of

Doctor of Philosophy

Centre for Information Technology Innovation

Queensland University of Technology

2003

Certificate of Completion

Key Words: Decision-making processes; IS implementation; IS project risk management; software packages; tacit knowledge.

RISK MANAGEMENT AND TACIT KNOWLEDGE IN IT PROJECTS: MAKING THE IMPLICIT EXPLICIT

Hazel Taylor

ABSTRACT

This research addressed the need for in-depth investigation of what actually happens in the practice of risk management in software package implementation projects. There is strong ‘official’ sanction in the IT literature for the use of formal risk management processes for IT projects but there is a confused picture of their application in practice. While many potential risk factors for IT projects have been identified, and formal procedures have been prescribed for the management of these risks, there has been little work investigating how project managers assess these risks in practice and what countermeasures they employ against these risks in their projects. In particular, the study used an interpretive critical decision interview approach to focus on those areas of risk management knowledge that project managers have acquired through experience, i.e. tacit knowledge.

A new categorization of risk factors emanating from three sources -- vendor, client,

and third party –reveals risk factors not previously identified. Some of these new factors arise from the three sources noted, while others arise from the package implementation focus of the projects and from aspects arising from the location of the projects in Hong Kong. Key factors that cause problems even when anticipated and mitigated, and the most often unanticipated problems are also identified.

The study further presents an examination of the studied managers’ risk

management practices, and the strategies they use to address both potential and actual problems. This examination revealed close conformance with recommended literature prescriptions at some stages of projects, and significant variation at other stages, with strategies applied being broad and general rather than risk specific. A useful categorization of these strategies into four broad groups relating to different sets of risk factors is presented, reflecting the actual practice of respondents.

Tacit knowledge was revealed throughout these investigations in the variances

observed between prescribed and actual practice, and particularly from an examination of project managers’ decision-making practices from two different perspectives - rational and naturalistic. A hybrid decision-making model is proposed to capture the actual processes observed, and to provide prescriptive guidance for risk management practice.

The investigation makes a contribution to the field of IT project risk management in three ways. First, the investigation has addressed the need for empirical studies into IT risk management practices and the factors influencing project managers in their choice and application of strategies to manage risk. Second, by examining how experienced IT project managers approach the task of managing risk in software package implementations, the study has extended our understanding of the nature of the knowledge and skills that effective IT project managers develop through experience. Third, the study makes a theoretical contribution to our understanding of IT project risk management by examining the decision-making processes followed by IT project managers from the perspective of two contrasting theories of decision-making – the rational method and the Naturalistic Decision Making theory.

TABLE OF CONTENTS

1 Introduction ............................................................................................................. 11

1.1. Background to the research ................................................................................11 1.2. Research aims ........................................................................................................13 1.3. Justification for the research...............................................................................14 1.4. Methodology..........................................................................................................15 1.5. Limitations and key assumptions.......................................................................16 1.6. Definition of key terms........................................................................................17 1.7. Outline of thesis structure...................................................................................19

2 Theoretical Foundations.........................................................................................20

2.1. Introduction...........................................................................................................20 2.2. IT software project risk management ...............................................................21

2.2.1. Project management.............................................................................................21 2.2.2. Project risk management .....................................................................................23 2.2.3. IT project risk management literature...............................................................24 2.2.4. Rational decision-making ....................................................................................29 2.2.5. Naturalistic decision making ..............................................................................31

2.3. Tacit knowledge ....................................................................................................34 2.3.1. Historical beginnings............................................................................................34 2.3.2. Implicit learning ....................................................................................................35 2.3.3. Declarative and procedural knowledge.............................................................37 2.3.4. Categories of knowledge .....................................................................................39 2.3.5. Tacit knowledge research in the management field .......................................45

2.4. The present study..................................................................................................46 2.5. Summary.................................................................................................................51

3 Methodology............................................................................................................52

3.1. Introduction...........................................................................................................52 3.2. Research approach................................................................................................53

3.2.1. Critical incident technique and critical decision method...............................55 3.3. Research procedures ............................................................................................57

3.3.1. Sample selection (organizations and subjects).................................................57 3.3.2. Respondent details................................................................................................60 3.3.3. Project details.........................................................................................................61 3.3.4. Instrumentation.....................................................................................................63 3.3.5. Data collection - interviews ................................................................................64 3.3.6. Refinement of interview protocol .....................................................................66 3.3.7. Reflections on interview process .......................................................................68

3.4. Analysis and coding processes............................................................................69 3.4.1. Research Question 1: Key risk factors..............................................................73 3.4.2. Research Questions 2 and 3: Risk management practices and strategies...76 3.4.3. Research Questions 4 and 5: Tacit knowledge and gaps between practice and theory................................................................................................................................77

3.5. Researcher’s perspective......................................................................................79 3.6. Ethical considerations..........................................................................................80 3.7. Summary.................................................................................................................80

4 Results and discussion of research question 1: Key risk factors............................82

4.1. Introduction...........................................................................................................82 4.2. Risk factors.............................................................................................................83 4.3. Comparison of risk factors with previous research........................................88

4.3.1. Risk factors in previous literature not mentioned in this study ...................95 4.3.2. New risk factors identified in this study ...........................................................96 4.3.3. Discussion of new risk factors identified in this study ............................... 106

4.4. Frequency with which risk factors were mentioned ................................... 108 4.4.1. Comparison of risk factor rankings with previous research...................... 109 4.4.2. Discussion of comparison of risk factor rankings....................................... 111

4.5. Anticipated and unanticipated problems, and ‘troubled’ projects ............ 112 4.5.1. Anticipated problems........................................................................................ 113 4.5.2. Unanticipated problems ................................................................................... 114 4.5.3. Problems causing projects to become ‘troubled’......................................... 116

4.6. Summary of discussion: Research Question 1 – Key risk factors ............ 117 5 Results and discussion of research questions 2 and 3: risk management practices

and strategies.............................................................................................................119

5.1. Introduction........................................................................................................ 119 5.2. Research Question 2: Risk management practices ...................................... 119

5.2.1. Pre-project risk management practices.......................................................... 120 5.2.2. Risk management practices at project start-up stage .................................. 122 5.2.3. Risk management practices during the course of the project.................... 123 5.2.4. Risk management practices after project completion................................. 123 5.2.5. Comparison of risk management practices with prescribed processes ... 124

5.3. Research Question 3: Risk management strategies ..................................... 128 5.3.1. Strategies to manage risks emanating from pre-sales practices................. 129 5.3.2. General categorization of risk management strategies ............................... 132 5.3.3. Control strategies ................................................................................................. 135 5.3.4. Negotiation strategies ........................................................................................... 140 5.3.5. Research strategies................................................................................................ 149 5.3.6. Monitoring strategies............................................................................................ 150 5.3.7. Comparison of risk strategies with prescribed processes........................... 151

5.4. Summary of discussion: Research Questions 2 and 3................................. 154 6 Results and discussion of research questions 4 and 5: tacit knowledge and gaps

between practice and theory .................................................................................... 156

6.1. Introduction........................................................................................................ 156 6.2. Research Question 4: Project managers’ explicit and tacit knowledge.... 156

6.2.1. Explicit knowledge applied to project risk management............................. 157 6.2.2. Tacit knowledge applied to project risk management ................................. 158 6.2.3. Environmental and situational cues ............................................................... 161

6.3. Research Question 5: Gaps between risk management theory and practice, and decision-making processes .............................................................................................. 163

6.3.1. Decision-making processes in initial assessment situations of routine projects ............................................................................................................................... 165 6.3.2. Decision-making processes in initial assessment situations of ‘troubled’ projects ............................................................................................................................... 169

6.3.3. Decision-making processes in problem-arising situations of routine and ‘troubled’ projects ............................................................................................................... 173

6.4. Summary of discussion: Research Questions 4 and 5................................. 178 7 Conclusions and implications............................................................................... 180

7.1. Introduction........................................................................................................ 180 7.2. Conclusions about each research question ................................................... 183

7.2.1. Research Question 1 ......................................................................................... 183 7.2.2. Research Question 2 ......................................................................................... 187 7.2.3. Research Question 3 ......................................................................................... 188 7.2.4. Research Question 4 ......................................................................................... 191 7.2.5. Research Question 5 ......................................................................................... 193

7.3. Implications for theory ..................................................................................... 195 7.4. Implications for practice................................................................................... 196 7.5. Limitations .......................................................................................................... 197 7.6. Further research ................................................................................................. 198 7.7. Self Assessment of Research Quality ............................................................. 200

7.7.1. Application of sound and rigorous qualitative research methods ............ 200 7.7.2. Recognized principles for conducting interpretive research ..................... 202

7.8. Summary.............................................................................................................. 205 8 References.............................................................................................................. 207

9 Appendix A: Initial Interview Protocol................................................................. 215

10 Appendix B: Ethics consent package ................................................................. 219

11 Appendix C: Practitioner Report ......................................................................... 222

LIST OF TABLES AND FIGURES

Fig. 2.1: Project risk management processes........................................................................................23 Fig. 2.2: Recognition-primed decision model (after Klein et al., 1989)...........................................33 Table 2.1: Categories and sub-sets of knowledge................................................................................43 Table 3.1: Organization details ...............................................................................................................60 Table 3.2: Project management perspectives .......................................................................................60 Table 3.3: Types of projects ....................................................................................................................62 Fig. 3.1: NVivo coding structure ............................................................................................................70 Table 4.1: Summarized risk factors ........................................................................................................83 Table 4.2: Number of respondents (out of 25) identifying risk factors ..........................................84 Table 4.3: Comparison of Risk Factors Identified in Present and Previous Studies....................89 Table 4.4: New risk factors identified in the present study ............................................................ 106 Table 4.5: Top 17 Risks (mentioned by at least ten respondents) ................................................ 109 Table 4.6: Comparison of risk rankings: Schmidt/Keil studies and present study.................... 110 Table 4.7: Problems anticipated, and number still arising .............................................................. 113 Table 4.8: Unanticipated problems arising during the project....................................................... 114 Table 4.9: Problems responsible for ‘derailing’ troubled projects................................................. 116 Table 5.1: Problem situations............................................................................................................... 129 Table 5.2: Strategies and risks they addressed................................................................................... 133 Table 5.3: Strategies used in initial assessment and problem arising situations .......................... 134 Table 6.1: Examples of tacit knowledge associated with risk management strategies .............. 160 Table 6.2: Strategies and stages in routine and ‘troubled’ projects................................................ 165 Fig. 6.1: Partial rational risk management processes during initial assessment of routine

projects....................................................................................................................................... 166 Fig. 6.2: Descriptive RPD model of processes at start of routine projects ................................. 168 Fig. 6.3: Rational risk management processes during initial assessment of ‘troubled’

projects....................................................................................................................................... 171 Fig. 6.4: RPD model incorporating rational and naturalistic situation assessment during

initial assessment of ‘troubled’ projects ............................................................................... 172 Fig. 6.5: RPD model for problem arising situations in all projects............................................... 177 Table A.1: Critical decision probe questions (adapted from Klein et al., 1989) ......................... 217

STATEMENT OF ORIGINAL AUTHORSHIP

The work contained in this thesis has not been previously submitted for a degree or

diploma at any other higher education institution. To the best of my knowledge and belief,

the thesis contains no material previously published or written by another person except

where due reference is made.

Signed:

Date:

ACKNOWLEDGEMENTS

Thank you to my husband, Paul, for his unfailing support, encouragement and

advice.

11

C h a p t e r 1

1INTRODUCTION

The research reported in this thesis was motivated by reports in the literature (for

example, Boehm, 1991; Drummond, 1996; Keil & Carmel, 1995; McFarlan, 1981) that

continuing problems with IT projects can be attributed, at least in part, to failure to apply

rational risk management techniques that have for some time now been considered to be

‘best practice’ in project management. The study set out to explore the practice of risk

management in IT projects, with a focus on vendor-driven software package

implementation projects. This exploratory research used a critical decision method to

surface tacit knowledge held by IT project managers regarding risk management in their

projects, and examined the findings from the perspectives of two contrasting theories of

decision-making – rational and naturalistic. The results reveal new risk factors and

pragmatic strategies for managing risk that have not been identified in the literature. In

addition, several gaps between practice and theory are highlighted. In these areas the

predominant theoretical approach prescribed in the literature does not adequately describe

the more complex realities of practice in these uncertain and changing project

environments.

This introduction outlines the rationale for the thesis and the nature of the problems

it addresses. The chapter is organized into six sections: 1.1) background to the research;

1.2) the research questions; 1.3) justification for the research; 1.4) outline of methodology;

1.5) limitations and key assumptions; 1.6) outline of subsequent chapters.

1.1. Background to the research

Reports about problems with IT projects have appeared regularly in the popular and

academic literature for over 30 years, including several well-publicized major failures

(Drummond, 1996; Lyytinen, Mathiassen, & Ropponen, 1998). Risk management practice

has been identified as one critical factor of the success of IT development projects

(Schmidt, Lyytinen, Keil, & Cule, 2001), and a significant stream of research has focused

on identifying risk factors and prescribing risk management frameworks to aid managers

in making decisions about risk factors in IT software development projects (see, for

example: Alter, 1996; Barki, Rivard, & Talbot, 1993, 2001; Boehm, 1991; Keil, Cule,

12

Lyytinen, & Schmidt, 1998). However, reports of troubled IT projects continue, and while

there is a presumption that this is at least partially due to the risk management techniques

recommended in the literature not being applied in practice (Keil et al., 1998), there is little

research on whether this is indeed the case.

While many potential risk factors for IT projects have been identified, and formal

procedures have been prescribed for the management of these risks, little is known about

how project managers assess each of these risks in practice and what countermeasures

they employ against these risks in their projects. In particular, we know little about the

knowledge and skills being applied by more experienced project managers, nor of the

extent to which these knowledge and skills derive from formal education or training.

Software packages are especially interesting in the context of IT risk management

practice, in that their use is purported to ameliorate or avoid many of the risks associated

with custom developments (Lassila & Brancheau, 1999; Martin & McClure, 1983). With

the trend away from custom system development towards generic software packages

(Russo, 2000), interest in specific risk issues related to software package implementation

projects is increasing (Parr, Shanks, & Darke, 1999; Sumner, 2000). And while it appears

that these projects have some unique risks (Sumner, 2000), again there is little research

addressing the particular practices of managers of these projects (Davis, 1998; Gable,

1998).

One common theme in the risk management literature referred to above is that

projects with higher levels of uncertainty and complexity are more risky. Research has

investigated how experienced professionals make decisions in poorly-defined and

uncertain circumstances in a variety of contexts, including fire-fighting, nursing, the armed

forces, and hardware and software design (Klein, 1999; Vicente, 1999). This stream of

research has led to the development of a descriptive theory - Naturalistic Decision

Making (Zsambok, 1997), and a model of decision-making - the Recognition-Primed

Decision model (Klein, 1993; Klein, Calderwood, & MacGregor, 1989), which propose

that situation assessment, tacit knowledge (i.e. knowledge gained through experience), and

judgment are important factors in decision-making in real-life settings, and that the rational

processes assumed by academic prescriptive models and frameworks for decision-making

(see, for example, Barki et al., 2001; Boehm, 1991; Charette, 1996b; Fairley, 1994;

Heemstra & Kusters, 1996; Keil, 1995; Powell & Klein, 1996) are often not applied in

practice. Moreover, studies of skilled practitioners in areas such as business management,

13

sales, and military leadership have shown that, when dealing with complex and ill-defined

situations, these experts supplement their formal education with tacit knowledge and

judgments derived from their experience (Sternberg & Horvath, 1999). Rational decision

processes are a basic assumption of the risk management frameworks referred to earlier,

and are typically learned through a formal education process. However, in practice,

experienced project managers of complex and uncertain software projects may rely more

on judgment and tacit knowledge than on the prescribed rational frameworks.

The thesis reports on an exploratory and descriptive field study of current practice in

risk management of software package implementation projects. In particular, the aim was

to gain an understanding of the dynamics of the risk management process and the actions

taken with respect to possible risks for a given project in its situational context. This study

of current practice provides a useful window into inherent work constraints that limit the

applicability of the prescriptive models found in the literature. Further, the study: (1)

highlights aspects of current practice that may undermine the effectiveness of risk

management practices; (2) lays the foundation for a formative model encompassing

aspects of both rational and naturalistic decision-making that can provide better

descriptions of actual practice; and (3) offers more practical prescriptive guidance in order

to support practitioners effectively.

1.2. Research aims

The aims of the research were three-fold. The study explored, firstly, the key risks

that experienced project managers attended to in software package implementation

projects, and how they managed these risks in practice. Secondly, the study identified

knowledge (tacit or explicit) that experienced project managers used in order to plan for

and address critical risk situations that arose during the course of their projects. Thirdly,

risk management practices were compared with generally accepted prescriptions in the

literature to identify gaps between theory and practice. These gaps were examined from

the perspective of both rational and naturalistic models of decision making in order to

investigate how well each of these theories described and explained practices described by

respondents. Specifically, the thesis focused on vendor project managers working

primarily with software package implementation projects, and investigated the following

inter-related research questions:

14

1) What key risks do project managers attend to in software package implementation

projects, and how do these compare with the risk factors identified in the literature for

software packages as well as for custom development?

2) What risk management practices do project managers employ when planning software

package implementation projects, and how do these compare with prescriptions in the

literature?

3) What strategies do project managers use to address risks in IT projects – both to

prevent problems in the first place and to deal with the problems if they do arise (i.e. if

the risks identified as potential problems turn into actual problems)?

4) What knowledge (tacit or explicit) do experienced project managers use in order to

plan for and address critical risk situations that may arise during the course of software

package implementation, and in particular, what environmental and situational cues do

managers attend to when managing their projects?

5) What gaps are there between actual practice and the rational risk management

processes as recommended in the IT risk management literature, and to what extent

can decision making theories such as Naturalistic Decision Making explain any such

gaps?

The underlying theoretical foundations for these research questions include project

risk management, tacit knowledge and decision-making. The literature related to these

underpinning areas is discussed in Chapter 2, and a context and motivation for the present

research is established in that chapter. A brief summary of the rationale for the current

study is given in the next section.

1.3. Justification for the research

With the growing demand for high quality information systems in organizations, and

with the increasing use of commercial packages rather than custom-developed software

(Russo, 2000), strategies for managing risk during the implementation of these packages

have become increasingly important. The exploration of how experienced project

managers approach the task of managing risk in the implementation of software packages

on client sites provides insights for understanding when and why recommended risk

management processes are not always being applied in practice. Improved understanding

of the knowledge, both explicit and tacit, that successful IT project managers draw on in

15

determining strategies to address risk in implementation projects can aid researchers and

practitioners in developing better implementation project risk management techniques,

and can also be used to develop specific training programs for novice project managers.

The investigation reported here makes a contribution to the field of IT project

management in three ways. Firstly, the investigation addresses the need for more empirical

studies into IT risk management practices (Schmidt et al., 2001) and the factors

influencing project managers in their choice and application of strategies to manage risk.

Secondly, by examining how experienced IT project managers approached the task of

managing risk in software package implementations, the study extends our understanding

of the nature of the knowledge and skills that effective IT project managers develop

through experience. Thirdly, this research contributes to a better understanding of

discrepancies between theoretical prescriptions and practical applications of risk

management in package implementation projects, by examining the processes and factors

identified in the light of both rational and naturalistic decision-making theories.

The next section outlines the methodological approach chosen for the research.

1.4. Methodology

The exploratory and descriptive nature of the research required research methods

that met the need for contextual information, as the contexts of the situations examined

were of paramount importance in gaining an understanding of the dynamics of the risk

management process and the decisions made with respect to possible risks for a given

project. Investigations of this type lend themselves to ‘broadly interpretive methods of

research’ (Walsham, 1995). Thus, a primarily qualitative analysis approach was used. The

study used a critical decision interview approach (DuBois, 2002; Klein et al., 1989) to

explore and describe the risk management approaches used by experienced IT project

managers during software package implementation projects. This semi-structured

interview method enabled the elicitation of critical risk situations illustrating key

constraints and environmental and situational cues that could be involved in project

managers’ risk assessment and risk management of projects. Interview transcripts were

analyzed with a qualitative content analysis procedure, using the software package NVivo

version 2.0, to support and manage the detailed coding and analysis process. The risk

management decisions made by IT project managers during their projects were examined

from two perspectives using frameworks drawn from standard risk management guides

16

typically recommended to practitioners (PMI Standards Committee, 1996), and from

naturalistic decision making theory and the Recognition-Primed Decision model

developed by Klein, Calderwood, and MacGregor (1989).

1.5. Limitations and key assumptions

The purpose of this research was to learn more about risks and risk management in

software package implementation projects, and the knowledge, both tacit and explicit, that

project managers draw on when managing these project risks. The methodology relied on

a variation of the critical incident interview approach to elicit information about project

managers’ approaches to risk management, and as such was reliant on their self-reports

and recollections of their plans and actions and the circumstances surrounding these. An

assumption of this approach is that people can and will provide information that draws

not only on their explicit knowledge about what ought to be done, but also on their tacit

knowledge, gained from experience, about what actually works. While it is acknowledged

that one aspect of tacit knowledge is that it is difficult to articulate, previous use of the

critical incident technique (Sternberg et al., 2000) and the critical decision method (Klein

et al., 1989) for eliciting knowledge has demonstrated that these approaches can describe

the function served by tacit knowledge in proficient task performance.

All of the project managers interviewed for this research were based in Hong Kong,

and there is a possibility that the findings were country/culture specific. To some extent

this limitation is mitigated by the international nature of the respondents, all of whom had

studied or worked abroad during some stage in their careers, and the international nature

of the software packages the managers were working with, since, for the most part, these

packages are marketed globally. While some of the risks identified by these managers were

clearly related to their Hong Kong location, many of the risks correspond to those

reported in the literature previously, and the location-specific risks also have implications

for managers in other countries, particularly in view of the increasing trend towards

multinational projects that cross traditional country/culture boundaries.

Finally, the study focused solely on packaged software implementation, and the

exploratory nature of the research and use of in-depth interviews placed practical

limitations on the sample size. However, the in-depth analysis of information across a

range of informants and organizations in the packaged software field has enhanced our

knowledge of risk management practices within this range of empirical circumstances, and

17

hence can increase our confidence in applying this knowledge to new settings sharing

similar circumstances to those investigated in this research (Baskerville & Lee, 1999). In

particular, while this research has focused mainly on software package implementation

projects, these projects share many features in common with custom development

projects and a future research opportunity is to explore the extent to which the findings

from the present study are applicable in other IT project settings.

1.6. Definition of key terms

There is some variation in the literature in the definition of certain key terms. While

detailed discussion of these key terms and differences in definition can be found in

chapters 2 and 3 of the thesis, a brief glossary of key definitions that I have applied in this

study is given to aid the reader.

Critical Decision Method: A variation of the critical incident technique that has been

developed specifically for knowledge elicitation applications, where the aim is to reveal

aspects of expertise such as the critical cues used in making perceptual and conceptual

discriminations, and the underlying basis for judgment decisions (Klein et al., 1989).

Critical Incident Technique: A method for obtaining specific, behaviourally focused

descriptions of job performance developed by Flanagan (1954). The critical incident

technique has been applied to the elicitation of tacit knowledge related to job performance

by a group of researchers working to identify tacit knowledge in applied settings

(Sternberg et al., 2000; Sternberg & Horvath, 1999; Sternberg & Wagner, 1986).

Explicit Knowledge: Knowledge that is formally learned and easily articulated. Explicit

knowledge is held in repositories such as libraries, books, formal data media, written rules

and procedures and is acquired through formal learning procedures such as schooling,

reading, and formal training.

Implicit Knowledge: (Also referred to as tacit knowledge). The broad category of

knowledge that individuals find difficult to articulate and have learned by experience, by

practice (‘doing’) or by ‘osmosis’. Researchers have used different degrees of granularity,

and different terms, in discussing types of implicit knowledge. See Section 2.3.4 for a

detailed discussion.

18

Project Executive: A senior manager, referred to as a programme manager in some

firms, who has responsibility for supervising multiple project managers within the vendor

firm and overseeing the projects they are working on. Project executives typically take a

strategic and long-term management perspective within their firms, act as executive

sponsor or steering committee head for the projects under their oversight, are responsible

for balancing the competing resource needs of their projects within their firms, and deal

with high-level contract negotiations with client firms.

Project Manager: The manager who is assigned overall responsibility for a specific

project. In vendor-driven projects, typically there will be a vendor project manager with

responsibility for the project from the vendor perspective, and a client project manager

with responsibility for the project from the client perspective. Vendor project managers

were the focus of the present research. Vendor project managers manage the daily

activities of the project, co-ordinate tasks, manage the project team, liaise with their

counterparts in the client firm, and are responsible for ensuring the project schedule,

budget, and requisite quality are achieved.

Risk: Potential problems or issues that may arise and adversely impact on the

progress or outcome of a project. Note that while typical definitions of risk with respect

to project management relate to potential problems that could impact on the project itself,

the vendor respondents in this study also consider a risk to be any potential problem or

situation arising from involvement in the project that could adversely impact on their own

firm See Section 4.3.2, Environment factors for more discussion.

Risk Management Practice: An organization’s policies and procedures regarding the

management of risk, in particular the planning processes prescribed to evaluate and

address risk at the start of a project, and prescribed procedures for managing risk during

the project.

Risk Management Strategy: Specific strategies applied by managers to address particular

risks during the course of the project. These strategies may be prescribed as part of the

organization’s standard risk management practice, or they may be tactics that project

managers have developed through their own experience.

Tacit Knowledge: see Implicit Knowledge

19

1.7. Outline of thesis structure

In this introductory chapter I have set the context and foundations for the thesis,

and introduced the research aims and questions. I have presented a justification for the

research, briefly overviewed the methodology, and discussed key assumptions and

limitations. The detailed description of the research is contained in subsequent chapters as

set out in the outline of the thesis structure below.

The thesis is structured into eight chapters, including this introduction. In Chapter 2

I discuss the theoretical foundations underlying the research questions investigated in this

thesis. In particular, I review the literature on project risk management, with a particular

emphasis on risk management research in the IT field. I discuss the assumption

underlying this project risk management literature of a rational decision making approach,

and consider an alternative theory of decision-making, Naturalistic Decision Making, that

incorporates a focus on knowledge gained from experience – tacit knowledge. I then

examine research in the cognitive psychology field on tacit knowledge, and explore how

tacit knowledge has been investigated in related fields such as business and management. I

show how researchers working from very different starting points – decision-making on

the one hand, and tacit knowledge on the other hand – have converged in developing a

practical method for examining the question of knowledge gained from experience. This

method forms the basis of the research approach of this thesis. Finally, I provide a basis

for the aims of the present research and discuss in detail the research questions noted in

Section 1.3 above.

In Chapter 3 I discuss the chosen research approach, the critical decision interview

method, and show why it is an appropriate method for the investigation of the research

questions. I then detail the research procedures followed, and explain the analysis

approach. Demographics and simple descriptive results are reported in Chapter 4, while

Chapters 5, 6, and 7 contain results and discussion for the five research questions

discussed above. I present conclusions and implications from the research in Chapter 8.

20

C h a p t e r 2

2THEORETICAL FOUNDATIONS

2.1. Introduction

In this chapter I discuss the theoretical foundations underlying the research

questions that were outlined in Chapter 1. Section 2.2 contains a review of the literature

on project risk management, with a particular emphasis on risk management research in

the IT field. Section 2.2.1 overviews project management in general, and Section 2.2.2

focuses on project risk management. In Section 2.2.3, I review the IT project risk

management literature. In Section 2.2.4, I discuss the underlying assumption of a rational

decision making approach to risk management that pervades the risk management

literature. In Section 2.2.5 I discuss an alternative approach to decision making,

Naturalistic Decision Making (NDM), based on empirical studies of decision makers in

real-life situations. NDM provides a descriptive model of decision-making, the

Recognition-Primed Decision model (RPD), which will be explored for its potential to

shed light on the research questions of this thesis.

NDM introduces the concept of knowledge based on judgment and experience –

tacit knowledge, and in Section 2.3 I discuss tacit knowledge, and how tacit knowledge has

been investigated in related fields such as business and management. Section 2.3.1

overviews the historical beginnings of the study of tacit knowledge, and in 2.3.2 I discuss

the related concept of implicit learning. In Section 2.3.3 I review Anderson’s (1982) skill

acquisition framework of declarative and procedural knowledge, and in Section 2.3.4 I

draw together various categories of knowledge that have been proposed by researchers in

the business and management arena. In Section 2.3.5 I focus on empirical tacit knowledge

research in the management field conducted by the group of researchers working with

Sternberg and Wagner (Sternberg et al., 2000; Sternberg & Horvath, 1999; Sternberg &

Wagner, 1986; Wagner, 1987; Wagner, Sujan, Sujan, Rashotte, & Sternberg, 1999). These

researchers have used a critical incident approach to examining tacit knowledge that is

very similar to the critical decision method developed by NDM researchers (Klein et al.,

1989). These two methods form the basis of the chosen research approach discussed in

Chapter 3.

21

Finally, in Section 2.4, I show how the literature reviewed in the previous sections

forms a basis and context for the aims of the present research and discuss in detail the

research questions introduced in Chapter 1. Section 2.5 provides a brief summary of the

chapter.

2.2. IT software project risk management

Reports of problems with IT projects – abandoned projects, cost and schedule over-

runs, failure to deliver required functionality – have appeared regularly in the popular and

academic literature for over 30 years (Boehm, 1991; Drummond, 1996; Keil, 1995;

McFarlan, 1981). During this period there has also been a steady flow of advice for IT

project managers in both the academic and practitioner literature on IT project

management, development methodologies, and risk management techniques. IT software

project management includes project risk management and is a subset of project

management in general. In the following sections I first overview project management in

general, with illustrations from IT projects (Section 2.2.1), and I then discuss the generally

accepted approaches to project risk management (Section 2.2.2). Following this, I review

the two major strands, prescriptive and descriptive, of IT project risk management

literature, and researchers’ recommendations for successful management of IT projects

(Section 2.2.3). Finally, I discuss the possible implications of the premise of rational

decision-making that underlies accepted general and IT project management techniques

(Section 2.2.4) and (Section 2.2.5) review an alternative research approach to decision

making - naturalistic decision making, showing how this approach is relevant for the

concerns of this thesis.

2.2.1. Project management

The body of knowledge on project management and project risk management is

extensive and well established (e.g. PMI Standards Committee, 1996), and provides a basis

for understanding project management techniques that are recommended for the IT

industry by researchers such as, for example, Boehm (1991), Heemstra and Kusters

(1996), and Powell and Klein (1996). Generally accepted definitions of projects, project

life cycle, project management, and project risk management are summarized next.

Projects are distinguished from operations in that projects are temporary and unique

activities, with every project having a distinct beginning and end, producing a unique

22

product or service. Operations, in contrast, are on-going and repetitive activities, with no

distinct end. While projects may have a repetitive aspect in that a new project may seek to

produce a similar outcome to previous projects – e.g. a new building or a new software

application – the specific outcome for each new project is unique in that its particular

scope, context and requirements will be different from previous similar projects. Projects

typically have specific goals, aim to achieve some sort of change, are constrained by

limited resources, and require co-ordination in order to meet the goals.

All projects go through a project life cycle, comprising several phases including

initiation, planning, and implementation, with some further breakdown within each phase.

For IT projects, the systems development methodology chosen for a software

development project is an instance of a particular approach to the project life cycle. The

project life cycle gives a higher-level view for overall control and management of the

project, while the choice of systems development methodology determines how the

necessary technical tasks are carried out (McLeod & Smith, 1996 pg 73). The initiation

phase of the project life cycle typically corresponds to initiation and feasibility stages of

traditional systems development methodologies. The planning phase of the project life

cycle occurs if management approves going ahead with the project, and would typically

include deciding upon the specific methodological approach to take for the systems

development (for example, waterfall approach, prototyping, rapid application

development). The implementation phase of the project life cycle then utilizes the chosen

methodology to complete the project.

Project management involves the application of general and specific management

knowledge and skills to project activities in order to meet the stated goals of the key

stakeholders in the project. Such goals typically include scope, cost, time and quality

objectives. IT projects are sometimes claimed to be less amenable to traditional project

management techniques because of the high levels of uncertainty in terms of knowledge

about, for example, specific requirements, how long a particular application will take to

program, and issues relating to the acceptance of the new application by the organization

(McLeod & Smith, 1996 pg 7). However, most IT project management researchers argue

that more rigorous application of project management techniques, and in particular, risk

management techniques, is the best approach to dealing with the uncertainties that are

typical of IT projects (see, for example, Barki et al., 2001; Boehm, 1991; Keil et al., 1998).

Project risk management is one of the project management knowledge areas applied over the

23

whole project life cycle, with risk assessment being part of the planning phase, and risk

monitoring continuing throughout the implementation phase.

2.2.2. Project risk management

Project risk management includes four major processes, illustrated in Figure 2.1 and

discussed below (PMI Standards Committee, 1996).

Fig. 2.1: Project risk management processes

The risk identification process involves identifying both the sources of potential risk

to the progress or outcome of the project, and the risk symptoms, sometimes called

triggers or cues, that can provide early warning of an actual risk event. Risks are typically

identified using some sort of checklist, specific to the application area of the project. Risk

interviews of key personnel can also help to identify atypical project risks not included on

the checklists.

Once the potential risks have been identified they must be assessed in terms of the

likelihood of their occurrence, and the potential impact if they do occur. Typical

assessment techniques include quantitative analysis of probabilities of occurrence, and

cost of impact if the risk event does occur, in order to give an expected monetary value

(probability times cost) for each potential risk. Tools such as simulations and decision

trees are used to aid this analysis when probability and cost figures are available, but expert

judgment is also used if the probabilities or costs are difficult to estimate because of

uncertainties about the potential risk. When expert judgment is used, likelihood of risk

and impact are usually estimated on a low-medium-high scale. The expert judgment

approach is frequently applied in IS projects because of the typically high levels of

uncertainty in these projects (McFarlan, 1981; McLeod & Smith, 1996). The analysis of

the risks results in a prioritized list of risks, and also an overall assessment of the level of

‘riskiness’ of the whole project. In IT projects, the level of riskiness of a project is often

considered as a contingent factor in determining the management approach to take for the

project. For example, Davis (1982), McFarlan (1981) and Barki et al. (2001) all

RISK IDENTIFICATION

RISK RESPONSE PLANNING − Risk Elimination − Risk Mitigation − Risk Acceptance − Contingent Action Planning

RISK MONITORING − Progress Feedback − Progress Analysis − Corrective Action

RISK ASSESSMENT − Risk Analysis − Risk Prioritization

24

recommend a contingency approach to IT project management depending on the degree

of uncertainty (i.e. riskiness) about the project.

Some potential risks can also represent potential opportunities depending on the

outcome of managing the risk (PMI Standards Committee, 1996), and hence a thorough

risk assessment phase also includes consideration of potential opportunities and how to

take advantage of these. For example, in IT projects, working with a new programming

language could be considered a potential risk to the project’s success because of the team’s

inexperience with the language, but it also can open up new future opportunities if the

project team can master the skills required with the new language. However, typically in

the IT risk literature, the focus is solely on the negative aspects of risk, with little or no

consideration of the broader range of possible outcomes (Lyytinen et al., 1998).

The list of risks identified, prioritized as discussed above, is the input for the risk

response-planning phase, during which plans are developed to address the various risks

(and take advantage of potential opportunities). Each risk is considered, to determine

what response is required. Preventive steps are taken to eliminate or mitigate the impact

of high priority risks, while lower priority risks may simply be regarded as acceptable and

no further action is taken. Contingent actions are planned to occur in the event that a

particular risk event actually happens in spite of the preventive steps. Finally, in the risk

monitoring stage, the project is checked for risk symptoms, feedback on progress is

analyzed and compared with the original plan, and corrective measures are taken as

necessary in accordance with the pre-planned contingent actions.

Given the many problems that continue to recur in the IT project arena, the

question of how project risk management is handled in the IT field is clearly of interest

and is the aspect of IT project management focused on in this thesis. I next review the

recommendations in the IT literature for successful project risk management.

2.2.3. IT project risk management literature

The IT project risk literature falls into two major strands, the descriptive strand,

which has focused on the first stage of risk management, identifying risk factors in

software development projects; and the prescriptive strand, which gives advice on how to

carry out risk management, and in some cases includes contingency frameworks for

deciding when to apply certain risk management approaches.

25

Descriptive research on IT risk factors. The descriptive risk factors strand of

research has focused on developing complete and comprehensive check-lists of risk

factors to be considered when planning and managing an IT project. There is now a

substantial body of work on the typical risk factors faced by software project managers,

and also the priorities placed on these risk factors by managers (Alter & Ginzberg, 1978;

Barki et al., 1993; Boehm, 1991; Heemstra & Kusters, 1996; Schmidt et al., 2001; Sumner,

2000). The risk check-lists vary in detail and emphasis, but the risks identified all generally

fall within Sumner’s (2000) eight categories of i) organizational fit, ii) skill mix, iii)

management structure and strategy, iv) software systems design, v) user involvement and

training, vi) technology planning, vii) project management, and viii) social commitment.

While the risk check-lists developed by researchers are extensive and based on surveys and

interviews with experienced and successful IT project managers, these checklists report

what the managers say they attend to when considering IT projects, not how they actually

use such check-lists. Perhaps the lists tell us more about managers’ views of what went

wrong with their projects, than about what they actually do to try and ensure a project

stays on track. We know very little about what managers actually do and how they assess

and manage the risks of a given project.

Whether project managers actually use risk checklists or not, they would seem a

useful aid for assessing potential project risks. Yet, in practice, they are not always so

practical. As Powell and Klein (1996) note, there may be a tendency to assume the

checklist is complete, and therefore to overlook possible risks specific to a given project,

but not included in the checklist. This is especially of concern since the reported checklists

vary considerably in the risk factors on which they focus. For example, many studies cite

top management support or the existence of a project sponsor as a crucial factor of

success (Heemstra & Kusters, 1996; Moynihan, 1997; Parr & Shanks, 2000; Schmidt et al.,

2001; Sumner, 2000). These researchers, and Barki et al.’s (1993) initial literature survey of

risk factors, all included top management support as a risk variable. However, this variable

was dropped from the final measure of software development risk offered by Barki et al.

as a tool for organizations wishing to undertake risk assessment of projects, because it did

not load on any of the five factors they identified in their factor analysis. Similarly,

Boehm’s (1991) often cited top ten risk items does not include the lack of top

management support or a project sponsor as a risk factor. Project managers who base

their risk assessment on McFarlan’s (1981) approach will find that he provides a primarily

task and technology focused view, while more recent checklists place less emphasis on

26

technology and more on organizational, environmental and individual characteristics

(Schmidt et al., 2001). Further, identifying risks on a checklist is not a substitute for

actually taking action on the risks (Powell & Klein, 1996). Schmidt et al. note “we still lack

a basic understanding of why certain managerial actions serve to mitigate multiple risks,

and under what conditions these actions are no longer effective.”

There is also little known about the extent to which the risk taxonomies and

checklists discussed above are applicable in all software development situations and

whether different taxonomies are needed for different project contexts (Moynihan, 1997).

For example, Heemstra and Kusters (1996), drawing on evidence from five action

research projects, note that the generic 36 risk factor checklist they developed was not

suitable for one of the projects, which involved a package implementation and required

relatively more emphasis on human aspects and less on technical development risks. And

after analyzing seven case studies of firms implementing enterprise resource planning

(ERP) systems, Sumner (2000) concluded that nine of the twenty detailed risk factors

identified were unique to ERP systems and thus had not been adequately covered in

previous checklists. These particularly relate to, firstly, the client’s commitment to change

business processes and data definitions to match the ERP package; secondly, the ERP

firm’s ability to obtain and retain personnel skilled in both the specific ERP technology

and the client business or alternatively to work effectively with outside consultants

providing these skills; and thirdly, the adequacy of training given to both the ERP firm’s

staff and the client firm’s staff. These risk factors correspond to Parr and Shanks (2000)

critical success factors of using ‘vanilla’ ERP (i.e. change the client processes, not the ERP

package), and having a balanced team (i.e. a mix of internal/external and

business/technical expertise). Parr and Shanks do not mention training. Both Sumner and

Parr and Shanks also identify full-time commitment from key client personnel and top

management support as important factors but Sumner notes these are not unique to ERP

projects.

Thus the risk taxonomies and suitable strategies for software package

implementation are likely to be different from those for in-house software development.

We would, however, expect software package implementation risks to be closer to the

bespoke project risks that Moynihan (1997) identified, since both package

implementations and bespoke projects address an interface between two organizations –

vendor-client for packages and external developer-client for bespoke development. In

27

contrast, in-house development involves an internal interface between different

departments (IT and client department) of the same organization. Moynihan emphasizes

the need for external organizations (i.e. those offering bespoke custom development

services) to take care when ‘scouting out’ new projects, and notes that his risk themes

contain clues to the sorts of intelligence about the client and his/her organization that the

vendor/developer should gather before undertaking a new project. These clues include

risks such as potential conflict between user departments, multi-vendor projects, and

inexperienced client project managers, none of which were ranked highly by the project

managers participating in Schmidt et al.’s (2001) Delphi survey. However, Schmidt et al.

noted that some of the risks in their survey that were not ranked highly appeared to be

risks that the managers perceived as being outside their control. Since the managers in

Schmidt et al.’s Delphi survey were all in-house managers considering risk issues related to

projects within their organizations, they presumably would have to take on projects if

mandated to do so, and thus would have no choice about having to deal with issues such

as user conflict and multiple vendors. The managers in Moynihan’s study, on the other

hand, were all managing bespoke development projects for external clients, and could

choose to simply decline a project if they perceived the risk to be too high.

Prescriptive IT risk management literature. While the descriptive risk factors

strand has focused on providing comprehensive identification of the potential sources of

risk for IT projects, the prescriptive risk management strand of articles aims to give

guidance on how to manage the risk process for IT projects, in order to ensure the best

chance of success. Risk management researchers (see, for example, Barki et al., 2001;

Boehm, 1991; Charette, 1996b; Fairley, 1994; Heemstra & Kusters, 1996; Keil et al., 1998;

Powell & Klein, 1996) all provide frameworks that are variations and elaborations on the

general project risk management processes discussed above, of risk identification, risk

assessment, risk response planning, and risk monitoring. These researchers argue that past

IT project failures may have been avoided with better risk management methods, while

other researchers have warned of the risk of complacency when traditional risk

management techniques have been applied, because of difficulties in precisely quantifying

risk (Pfleeger, 2000), and the potential for risk analysis to actually compound the risk by

creating a false sense of security and control (Drummond, 1996).

Some of the writers in the prescriptive risk management strand also propose various

contingency approaches, aimed at providing the project manager with decision tools for

28

deciding when to apply certain risk management methods in order to achieve the best

chance of project success. For example, Barki et al. (2001) embed their risk management

processes within a contingency model, suggesting that the degree of formal planning,

internal integration, and user participation should be matched to the level of risk exposure

identified for a particular project, with high-risk projects requiring higher levels of

planning, integration, and participation. Contingency approaches have also been

advocated by Davis (1982), who proposed selecting different requirements determination

approaches for projects with differing degrees of uncertainty, and McFarlan (1981), who

based his recommendations for risk resolution on an assessment of the project’s risk in

terms of size, structure and experience with technology. Other researchers have noted that

the application of contingency approaches is not straightforward in practice, and that

many projects appear to employ strategies that, according to the contingency approach

recommendations, do not necessarily match the level of uncertainty in the project

(Baskerville & Stage, 1996; El Louadi, Galletta, & Sampler, 1998; Moynihan, 2000a).

For example, in one of the few empirical studies exploring project managers’

strategies, Moynihan (2000a; 2002) used an exploratory analysis with twenty project

managers, who worked with bespoke system development projects, to discover strategies

that these managers said they would use when faced with certain hypothetical risky

situations. Moynihan found the managers proposed broad strategies for a variety of

different situations, with no clear links between the type of strategy and the uncertainty

level of the situation. He also found that a major underlying principle seemed to be the

need for the project managers to protect themselves and their immediate organization

against any adverse impact from project problems. Moynihan’s project managers did

appear to apply some aspects of the contingency method proposed by Barki et al. (2001)

in that they advocated increased formal planning and documentation when faced with

higher project risks, particularly risks relating to uncertain requirements and ‘people

problems’ (Moynihan, 2002). While Moynihan’s study is interesting in that it identifies

some risks not previously featured and provides insight into project managers’ espoused

theories about strategies to apply, it tells us little about how project managers have actually

managed risks on real projects they have worked with, and it sheds little light on whether

IT project managers actually follow any of the prescriptions for risk management in the

literature.

29

Obviously, there is an abundance of advice in the literature identifying the most

likely sources of risk for IT projects, and on the best approaches to take in order to

manage risks and ensure successful project outcomes. Yet, the reports of IT projects in

trouble still continue. Is this because IT project managers are failing to apply appropriate

risk management techniques, or are there underlying assumptions in the literature

discussed above that do not reflect actual practice? Rational decision-making is one such

assumption underlying project management theory, and I discuss the concept of rational

decision-making in the next section.

2.2.4. Rational decision-making

The approach to project risk management outlined above, both generally and in the

IT field, is premised on a rational approach to decision-making by project managers,

which assumes that project managers will decide what to do about risks by first calculating

the likelihood of the risk and its potential impact and then choosing a response based on

the expected value (likelihood times impact) assigned to the risk (Lyytinen et al., 1998;

March & Shapira, 1987). However, empirical studies of how managers’ handle risk suggest

that managers typically are insensitive to probability estimates of risk, and also focus on

only a few key aspects of risk in a situation at any given time (March & Shapira, 1987).

This more subjective perspective of managers’ approaches to decision making is

supported by Mintzberg et al.’s (1976) work on unstructured decision processes. While the

literature on strategic decision-making has focused on a rational and analytical method for

the evaluation and choice of decisions, Mintzberg et al.’s study revealed very little use of

such an approach, and found that the exercise of managerial judgment was the preferred

mode of decision selection.

A second underlying assumption of the rational approach to IT project risk

management is that problems have occurred in IT projects because project managers do

not practice rational risk management techniques. Yet, there is little empirical research to

support this view. Researchers reporting case studies of failed projects, and reviewing risk

management procedures in these projects, have drawn differing conclusions. For example,

Lyytinen, Mathiassen, and Ropponen (1996) analyzed two previously published case

studies, one of a successful project, and one of a project that was finally abandoned, and

concluded that the successful project demonstrated active risk management, while the

abandoned project showed no evidence of specific risk management strategies. Baskerville

and Stage (1996) and Heemstra and Kusters (1996) reported successful action research

30

applications of rational risk management strategies. However, Drummond (1996), in an

extensive analysis of the failure of the London Stock Exchange Taurus project suggested

that risk management techniques may have actually compounded the problems by

fostering an illusion of control. Researchers promote rational prescriptive frameworks for

risk management presumably because they believe that practitioners will adopt such

approaches if they learn about them through these prescriptive writings or training.

However, a review of the decision-making training literature by Means, Salas, Crandall and

Jacobs (1993) found little evidence that people made better (rationally based) decisions

after training.

In IT projects, there is evidence to suggest that project managers are not likely to be

rational decision makers when assessing risk (Schmidt et al., 2001). Indeed, Charette

(1996) claims that most software project managers know little about formal risk

management or how to integrate risk management techniques into their existing project

management activities. On the other hand, Boehm (Boehm, 1991) claims, from his

anecdotal experience, that successful project managers use a general concept of “risk

exposure” (potential loss that a risk event would entail times the probability of the risk

event), even if they do not use the formal terminology of risk identification, assessment,

and monitoring. Other researchers note the lack of risk anticipation by IS developers

(Willcocks & Margetts, 1994) and there is evidence that some IT project managers focus

mainly on a few factors and largely ignore others (Moynihan, 1997). IT project managers

tend to take what Simon (1979) calls a satisficing approach to software development

decisions; that is they choose courses of action that are “good enough” to meet the

identified needs, but not necessarily the optimal solution (Lyytinen et al., 1996). Rather

than react to a risk choice by simply choosing the alternative that has the lowest expected

risk value, IT managers look for ways to manage or change the risk or avoid it altogether,

in order to influence and control the likely outcome (Lyytinen et al., 1998). Pablo (1999)

observes that software development managers focus more on the impact of a possible

risky event, and comparatively less on the likelihood of the event or the extent to which it

can be controlled, than managers from either the oil and gas or the commercial banking

industries.

While it seems that IT project managers do not use a rational process for risk

assessment, much of the writing is anecdotal or conjecture, and not supported by

empirical research into what managers actually do in terms of assessing and managing

31

risks. Thus we need to know more about how experienced IT project managers manage

the risks in their projects, and how they make decisions about what to do in risky

situations. By understanding how experienced decision-makers actually make decisions

about risky situations, rather than simply prescribing a theoretical framework, we may gain

a better understanding of how to improve decision-making training for novices. The

naturalistic decision-making approach described next may have more to offer than the

commonly accepted prescriptive rational analyses.

2.2.5. Naturalistic decision making

A more recent strand of applied decision-making research has focused on an

approach called Naturalistic Decision Making (NDM), which aims to understand how

people use experience to make decisions in real-life settings, rather than to prescribe a

particular decision-making approach (Klein, 1999). Experienced decision makers in NDM

situations typically use their experience to gain understanding of the situation and

feedback on it, rather than comparing and evaluating multiple alternative solutions (Klein,

1999; Zsambok, 1997). Features of NDM situations include the use of experience, the

importance of context and cue learning, and the need to cope with uncertainties and

difficulties such as ill-defined goals, time pressure, inadequate information, and dynamic

and continually changing conditions. Clearly, there are several similarities between these

features of NDM situations and IT projects. For example, the issues of unrealistic time

schedules, unclear and changing goals and requirements, amount of organizational change

required, and stability of corporate environment feature highly in the IT project risk

factors identified by several authors (Barki et al., 1993; Boehm, 1991; Heemstra &

Kusters, 1996; Moynihan, 1996; Schmidt et al., 2001).

NDM deviates from rational decision-making research in considering the strategies

used by experienced personnel as possible standards for performance, even when these

standards do not conform to the recommended rational approach of option identification

and analysis (Klein, 1999). Cognitive task analysis methods such as the critical incidents

technique (Flanagan, 1954) and the critical decision method (Klein et al., 1989) are

especially useful to understand jobs that are complex and dynamic, and the critical cues

observed by experts in such jobs (DuBois, 2002; Gordon & Gill, 1997), and have been

successfully applied to investigate decision making in real-life situations (Klein et al.,

1989). These studies can lead to focused training programs using scenarios and cognitive

feedback and modelling to develop situational awareness (Klein, 1997a). Such programs

32

have been shown to improve meta-cognitive skills and decision-making (Cohen, Freeman,

& Thompson, 1997; DuBois, 2002).

The Recognition Primed Decision (RPD) model has been developed in an attempt

to explain how people make decisions in NDM situations (Klein, 1993, 1997b; Klein et

al., 1989), and is of interest for the present study. The RPD model seeks to address

research findings suggesting that experts have rarely reported considering more than one

option at a time (Klein, 1997a; Klein et al., 1989). Instead, they say that they focused on

awareness of the situation, by observing contextual cues and patterns in order to size up

the situation and recognize aspects of it as typical of other cases in their experience,

maybe with certain anomalies. The anomalies often provide the early warnings that alert

experts to prepare for contingencies.

In the RPD model, illustrated in Fig 2.2, decision makers first experience a particular

situation in its own changing context. This leads to an assessment of the familiarity of the

situation. Recognition of aspects of the situation as typical in the decision maker’s

experience will have the following by-products: certain relevant cues will be observed;

there will be certain expectancies about the situation; plausible goals for this situation will

be established; and certain typical actions will be identified that might achieve goals related

to this situation. The expectancies are compared with the situation and its context and if

anomalies are detected further clarification is sought and the situation is reassessed. If

there are no anomalies then the decision maker chooses a promising course of action

based on past experience with similar situations, and conducts a mental simulation of the

action to rehearse whether it will work. If the mental simulation is satisfactory, the action

is implemented. Otherwise, the action may be modified and tested again, or another

action may be chosen and tested.

Particular features of the RPD model that distinguish it from rational decision-

making models include the emphasis on situation assessment rather than on option

evaluation; the focus on serial evaluation of actions, which allows a satisficing approach of

selecting the first workable action, rather than concurrent evaluation of all actions to select

the best action; and the assertion that decision-makers evaluate actions by using mental

simulations rather than by weighing up the strengths and weaknesses of each action. While

NDM, and the RPD model, in particular, have been explored in several different areas,

including decision making in fire-fighting, armed services, airline flight control, nursing,

33

chess playing and hardware and software design, there has been little application of this

theory as yet to management and business studies.

Fig. 2.2: Recognition-primed decision model (after Klein et al., 1989)

Given that IT projects have many of the features for which NDM applies, this study

explores the utility of NDM and the RPD model for describing and understanding risk

management decisions in IT projects. A particular aspect of NDM and RPD is the focus

on knowledge gained from experience, i.e. tacit knowledge. I review the concept of tacit

knowledge next.

Experience Situation in Changing Context

Is the situation

familiar?

Recognize the Situation

Goals

Expectancies Actions

Cues

Are expectancies

violated?

Will it work?

Imagine

Action

Implement Action

Modify Action

Reassess situation

Seek more information

Yes, but

Yes

Yes

No

Yes

No

No

34

2.3. Tacit knowledge

The concept of tacit knowledge has been used by researchers in a wide range of

disciplines, with a corresponding variety of meanings and characterizations. Consequently,

there is some confusion in the literature over the exact definition of tacit knowledge, and

its relationship to similar concepts such as implicit learning, procedural knowledge, and

practical intelligence (Berry & Dienes, 1993; Castillo, 2002). Thus, I firstly give an

overview of the historical beginnings of the tacit knowledge concept (Section 2.3.1), and

then review and clarify the related concepts of implicit learning (Section 2.3.2), and

declarative and procedural knowledge (Section 2.3.3). In Section 2.3.4 I discuss various

categories of tacit and explicit knowledge that have been proposed by researchers. Finally

in Section 2.3.5 I review empirical tacit knowledge research in the management field. In

particular, I discuss the definition of tacit knowledge developed by Sternberg et al. (2000),

and their critical incident approach to the elicitation of tacit knowledge, which is similar to

the critical decision method used by the NDM researchers. The critical incident and

critical decision methods form the basis of the methodology proposed in Chapter 3 to

investigate the research questions of this thesis, and will be discussed in detail in that

chapter.

2.3.1. Historical beginnings

The study of tacit knowledge originated from the philosophical work of Polanyi

(1966), who laid a theoretical foundation, and coined the often quoted phrase “we can

know more than we can tell”. Drawing on Ryle’s work (1949), Polanyi argued that there

are two aspects of knowing, “knowing what” and “knowing how”, and that these two

aspects of knowing are always both present in any instance of a person’s knowledge.

However, frequently we are able to articulate what we know without being able to explain

how we know. For example, we can easily say whether we recognize a person’s face, but we

are generally unable to explain how we know that the face is a familiar one. Thus,

according to Polanyi, we know that tacit knowledge exists because we can see the practical

outcomes of its application and can thus infer that there must be some implicit or tacit

knowledge that the person has but cannot articulate. Using the example of a skilled car

driver, Polanyi maintained that even if we focus on thoroughly and explicitly specifying

the (tacit) knowledge that the driver uses to complete a complicated manoeuvre, another

person will not be able to replicate the manoeuvre just by studying the explicit

instructions. Thus Polanyi argued that the aim of explicitly and objectively formalizing all

35

knowledge may not be achievable, as the implicit or tacit knowledge cannot be fully

replicated as formal explicit knowledge.

While Polanyi set a philosophical foundation for the concept of tacit knowledge he

said little about the processes of acquiring or learning tacit knowledge. Two major lines of

research in the cognitive psychology field have involved implicit learning and the

processes of skills acquisition.

2.3.2. Implicit learning

Implicit learning has been the focus of a wide range of studies in the social sciences.

Implicit learning occurs when a person acquires knowledge without a conscious attempt

to do so, and largely without explicit awareness of what was acquired (Reber, 1993). Thus,

it is difficult for the person to articulate or describe the resulting knowledge acquired.

There is some confusion in the literature over the implicit-explicit distinction, with some

researchers referring to the type of knowledge possessed, while others refer to the

different learning processes of acquiring knowledge (Berry & Dienes, 1993). In general,

explicit knowledge is knowledge a person can easily explain or describe, while implicit or

tacit knowledge is knowledge that a person may be unaware of having, and that is difficult

to articulate. Explicit learning occurs in more formal teaching and learning settings or

when conscious learning strategies are applied. In contrast, implicit learning occurs when a

person acquires knowledge without the use of conscious strategies, and often without

being aware of the knowledge gained.

Implicit knowledge is not always acquired implicitly, nor is explicit knowledge always

acquired explicitly. Some researchers (Anderson, 1982, 1983; McCloy, Campbell, &

Cudeck, 1994) have hypothesized that skills acquisition passes through three stages, with

the final stage occurring when the knowledge about how to execute the skill, originally

learned explicitly, is ‘internalized’ or held implicitly (see following section on declarative

and procedural knowledge for discussion on this view). Other researchers (Nonaka, 1994;

Nonaka, Toyama, & Konno, 2001; Takeuchi, 2001) have developed a view of knowledge

creation that involves a continuous interaction between implicit and explicit knowledge in

order to produce new knowledge. Tacit or implicit knowledge can be converted to explicit

knowledge by “reflection in action” (Schön, 1983), by the use of metaphor and analogy

(Nonaka, 1994), or by using mentoring and story-telling (Swap, Leonard, Shields, &

Abrams, 2001). However, although it is possible to ‘externalize’ implicit knowledge, some

36

aspects of implicit knowledge, particularly those related to creativity, intuition, and skill

performance, are unlikely to ever be made completely explicit (Leonard & Sensiper, 1998;

Polanyi, 1966; Reber, 1989).

Just as Polanyi considered that the two aspects of knowing (knowing what and

knowing how) are always present in any instance of knowledge, so Reber (1993) argues

that implicit and explicit learning are not completely separate, but are interactive or co-

operative processes existing along a continuum. In particular, in complex learning

situations a person’s performance is likely to involve both implicit and explicit learning

processes (Anderson, 1982; Argyris & Schön, 1978; Berry & Dienes, 1993; Nonaka, 1994).

Several laboratory studies have investigated implicit learning, including studies on

artificial grammar learning (Reber, 1989), rule-governed stimulus sequences (Lewicki, Hill,

& Bizot, 1988), and the control of complex systems (Berry & Dienes, 1993). Studies of

patients suffering from neurological or psychological impairment have also supported the

idea of a distinction between implicit and explicit learning (Berry & Dienes, 1993; Reber,

1993). These studies have demonstrated that subjects can acquire knowledge that they are

unaware of. For example, in artificial grammar learning experiments (Reber, 1989)

subjects were able to learn to apply grammar rules derived from sequences of letters that

they had been shown previously in order to determine whether a new sequence was

grammatically correct. Even though their success rate was significantly better than could

be expected by chance alone, the subjects had no awareness of and could not explain the

rules they were applying. Typically, they reported that they were just guessing or using

intuition.

While these laboratory studies have demonstrated the acquisition of tacit knowledge,

there is still speculation about how that knowledge is stored. Two possible structures have

received attention (Berry & Dienes, 1993; Reber, 1993). The first is the abstract view,

where people develop mental models or rules that they can apply to new situations of a

similar type. The second is the exemplar view, where people use analogous past

experiences to help determine a response to a new but similar situation. Berry and Dienes

suggest that the relative abstractness of tacit knowledge may lie on a continuum, while

Reber argues that an abstract tacit knowledge base may gradually develop as the number

of past experiences becomes large. Stanley et al. (1989) propose that people typically use

exemplars or close analogies to (implicitly) determine their responses, but if pressed to

describe or explain their actions they will attempt to do so using a mental model or rule-

37

based approach. Like Reber, Stanley et al. suggest that the more abstract, mental model

approach is likely to develop as experiences increase, and they note that mental models

and rules are more explicit in their nature, reflecting the continuous nature of explicit and

implicit modes.

A second line of psychological research, led by Anderson (1982; 1983), has focused

on a slightly different learning distinction, the declarative-procedural dimension of skill

acquisition.

2.3.3. Declarative and procedural knowledge

Anderson (1982) proposed a framework for skill acquisition that describes learners’

progression through three stages during the development of a cognitive skill. Drawing on

work by Fitts (1964) and Ryle (1949), Anderson’s first stage of acquiring declarative

knowledge, corresponding to Fitts’ cognitive stage, can be thought of as learning to

“know what”. In this first declarative stage, learners acquire knowledge of the facts and

procedures that are required in order to perform the skill. At this stage learners are able to

answer questions about what they ought to do when applying the skill, but they may not

be able to demonstrate the skill in practice. The practical ability to apply the skill comes

only after the acquisition of procedural knowledge, which is Anderson’s second stage of

learning and corresponds to Fitts’ associative stage. This procedural knowledge stage

involves learning to “know how” by actual practice of the skills. At this second stage

learners are able to integrate their declarative knowledge (knowing what to do) with a

knowledge of how to do the task, and are actually able to perform the task successfully

(Anderson, 1982; McCloy et al., 1994). Anderson’s third, tuning stage corresponds to

Fitts’ autonomous stage, and occurs when the skill can be done more and more

automatically, without thinking, until the practice of the skills becomes smooth and

unconscious. For example, when learning to drive a car, a person will typically first acquire

some knowledge about what to do in terms of controlling the steering, accelerator and

brake, and then will practice how to perform these tasks until s/he is competent. At first,

all of the learner’s attention will be focused on the driving, but with continued practice a

driver can safely control the drive while listening to the radio or holding a conversation.

At this point, the knowledge related to the driving skill is largely outside the scope of

consciousness.

38

Intuitively, it seems reasonable to relate the acquisition of declarative knowledge with

explicit learning, and the gaining of procedural knowledge with implicit learning. But

Anderson’s theories regarding the acquisition of procedural knowledge after the learner has

progressed through the declarative learning stage seem to be at odds with the ideas

developed by the implicit learning theorists, who view implicit learning as taking place

without conscious awareness, and without necessarily any prior explicit learning.

However, although Anderson’s descriptions of the declarative stage tend to suggest that

he has some kind of verbal instruction in mind, it can be argued that the acquisition of

“knowing what” can happen by observation, as for example when a child watches a

sibling riding a bicycle. Indeed, such observation need not have a ‘conscious’ component

at all. For example, many of the skills we acquire as children we learn by imitating others,

without being consciously aware that we have absorbed any declarative “knowing what”

knowledge at all. And the implicit learning experiments discussed in the previous section

all have a learning stage where the subjects are exposed to (observe) the clues that they

later apply even though they are unaware that they have made any such observations. So

Anderson’s declarative-procedural distinction may be argued to be a learning process that

applies to both explicit and implicit learning, with learners being unaware of the

declarative stage when they learn implicitly.

Recall also that theorists do not view the implicit-explicit dimension as dichotomous,

but rather to be a continuum (Berry & Dienes, 1993; Polanyi, 1966; Reber, 1993). So the

idea that implicit (procedural) knowledge may be derived from knowledge that was once

learned explicitly (declaratively) simply reflects the continuous nature of the implicit-

explicit dimension. However, implicit (procedural) knowledge that was originally derived

from an overt declarative process is likely to be easier to articulate than implicit knowledge

acquired when the learner is unaware of any initial declarative stage (Berry, 1987).

Anderson’s theory, like the implicit learning studies, has been based mainly on

experimental laboratory research. The implicit learning studies have focused on

determining whether subjects can acquire knowledge tacitly, while Anderson’s work has

focused on building a theory of how skills are acquired. I turn now to consider research

investigating the tacit knowledge concept in applied domains, particularly in the areas of

management, business, medicine and the armed forces. Research in these areas has

focused more on theorizing about various categories of knowledge and the nature of

39

different types of tacit knowledge, and on whether and how this tacit knowledge can be

elicited and transferred to others.

2.3.4. Categories of knowledge

In business research, interest and research in tacit knowledge has increased in recent

years. However, in applied management studies, researchers have used various terms and

concepts to describe types of knowledge, with a corresponding lack of consistency in the

operationalization of the tacit knowledge concept, and on what distinguishes tacit

knowledge from explicit knowledge (Ambrosini & Bowman, 2001; Castillo, 2002). While

most writers have simply used the term explicit to describe knowledge that is formally

learned and easily articulated, Spender (1996) uses the term conscious, Ambrosini and

Bowman (2001) use the term objective, and Blackler (1995) and Lam (2000) have followed

Collins (1993) in using the term embrained to refer to this category of knowledge. While

there is general agreement on the use of the terms implicit or tacit to refer to the broad

category of knowledge that individuals find difficult to articulate and have learned by

experience, by practice (‘doing’) or by ‘osmosis’, researchers have used different degrees of

granularity, and different terms, in discussing types of implicit knowledge. Other

researchers have also investigated the idea of collective explicit and implicit knowledge –

knowledge that is held by a group or organization rather than a single individual. Again

various terms and definitions have been used. I review research related to individual and

collective tacit knowledge in detail next and summarize the various concepts, terms, and

definitions in Table 2.1 at the end of this section.

Individual tacit knowledge. Of the group of researchers focusing on individual

tacit knowledge, some, such as Ambrosini and Bowman (2001), view tacit knowledge as

relating solely to an individual’s skills development, while others (Blackler, 1995; Castillo,

2002; Lam, 2000) make no distinction between tacit skills and tacit knowledge. Nonaka

(1994) and Takeuchi (2001), however, believe that tacit knowledge can have both a

technical and cognitive dimension. Technical tacit knowledge is skills know-how, learned

implicitly through experience. It is usually not possible for an individual to articulate or

describe this technical know-how, but it can still be transferred to others by non-verbal

means such as apprenticing to an expert, being mentored by an expert, or observation and

behaviour modelling. Cognitive tacit knowledge is knowledge that is developed implicitly

using “mental models” or exemplar situations. These mental models are so ingrained that

we take them for granted. While experts can be asked to articulate their cognitive tacit

40

knowledge, their verbal reports may be inaccurate as they don’t really know at a conscious

level why they choose certain actions (Schön, 1983). Consequently their explanations may

be more related to what they think ought to underpin their knowledge, rather than what

actually does (Hsia, 1993; Johnson, 1983; Parnas & Clements, 1986). However, Nonaka

and Takeuchi argue that this tacit knowledge can be made at least partially explicit by the

use of metaphor, analogy and prototype, and Sternberg and Wagner (1986) and Klein et

al. (1989) have both developed interview techniques based on the use of story-telling

approaches to facilitate the elicitation of this type of tacit knowledge. These approaches

are discussed in more detail in Chapter 3.

Researchers have also focused on distinguishing levels of individual tacit knowledge

or skills relating to the extent to which the knowledge can be articulated or transferred to

others. Both Castillo, (2002) who also considers a collective dimension to tacit knowledge,

and Ambrosini and Bowman (2001) differentiate three levels of implicitness of an

individual’s tacit knowledge. Castillo’s first level, nonepistle tacit knowledge, is knowledge

that is the result of implicit learning and is completely inarticulable, or “deeply ingrained”

in Ambrosini’s terms. This nonepistle tacit knowledge is extremely difficult, if not

impossible for individuals to access, and therefore is unlikely to be explicitly transferable

to other individuals (Ambrosini & Bowman, 2001; Castillo, 2002; Leonard & Sensiper,

1998). However, Nonaka’s skills know-how form of tacit knowledge falls into the

nonepistle category, since it is implicitly learned and inarticulable, but it can still be

transferred in a more implicit fashion by the methods of apprenticing, observation, and

mentoring (Leonard & Sensiper, 1998; Nonaka, 1994; Spender, 1996).

Castillo’s second level of tacit knowledge is sagacious knowledge, corresponding to

Ambrosini’s “imperfectly articulated tacit skills” and Nonaka’s cognitive form of tacit

knowledge. Sagacious knowledge is a tacit form of knowing that “emanates in an acute and

keen practical sense”. Ambrosini concurs with Nonaka in viewing this knowledge as

accessible through metaphor and analogy.

Finally, Castillo’s third level of tacit knowledge is semantic knowledge, which is explicit

knowledge that has been made implicit, or internalized in Nonaka’s terminology. Semantic

knowledge is often discerned in conversations between experts, who base their

communication on the implicit assumption that they share a common understanding of

the technical foundations and abstract expressions of their expert area, and thus never

explicitly discuss basic terminology and definitions (Castillo, 2002). However, semantic

41

knowledge was once explicit and so can be re-captured relatively easily by asking the right

questions (Ambrosini & Bowman, 2001; Castillo, 2002; Nonaka, 1994).

Collective tacit knowledge. Several researchers, including Blackler (1995), Lam

(2000), Spender (1996), and Collins (1993), have hypothesized about collective knowledge

and sub-categories of collective knowledge. There is some confusion in terminology

among these researchers with the same term being used for different definitions of

collective knowledge. The general term, collective knowledge, has been used to describe

the totality of the knowledge, both explicit and tacit, held by all members of a group,

organization or society, with different individuals within the group holding differing sets

of knowledge (Lam, 2000; Spender, 1996). Researchers have distinguished between

collective explicit and collective implicit knowledge as follows. Collective explicit

knowledge, called encoded knowledge by Collins, Blackler and Lam, and objectified

knowledge by Spender, is viewed as being held in common repositories such as libraries,

books, and formal data media (or in verbally transmitted lore for oral societies). As such, it

is readily accessible by all (authorized) members of the group, and is typically transferred

by formal learning procedures.

Collective implicit knowledge, termed embedded knowledge by Blackler and Lam, socio-

cultural knowledge by Castillo (2002), and simply collective knowledge by Spender, resides in

systemic routines, and the relationships between technologies, roles, and (unwritten)

formal and informal procedures in the group, organization or society. Collective implicit

knowledge may be thought of as “the way we do things round here”. As such, it will

comprise elements of individual members’ explicit and implicit knowledge, since much of

an individual’s knowledge of group procedures and routines can be easily articulated. The

key point is that, although individual members can articulate much of this knowledge, it

has not been formally captured and recorded in the group’s explicit knowledge repository,

and thus it remains at the implicit level for the group as a whole. Therefore it is only

accessible to other members of the group if they know the right person to ask, and is not

accessible to people outside the group. Blackler distinguishes a further subset of collective

implicit knowledge, encultured knowledge, which refers specifically to the knowledge that

individuals hold about the cultural or social norms regarding how to behave or interact

with others in the group in specific situations. Individuals usually learn encultured

knowledge implicitly as part of an on-going socialization process. Although it is rule-

driven, in that members of a group can usually explain the rule about appropriate

42

behaviour in a given context, the rule will change for each social context, and it is

impossible to completely specify all appropriate behaviours for all contexts (Collins, 1993).

43

Table 2.1: Categories and sub-sets of knowledge

Major category

How learned

How held

How manifested, articulated, and transferred Terms used References

Implicit Implicit Manifested in outcomes or actions. Most likely skills-based.

Inarticulable.

Transferred by demonstration, observation, apprenticing, behaviour modelling, actual practice or doing.

Nonepistle tacit knowledge

Deeply ingrained tacit skills

Embodied knowledge

Tacit knowledge – technical skills know-how

Subset of automatic knowledge

(Castillo, 2002)

(Ambrosini & Bowman, 2001)

(Blackler, 1995; Collins, 1993; Lam, 2000)

(Nonaka, 1994; Takeuchi, 2001)

(Spender, 1996)

Implicit Implicit Manifested in outcomes or actions, and demonstration of an “acute and keen practical sense” (Castillo, 2002). Most likely cognitively based, with mental models or exemplars.

Perhaps partially articulable, but attempts to explain may be inaccurate.

Transferred as above, and by mentoring, metaphor, analogy, storytelling, critical incident studies, and prototype.

Sagacious tacit knowledge

Imperfectly articulated tacit skills

Tacit knowledge – cognitive dimension

Practical thinking

Practical intelligence – tacit knowledge

Subset of automatic knowledge

(Castillo, 2002)

(Ambrosini & Bowman, 2001)

(Nonaka, 1994; Takeuchi, 2001)

(Scribner, 1986)

(Sternberg et al., 2000; Sternberg & Horvath, 1999; Wagner, 1987)

(Spender, 1996)

Individual tacit or implicit knowledge

Explicit Implicit Manifested in common or shared understanding of technical foundations and abstract expressions of expert area. Explicit knowledge that has been “internalized” (Nonaka, 1994).

Articulable.

Transferred by questioning to elicit or ‘surface’ the underlying explicit knowledge base.

Semantic tacit knowledge

Tacit skills that can be articulated

Internalized knowledge (from explicit to tacit)

Subset of conscious knowledge

(Castillo, 2002)

(Ambrosini & Bowman, 2001)

(Nonaka, 1994; Takeuchi, 2001)

(Spender, 1996)

Individual explicit knowledge

Explicit Explicit Manifested in individual’s ability to explain items from the collective store of ‘hard data’.

Readily articulated.

Transferred by formal learning procedures e.g. schools, reading, formal training etc.

Objective knowledge

Conscious knowledge

Embrained knowledge

Declarative knowledge

(Ambrosini & Bowman, 2001)

(Spender, 1996)

(Blackler, 1995; Collins, 1993; Lam, 2000)

(Anderson, 1983)

44

Table 2.1: Categories and sub-sets of knowledge continued

Major category

How learned

How held

How manifested, articulated, and transferred Terms used References

Collective or social explicit knowledge

Explicit Explicit Held in repositories such as libraries, books, formal data media, written rules and procedures (or in verbally transmitted lore for oral societies). The sum of explicit knowledge in a group, organization or society.

Readily articulated either verbally or in written form.

Transferred by formal learning procedures.

Encoded knowledge

Objectified knowledge

(Blackler, 1995; Collins, 1993; Lam, 2000)

(Spender, 1996)

Mainly implicit but can be explicit

Implicit Manifested in social interactions and shared understandings of social norms and behaviours.

May be rule-driven, but cannot be fully articulated explicated, as each application of the rule is dependent on the social context.

Transferred mainly by “socialization” - observation and informal behaviour modelling. Can also be transferred by direct explanation of the rule in a particular context.

Encultured knowledge

Subset of embedded knowledge (shared beliefs and understanding)

(Blackler, 1995; Collins, 1993)

(Lam, 2000)

Collective or social implicit knowledge

Explicit and implicit

Implicit Resides in systemic routines. Manifested as “many individual kernels of tacit and explicit knowledge that jointly determine a … system of facts, procedures, and routines …” (Castillo, 2002)

Can be articulated in systems terms in the relationships between technologies, roles, (unwritten) formal procedures and routines (Blackler, 1995).

Transferred informally by observation and by on-the-job training of “the way we do things round here”.

Embedded knowledge

Subset of embedded knowledge (organizational routines)

Sociocultural tacit knowledge

Collective knowledge

(Blackler, 1995)

(Lam, 2000)

(Castillo, 2002)

(Spender, 1996)

45

2.3.5. Tacit knowledge research in the management field

While management researchers have theorized about tacit knowledge, there is little

empirical research in the management arena (Ambrosini & Bowman, 2001). A notable

exception is the group of researchers working with Sternberg and Wagner (Sternberg et

al., 2000; Sternberg & Horvath, 1999; Sternberg & Wagner, 1986) who have bridged the

gap between psychology and management research in this area, and have brought the

operational discipline of the psychological approach to bear on the issue of tacit

knowledge in applied management areas. They are notable both for developing a clear

definition of their concept of tacit knowledge and for gathering a substantial body of

empirical evidence to support their concept. While Sternberg and colleagues’ definition of

tacit knowledge aligns best with Castillo’s (2002) sagacious knowledge (a form of knowing

that “emanates in an acute and keen practical sense”), it also covers an individual’s socio-

cultural collective knowledge in that tacit knowledge is defined as including an individual’s

ability to perform within his/her (social) environment.

Sternberg et al. define the concept of tacit knowledge quite simply as knowledge

acquired implicitly from everyday experience (Sternberg et al., 2000). It is difficult for the

possessor of this knowledge to articulate or explain what s/he knows. According to

Sternberg, tacit knowledge has three key features. The first feature is that it is generally

acquired with little or no support from the environment. That is, there is no systematic

support from other people or media (e.g. books) to help a person gain tacit knowledge.

Rather this is the knowledge that a person gains by experience, by observation and trial

and error, and some may gain more than others.

The second key feature of tacit knowledge is that it tends to be procedural i.e.

'knowing how' rather than 'knowing what', in that it is knowledge that guides behaviour.

Procedural knowledge and tacit knowledge are not equivalent, however, for two reasons.

Firstly, tacit knowledge includes only that subset of procedural knowledge that is difficult

for a person to articulate or explain. Thus, procedural knowledge that a person can readily

explain or demonstrate is not considered to be tacit knowledge. Secondly, procedural

knowledge is generally considered to be related only to skills acquisition, while Sternberg’s

tacit knowledge also encompasses Nonaka’s (1994) cognitive dimension, and includes the

knowledge of what to do in specific contexts.

46

Finally, according to Sternberg’s definition, tacit knowledge is knowledge that is

practically useful i.e. it has a direct practical outcome. This outcome may be of use to the

person in his or her work situation, but tacit knowledge can also have practical outcomes

in social and personal settings.

Sternberg and his colleagues have examined tacit knowledge in several different

settings, including academia, military leadership, sales and business management, and they

have devised an approach to measuring tacit knowledge that takes into account the

contextual and experience-based nature of the knowledge (Sternberg et al., 2000). Their

approach relies on a critical incident approach (Flanagan, 1954) to interviewing domain

experts in order to tap into any tacit knowledge they may possess. This approach is very

similar to the critical decision method used by the NDM researchers (Klein et al., 1989)

and discussed earlier. The research approach used in this thesis is based on the critical

incident and critical decision methods and is discussed in more detail in Chapter 3.

2.4. The present study

We know what risk factors IT project managers say they attend to when assessing

new projects (Barki et al., 1993; Moynihan, 1997; Schmidt et al., 2001), but we still know

little about how project managers in the field actually carry out risk management processes

for their projects. The question of what risk symptoms managers look for to give them an

early warning of impending problems has not been addressed at all, and we know little

about the countermeasures managers employ against risks in their projects. As I have also

discussed, the important risks and suitable strategies for projects with a software package

focus are likely to be different from those for in-house software development. There is an

increasing trend towards the use of (typically customized) software packages rather than

in-house custom-built systems (Russo, 2000), and indeed, some authors have suggested

that the use of packaged software can overcome some of the risks that have been

encountered when developing in-house systems (Lassila & Brancheau, 1999; Lucas Jr.,

Walton, & Ginzberg, 1988; Martin & McClure, 1983). While there has recently been some

focus on identifying risk factors for package-focused implementation projects, less is known

about the management of risks in these types of project. Thus the present research focused

primarily on risk management of software package implementation projects.

Several rational frameworks have been proposed to guide IT project managers

through the risk management process, yet we do not know whether IT managers take this

47

rational approach to their risk management decisions, or whether, as has been shown for

managers in general (Mintzberg et al., 1976), they are more likely to rely on expert

judgment and experience when making decisions. Anecdotal evidence and conjecture

suggests that IT project managers take a more naturalistic approach to their risk

management decisions, choosing actions on a satisficing basis. We know that risks are

situation-dependent (March & Shapira, 1987), and therefore managers must not only

identify risks but must also understand them in their context. The way in which project

managers evaluate and interpret the social and environmental contexts of their projects in

terms of risk management can have drastic impacts on project outcomes (Lyytinen et al.,

1998). In particular, the risk taking behaviour of managers may be better described by the

naturalistic decision making model, which reflects the exercise of judgment and

experience about “the subtle practical problems of sustaining appropriate risk taking in an

imperfectly comprehended world” (March & Shapira, 1987). Such judgment and

experience about subtle, practical problems are typical of the tacit knowledge that

experienced managers learn to attend to, and may be especially significant in

understanding how managers can best manage risk in software projects.

The present study builds upon and extends the previous research into situational risk

factors taken into account by information system project managers (Barki et al., 1993;

Moynihan, 1996; Schmidt et al., 2001). The studies by Barki et al. and Schmidt et al.

focused solely on identifying software development risks, and not on the potential

strategies or actions that managers can take to address these risks. Moynihan used a

follow-up study (Moynihan, 2000a, 2002) to investigate the strategies used by bespoke

software project managers (i.e. managers of custom-developed software projects carried

out for clients by external software development companies) for addressing these risks,

but noted that he was reporting the managers’ espoused theories about hypothetical

situations, which may have been different from the actions they would take in practice

with real projects. All three of these studies focused primarily on custom software

development rather than software package implementation, and none investigated how

project managers actually conduct risk assessment and risk response planning for their

projects, nor what cues more experienced project managers use when monitoring the

progress of projects to determine whether any of the potential risks will eventuate and to

decide what actions to take.

48

The focus of the present research was threefold. Firstly, the study aimed to learn

about the key risks that experienced IT project managers attended to when working with

packaged software implementation projects and how they actually managed these risks i.e.

the risk management processes they applied in practice. This first aim of the research

addressed the need for empirical studies into actual IT risk management practices in

packaged software implementation projects and the factors influencing project managers

in their choice and application of strategies to manage risk. In particular, experienced

project managers formed the focus of the study, since they were likely to reveal more

about what actually happens in practice, as they have learned from their experience both

about the key risks they need to watch out for, and the most effective means of managing

these risks.

The second aim of the present study was to extend our understanding of the nature

of the knowledge and skills that effective IT project managers apply to the task of

managing risk in implementation projects, by identifying knowledge (tacit or explicit) that

experienced project managers drew upon when planning for and reacting to critical risk

situations that arose in the course of their projects. This improvement in our

understanding of how project managers obtain the knowledge they use when managing

risk in their projects will contribute to the future development of improved training

programs for this aspect of management of IT projects.

Thirdly, the study explored the extent to which Naturalistic Decision Making theory

and the Recognition Primed Decision model could explain the decision processes

occurring in IT risk management practice, and any discrepancies found between the

practice of risk management and the prescription of rational risk management frameworks

recommended by researchers. By viewing the results in the light of both naturalistic and

rational decision making models, the study has enhanced our understanding of

discrepancies between theoretical prescriptions and practical applications of risk

management in IT projects.

Next I discuss the research questions arising from these three aims in more detail.

Research Questions 1 to 3 addressed the first aim of investigating risk management

processes in practice.

49

Research Question 1: What key risks do project managers attend to in software

package implementation projects, and how do these compare with the risk factors

identified in the literature for software packages as well as for custom development?

In Research Question 1, I focused on eliciting the key risk factors that experienced

IT project managers actually attended to in the course of their projects, rather than just

those they say they attend to, based, for example, on their awareness of explicitly reported

risk factor checklists. Thus, while the primary focus of this question was the generation of

risk factors particularly relevant to software implementation projects, the question also

aimed to surface tacit knowledge about the factors that are considered by respondents,

based on their experience, to be particularly relevant and hence worthy of their attention

in their projects.

Research Question 2: What risk management practices do project managers

employ when planning and running software package implementation projects, and how

do these compare with prescriptions from the literature?

For Research Question 2, I explored whether and how experienced IT project

managers actually carry out typically recommended risk management planning practices. In

particular, the study investigated how project managers evaluated and prioritized risks in

their projects, with the aim of answering questions such as: Do project managers actually

use a checklist to go over each project? Do they use a rational decision process to decide

which of the multitude of risks apply, and which are of key importance? Do they calculate

expected risk values based on the probability of the risk and its likely impact?

Research Question 3: What strategies do project managers use to address risks in

IT projects – both to prevent them causing problems in the first place and to deal with

the problems if they do arise (i.e. if the risks identified as potential problems turn into

actual problems)?

In this question I investigated the actions experienced IT project managers take

during the course of their projects. For example, what countermeasures do project

managers employ against key risk factors and how effective were these countermeasures?

How do they choose between possible strategies? What complexities in the project

context are project managers responding to when they utilize particular strategies? The

literature suggests that managers do not regard issues outside their control as risks (March

50

& Shapira, 1987) – they think of risks as things that can be controlled. Yet some of the

highest ranked risk factors in the literature are exactly those issues that are likely to be

outside a project manager’s control – e.g. departmental politics, and problems with

external vendors. Thus this study explored what managers actually do about risks that

might be considered outside their control, if they believed those risks were a significant

threat to the project.

Research Question 4: What knowledge (tacit or explicit) do experienced project

managers use in order to plan for and address critical risk situations that may arise during

the course of software package implementation, and in particular, what environmental and

situational clues do managers attend to when managing their projects?

In Research Question 4 I addressed the second aim of understanding of how project

managers obtain their risk management knowledge. The nature of the knowledge and

skills applied by experienced IT project managers was explored by drawing together the

findings from the first three research questions. In particular tacit knowledge was revealed

through an analysis of the variances identified between actual practices and literature

prescriptions in relation to the previous three questions. Environmental and situational

cues used by respondents to alert them to potential problems in their projects were also

examined.

Research Question 5: What gaps are there between actual practice and the rational

risk management processes as recommended in the IT risk management literature, and to

what extent can decision making theories such as Naturalistic Decision Making explain

such gaps?

This final research question addressed the aim of understanding more about the gaps

identified between practice and theory in IT project risk management, and exploring

whether decision-making theories can throw light on the actual practices of IT project

managers. The results of the investigation of Research Questions 1 to 4 revealed gaps

between the IT risk management theories and frameworks, as discussed in the previous

sections of this chapter, and the actual practice of managers of software package

implementation projects. In exploring Research Question 5, these gaps were examined

and discussed both from the perspective of rational risk management and decision-

making frameworks, and from the perspective of the Naturalistic Decision Making theory

discussed earlier in this Chapter.

51

2.5. Summary

In summary, I have reviewed in this chapter the theoretical foundations of project

risk management, Naturalistic Decision Making, and tacit knowledge that underpin the

research reported in this thesis. I have discussed how the rational assumptions behind the

traditional approaches to risk management in IT projects may not reflect the reality of

managing these projects in practice, and I have suggested that Naturalistic Decision

Making theory, and the Recognition Primed Decision model may more closely model the

processes that actually occur in the risk management of these projects. I have noted that

researchers from two very different fields – management researchers in tacit knowledge,

and the naturalistic decision-making researchers – have developed very closely related

methods of investigating processes actually occurring in practice. These methods form the

basis of the methodology used in this research and discussed in Chapter 3. Finally, I have

discussed the aims of the present research and shown how these aims relate to and build

on previous work. The methodological approach to investigating the research questions

developed above is discussed next, in Chapter 3.

52

C h a p t e r 3

3METHODOLOGY

3.1. Introduction

In Chapter 2, I identified five key research questions about risk management

processes in software package implementation projects. In this chapter, I describe the

methodology adopted to provide data to investigate these questions, which are repeated

below for completeness.

Research Question 1: What key risks do project managers attend to in software

package implementation projects, and how do these compare with the risk factors

identified in the literature for software packages as well as for custom development?

Research Question 2: What risk management practices do project managers employ

when planning software package implementation projects, and how do these compare

with prescriptions in the literature?

Research Question 3: What strategies do project managers use to address risks in IT

projects – both to prevent them causing problems in the first place and to deal with

the problems if they do arise (i.e. if the risks identified as potential problems turn into

actual problems)?

Research Question 4: What knowledge (tacit or explicit) do experienced project

managers use in order to plan for and address critical risk situations that may arise

during the course of software package implementation, and in particular, what

environmental and situational clues do managers attend to when managing their

projects?

Research Question 5: What gaps are there between actual practice and the rational

risk management processes as recommended in the IT risk management literature,

and to what extent can decision making theories such as Naturalistic Decision Making

explain any such gaps?

The study used the critical decision interview technique, discussed below, to

explore the risk management approaches that IT project managers use during software

53

package implementation projects, and to probe more deeply into specific risky situations

to discover possible tacit knowledge applied by the managers, and in particular, the cues

that they observe, and their strategies and bases for decisions.

In Section 3.2 I discuss the research approach for this investigation, and explain

why this particular method was chosen. Section 3.3 sets out the research procedures used.

In Section 3.4 I explain the analysis process in detail. The results of the analysis are

presented in Chapters 4 to 6. I consider ethical issues in Section 3.5, and give a brief

conclusion of this chapter in Section 3.6.

3.2. Research approach

The study was primarily an in-depth, exploratory investigation and identification

of IT project managers’ risk management processes, their tacit knowledge regarding the

risk situations they face during software package implementation projects and the

strategies they use to respond to these situations. I was interested in how managers

actually manage risk, rather than their theories about how risk ought to be managed. In

particular, I wanted to explore research participants’ own interpretations of risky and

potentially challenging situations, the decisions they make relating to these situations, and

the results of any actions they take to address these situations.

The exploratory nature of the research meant that the research methods used had

to meet the need for contextual information, and information about meaning and purpose

ascribed by the key participants to their actions. The context of the situations examined was

of paramount importance in gaining an understanding of the dynamics of the software

package implementation process, and of the ‘triggers’ that a project manager looked for to

provide clues about what strategy to adopt in a particular situation. Investigations of this

type lend themselves to ‘broadly interpretive methods of research’ (Walsham, 1995).

According to Orlikowski and Baroudi’s (1991) seminal article on various research

approaches in information systems, interpretive research is based on the epistemological

belief that understanding about social reality involves interpreting the meanings assigned

to phenomena by those interacting with them. Interpretive research techniques draw on

participants’ own words and experiences and interpretive researchers present their own

interpretations of the interpretations made by the participants. Typically, the intent of

interpretive studies is to gain greater understanding about the phenomena of interest in a

specific context in the belief that this understanding can provide insights that can be used

54

to inform other settings, but generalization in the positivist sense is not sought (Myers &

Avison, 2002; Orlikowski & Baroudi, 1991; Walsham, 1995b). Thus, since the primary aim

of the research was to gain greater understanding of participants’ interpretations and

actions regarding risk management, an interpretive approach was adopted.

There were three major requirements to be addressed in selecting a suitable

research technique, which are discussed next. Firstly, because of the exploratory nature of

the study, the choice of ‘risky situations’ had to be based on the research participants’ own

interpretations of their own experiences, not on my prior judgments as the researcher. In

particular, as Schmidt et al. note (2001), if I provided a set of risks based on my own

research perspective, I might overlook some factors, and might fail to capture some of

what practicing managers actually considered important. In particular, one facet of

individual tacit knowledge is that it is learned by experience and is not commonly known

(Sternberg et al., 2000; Sternberg & Horvath, 1999). Thus any attempt to capture tacit

knowledge, rather than just explicit knowledge, had to draw out the respondents’

definitions of which situations were risky or critical, rather than imposing a definition on

them.

Secondly, the research was focused specifically on exploring what actually happens

in practice, rather than on simply reporting what respondents thought they ought to do in

practice. These actual practices were likely to involve both explicit and tacit knowledge

about the tasks in question. The respondents’ tacit knowledge was of particular interest.

However, tacit knowledge tends to be context-specific and does not transfer well to

related tasks in a different context (Berry & Dienes, 1993). Thus what might appear to be

the same task may be solved quite differently on different occasions because of variations

in the context or the environment. In particular, since the ability to re-frame or re-cast a

problem in the light of its environment and context is a key distinction of expert practical

thinking (Klein et al., 1989; Klemp & McClelland, 1986; Scribner, 1986), the research

technique employed had to aid in surfacing the environmental clues that experts were

observing when formulating the issues or problems related to a particular situation and

when devising solutions or responses to critical risk situations or incidents.

Thirdly, in eliciting the participants’ responses to the situations they had identified,

it was important to recognize that it might be difficult for them to explain or articulate the

tacit aspects of their knowledge and interpretation of their responses and actions (Berry &

Dienes, 1993; Sternberg et al., 2000). Schön (1983) noted that much of managerial

55

behaviour appeared as “informal improvisation” or action based on intuition. If asked to

explain their actions, i.e. to say why they chose a response, rather than to simply describe

what they did and what happened, managers are either unable to explain or else appear to

make up an explanation in an attempt to satisfy the questioner. Thus the managers’

theories of action (in Argyris and Schön's terminology, 1978), may well be different from

their actual practice. Therefore it was important to use a research technique that had the

potential to prompt and assist recall of the underlying tacit knowledge, and thus get

beyond the theories or rationalizations that a person may use to explain his/her actions.

A research method that has been shown to be effective in meeting these

requirements is the critical decision method (DuBois, 2002; Klein et al., 1989). The critical

decision method is an adaptation of the critical incident technique (Flanagan, 1954), and is

very similar to the technique developed and used by Sternberg et al. (Sternberg et al.,

2000) for eliciting tacit knowledge in a number of business related situations. The critical

decision method was used to guide the interview approach in the research for this thesis.

In the next section, I first outline the critical incident technique and then show how the

critical decision method variation can meet the requirements discussed above.

3.2.1. Critical incident technique and critical decision method

A group of researchers identifying tacit knowledge in applied settings (Sternberg

et al., 2000; Sternberg & Horvath, 1999; Sternberg & Wagner, 1986) have developed a

detailed approach to the elicitation of tacit knowledge based around the use of the critical

incident technique, which is a method for obtaining specific, behaviourally focused

descriptions of job performance (Flanagan, 1954). In the critical incident technique,

respondents are asked to provide specific examples, with detailed contextual and

behavioural information, of situations that they consider to be important and relevant to

good or poor performance in the area under question. Thus the critical incident approach

focuses on what actions the respondents took, rather than on why they decided on a certain

action in a specific situation, and so helps to reveal respondents’ actual practice rather

than their theories of action (in Argyris and Schön's terminology, 1978). Sternberg et al.’s

use of the critical incident interview approach is designed to achieve the objective of

surfacing tacit knowledge, by encouraging the respondents to tell ‘stories’ that are

illustrative of good or poor performance in a particular area. By focusing on story telling,

Sternberg et al. argue that respondents are better able to recall specific and relevant details

about the particular context of the story, and to identify actual behaviours, rather than

56

reporting individuals’ theories about their behaviours. Thus it is more likely that the story

will reveal or uncover underlying tacit knowledge, and the emphasis on actual behaviours

will also keep the respondents focused on their actual practice rather than what they think

they ought to have done.

A variation of the critical incident technique, the critical decision method (Klein et

al., 1989), has been developed specifically for knowledge elicitation applications, where the

aim is to reveal aspects of expertise such as the critical cues used in making perceptual and

conceptual discriminations, and the underlying basis for judgment decisions. This

approach has particular application in investigations into non-routine incidents where

expert judgment or decision-making is required (DuBois, 2002) and is especially

appropriate for eliciting tacit knowledge that is not part of the formalized procedures for a

domain. It differs from the more general critical incident technique in that it allows more

cognitive probing to encourage respondents to reflect on their own strategies and bases

for decisions.

The critical decision method meets the requirements discussed in the preceding

section as follows. Firstly, by asking respondents themselves to determine which risk

situations are important and ‘challenging’, the approach ensures that the identification of

critical situations emerged from the data itself rather than being imposed by the

researcher. Secondly, the use of careful follow-up questioning, such as asking for another

similar ‘story’ where the expert did something different, helps to identify context specific

details and environmental clues that may be important factors in aiding the expert to

determine the specific action to take. The critical decision method’s focus on cue usage,

and reflection by respondents on their strategies and possible alternatives, helps to

elaborate the basis for selecting one option rather than another in a given situation. As

Klein et al. note (1989 pg 467), focusing the respondent on why other choices were

rejected can illuminate the real reasons for taking a particular action, and so get beneath

respondents’ theories or rationalizations about their actions, in order to reveal underlying

tacit knowledge.

The critical decision method has been used extensively in research for eliciting

expert knowledge in situations where the experts have difficulty accessing their

knowledge. Hence, the approach is particularly useful in addressing the third requirement

– ability to facilitate verbalization of difficult-to-articulate tacit knowledge, such as

contextual knowledge and judgment of typicalities. One of the key strengths of the critical

57

decision method is its concentration on identifying the detailed context of a given critical

situation, the specific environmental and situational cue usage, and the reflection by

respondents on similarities between a specific situation and other situations in their

experience. By focusing the respondent’s attention on these often overlooked aspects of a

critical decision, the method helps to surface tacit knowledge in these areas.

3.3. Research procedures

The critical decision interview method was used to investigate the research

questions of this thesis. Details of the sample of respondents and the interview method

are discussed below.

3.3.1. Sample selection (organizations and subjects)

Packaged software implementation firms were identified in Hong Kong, the aim

being to identify organizations significantly involved in software package implementation.

Because of the exploratory nature of the research, a purposeful sampling approach was

employed (Miles & Huberman, 1994 pg 28; Patton, 2002 pg 40). Under this approach, the

purpose of sampling is to obtain replicate and contrast cases in order to build an

“information-rich” set of data. The research approach was intensive, and as such placed

limitations on the sample size. Therefore, a sample of 20 IT project managers was

planned, as this was considered sufficient to cover the range and diversity of strategies and

risk management approaches, while still remaining manageable in terms of the resources

and time required for the research.

Initially, organizations and IT project managers were sought as set out in the next

two paragraphs with the aim of gaining insight about risk management in software

package implementation projects. However, as discussed in the subsequent paragraph, I

moved to a more confirmatory approach during the later stages of data collection, and

sought specific confirming and disconfirming cases (Patton, 2002 pg 239) in order to

extend my initial analysis and probe the limits of applicability of current risk management

prescriptions reported in the literature.

In the early stages of data collection, variation of type of organization was sought,

in terms of the range and size of clients served by the firm, and the size and type of

package the firm works with. An initial sample of small and medium enterprise firms

specializing in software package implementation was identified from the Dun and

58

Bradstreet Hong Kong directory (Dun & Bradstreet (HK), 2002), while large software

package firms in Hong Kong were identified on a reputational basis. Web sites of these

potential firms were scrutinized to further check their suitability for this study. Once an

initial contact was been made with a senior project manager within a firm, a snowball

strategy was used to obtain further contacts, both within the firm and in other firms, who

might be able to contribute further confirming or disconfirming information. I offered to

provide a summary of research findings to all participants as an incentive for their

involvement. A copy of this practitioner report is attached in Appendix C.

Within the firms, IT project managers were sought who would be recognized as

proficient in their domain based on their years in the profession, and on the number of

package implementation projects with which they had been involved. While I did not

define ‘expert’ in absolute terms, I assumed that the title of ‘senior project manager’ or

‘senior consultant’ was a good indicator that the respondent was indeed an expert. (Note

that the job titles vary from firm to firm, but these two titles in particular conveyed the

essence of the type of expertise and responsibilities I was looking for.) Such managers are

likely to have developed a certain amount of tacit knowledge based on their experience in

the job (Sternberg et al., 2000). In general, I used a guideline of seeking applicants with a

minimum of 5 years’ experience with at least 5 projects, although in one case a manager

with less experience was interviewed on the basis of his superior’s recommendation.

Since data analysis proceeded concurrently with the data collection, by the time I

had interviewed 14 respondents it had become clear that I needed to broaden the range of

respondents and organizations in order to confirm certain themes that were emerging

from the preliminary data analysis. In particular, using a snowball strategy, I sought

respondents who could provide different perspectives in the following areas:

a) In-house projects: Most of the previous research in the risk management area has

focused on the perspective of project managers of IT departments, either in terms

of in-house development, or as the in-house managers of an outsourcing project for

the client firm (e.g. Schmidt et al., 2001; Sumner, 2000). My initial data analysis

suggested differences between the views of the project managers from vendor

organizations I had interviewed so far and those of the in-house project managers

who had been interviewed or surveyed in previous research. Thus I expanded the

sample to include project managers who were working for in-house IT departments

in both commercial and government organizations, running both in-house and out-

59

sourced projects, in order to seek cases that could confirm or disconfirm these

emerging differences.

b) Projects at the pre-sales stage: Previous research has not distinguished between risk

assessment carried out at the pre-sales stage of vendor-driven projects, and at the

actual implementation stage of these projects. The respondents I had interviewed so

far were typically involved in the implementation stages of the projects, i.e. their

work began after the vendor firm had won the contract for the project. My

preliminary data analysis suggested that these respondents felt significantly

constrained in their ability to manage risk in their projects by the decisions that had

been made by the vendor pre-sales team. Thus I expanded the sample to include

respondents who had experience at the pre-sales stage of vendor-driven projects in

order to explore any differences in their perspective on risk assessment.

c) Projects from the project executive view: Finally, Schmidt et al. (2001) suggested

that the project managers surveyed in their study tended to view risks that were

completely outside their control as low-ranked risks. The respondents that I had

interviewed so far were making distinctions between risks that they could control

and risks that were beyond their control but within the purview of their project

executives. Hence, again, I expanded the sample to include respondents who were

working at the vendor project executive level to attempt to gain more insight into

this distinction.

Since I was interviewing respondents who had had a long career in IT project

management, several of my interviewees were able to talk about their experiences with

projects in two or three different categories. For example, four of the respondents had

worked in both project manager and project executive roles in the course of their careers,

and these respondents described their experiences in projects at both levels. Four

respondents discussed both in-house and vendor driven projects that they had managed

and two of the project executives also discussed their experience with pre-sales work.

Thus I was able to cover the extra areas described above by expanding the total sample to

25 respondents, who discussed a total of 60 projects. By the completion of the 25

interviews, the preliminary data analysis was showing a convergence of themes, with no

new themes emerging, even though the sample included a very wide range of respondents

in terms of type and extent of experience. Thus no further interviews were sought.

60

3.3.2. Respondent details

Table 3.1 summarizes details of the organizations contacted and the number of

respondents from each type of organization.

Table 3.1: Organization details

Type Organizations Respondents No. of Projects

Local ‘boutique’ s/w houses 3 5 10 Mid-range vendor firms (local HK and some mainland China projects)

3 5 8

Large multinational vendor firms (HK branch, Asia-Pacific projects)

4 12 34

In-house department, large commercial firm 1 1 3 In-house department, government organization 1 2 5 TOTAL 12 25 60

Table 3.2 shows the present roles of the respondents, and the number of projects

discussed from each perspective.

Table 3.2: Project management perspectives

Project Management Perspective (Present Role)

No. of Respondents

No. of Projects

Vendor project manager 17 38 Client (in-house) project manager 3 13 Project executive 3 5 Pre-sales consultant 2 4 TOTAL 25 60

The respondents included 20 males and five females; 17 were Hong Kong

Chinese, seven were ‘Western’ (including HK born European, African-American,

Australian, British, Scottish and South African), and one was Canadian Chinese. Ages

ranged from 20-29 through to over 60, with the majority, 88%, falling between 30 and 49

years of age. Four of the respondents finished their formal education at the high school

level, ten had bachelors’ degrees, and 10 had postgraduate qualifications. All of the

respondents had trained or worked abroad for part of their careers, and all had experience

working with team members from a wide variety of cultures.

The project management experience of the respondents ranged from 3 to over 30

years and 5 to over 150 projects. In fact, many of the interviewees, both senior and more

junior, found it difficult to be precise about their years of experience. Typically, the move

into the project manager role was a gradual one, with respondents moving through

61

various team leader and technical leader roles, and perhaps acting as a project manager for

a sub-project of a larger project, or assuming the project manager role mid-way through a

project as a result of staffing changes. Thus the more junior respondents were unsure

about whether to count stages in their careers when they had assumed a project manager

role even though they did not officially hold that title and the senior ones had simply

forgotten exactly when they ceased being a team leader and became a project manager.

Consequently I relied heavily on the recommendations of the senior executives in the

firms to determine whether a particular respondent had the required level of expertise.

All of the respondents interviewed, including the more junior ones, were

identified by the senior executives of their firms as ‘experts’ in the project management

field, and all had experience with at least five major projects, and typically with a number

of smaller projects as well. Two of the senior executives drew a distinction in their

comments about expertise between project management expertise and technical expertise and

made it clear that they were recommending staff who had demonstrated superior project

management expertise, even though their technical expertise was not necessarily

exceptional. Eight project managers were recommended on the basis that they had proven

themselves as ‘rescuers’ of ‘troubled’ projects, i.e. they were typically called in to take over

a project that had run into difficulties and were skilled at enabling these projects to be

completed.

Of the three respondents with less than five years’ experience, one project

executive respondent had only four years’ IT project experience, having moved from

accounting positions earlier in his career, but was strongly recommended by two senior

colleagues from his firm as someone with, in their opinion, a remarkable grasp of project

and risk management issues at the executive level. A project manager respondent with

only three years’ experience was specifically recommended by his manager as someone

who had demonstrated a high level of project and risk management expertise on difficult

projects during those three years. Finally, one pre-sales respondent had four years’

experience and had a reputation in his firm for a high success rate in winning project bids.

3.3.3. Project details

The respondents typically had already identified two or three projects that they

wished to discuss. The first project mentioned by most respondents was usually a

‘nightmare’ project, one where they had experienced major unanticipated difficulties, or a

62

‘troubled’ project that they had been asked to take over and rescue. The subsequent

projects discussed by the respondents were generally more typical examples of the way

their projects ran, with some unexpected problems, but ones which they were able to

manage and control relatively easily. Consequently, some of these subsequent projects

were described only briefly, often highlighting just a few risk areas in order to offer a

contrast to the issues that arose in the first ‘nightmare’ project. No respondent described a

project that was completely abandoned: even the ‘nightmare’ projects were eventually

completed, although typically substantially over time and budget.

The respondents described experiences with a very wide variety of projects,

ranging from small local projects with a team size of 1-5 people, budget of $US20,000,

and duration of 3 months, to very large, complex, multi-national projects with a budget of

millions of US dollars, team size of 70+ people and a duration of more than 2 years. In

total, since some of these projects were drawn from their experiences earlier in their

careers, the respondents discussed 60 projects carried out by 25 different organizations.

The various types of projects discussed are shown in Table 3.3. The clients (i.e.

the customers or end-users) in 46 of these projects (77%) were commercial organizations,

while the remaining 14 projects (23%) were carried out for government organizations.

Forty-seven of the 60 projects (78%) were vendor-driven projects, and 18 of these (30%)

included the involvement of a substantial third party. Thirteen projects (22%) were in-

house projects, and of these, six (10%) involved in-house development of a custom

system, while seven (12%) were outsourced custom development projects. These in-house

outsourced projects provided a contrasting perspective to the vendor-driven projects,

since essentially, all vendor-driven projects are outsourced projects from the client’s

perspective. However, the respondents describing vendor-driven projects were speaking

from the vendor’s perspective, while the respondents describing in-house outsourced

projects were relating their experiences from the client’s perspective.

Table 3.3: Types of projects

Vendor-driven In-house TOTALS Two parties Three parties Custom developed Out-sourced Commercial 26 (43%) 13 (22%) 4 (7%) 3 (5%) 46 (77%) Semi-government 3 (5%) 5 (8%) 2 (3%) 4 (7%) 14 (23%) TOTALS 29 (48%) 18 (30%) 6 (10%) 7 (12%) 60

63

Thirty-nine out of the 60 projects (65%) fell into a broad category of package

implementation project, typically including extensive customization, front-end web

development work, and/or major infrastructure upgrades. Only two of these 39 projects

were described as straightforward package implementations with little or no

customization. Four of the 60 projects (7%) actually involved the development of

packages for later deployment (two of these were joint ventures, one was an in-house

development, and in the fourth the vendor developed a package for the customer to

deploy to its own clients). Fourteen projects (23%) were purely custom development, and

the remaining three projects (5%) involved consulting and Y2K work.

3.3.4. Instrumentation

The exploratory nature of the research determined the degree of structure of the

instrumentation (Miles & Huberman, 1994 pg 35). For studies such as this one, that

require cross-respondent comparison, some standardization was needed in order to form

a basis for comparison. However, findings from initial less structured interviews can help

to refine the protocol for later interviews as the key areas and issues become clearer. This

study put particular emphasis on the need for rich contextual information, and to allow

information to emerge from the data. Hence a looser structure was chosen initially to

allow developing concepts to be well grounded in the data, and to provide complete and

thorough contextual descriptions. This approach put the initial emphasis on construct

validity (Miles & Huberman, 1994 pg 36). As the interviews progressed and key concepts

became clearer, the interview protocol was refined slightly to provide more structure.

Thus the emphasis gradually changed to a more confirmatory approach, with the need to

ensure that responses from interviewees could be compared. In particular, with the final

group of interviewees discussed above in Section 3.3.1, more emphasis was placed on

exploring perspectives of risk management particularly relevant to the respondent’s role,

i.e., as an in-house project manager, in pre-sales, or as a project executive. This latter

approach put greater emphasis on internal validity and generalizability. Thus an initial,

loosely defined, interview protocol was developed using the critical decision method as a

guide, and minor refinements were made during the course of the interviews, as discussed

later in Section 3.3.6. A copy of the interview protocol is attached in Appendix A.

64

3.3.5. Data collection - interviews

Semi-structured interviews were conducted with research participants, following a

set of core procedures based on the critical decision method (DuBois, 2002; Klein et al.,

1989). The interviews lasted approximately one hour. With the consent of the participants,

the interviews were recorded, and additional notes were taken during the interviews, in

order to provide descriptive elaboration of the elicited information. The transcripts of the

interviews were provided to the respondents for their comments and feedback. Where

applicable, any missing minor factual details about the projects discussed, such as size,

duration or approximate budget, were requested. Respondents typically provided such

extra detail immediately. One respondent requested the deletion of a commercially

sensitive reference in his transcript. No other respondents requested any changes, but two

provided, on their own initiative, more detail on standard risk assessment processes

typically followed in their firms, but not mentioned in detail in the projects they chose to

discuss with me. Four respondents requested that their interviews were not tape recorded,

and I took very detailed notes for these interviews.

The interviews consisted of the following six parts, which are discussed in more

detail below: a) introduction and explanation of the process; b) general description of a

specific project the participant had worked on recently, and his/her role in this project; c)

description of the risk management processes applied in the project; d) elicitation of

specific incidents during the course of the project that were risky and challenging and for

which the respondent considered a less experienced person might have made different

decisions; e) exploration of the typicality of this project in the respondent’s experience;

and f) collection of demographic data.

Interview introduction. An outline of the research project and the interview

protocol was provided to respondents by email, as part of the consent package (see

Appendix B), when the interview appointment was confirmed. At the start of the

interview I briefly reviewed the purpose of the research with the respondent – to

understand more about actual risk management practices for software package

implementation projects, and in particular to learn about the subtle practical problems

faced by managers of these projects and the judgments they exercise during the course of

the project. I then explained the interview process, and sought permission to tape record

the interview.

65

Choice of specific project. I asked the respondents to focus on a particular

recent project they were involved in so that they were reporting on events that had

actually occurred rather than talking about their general conceptions of rules and

procedures. This approach has been shown to be more effective in obtaining specific and

accurate information about prior events (Flanagan, 1954; Klein et al., 1989). I encouraged

the interviewees to choose a project where they faced some risky or challenging situations.

Then I obtained a general description of the project, including information about the type

of project, client organization, personnel involved, budget and schedule.

Risk management processes. I asked respondents to describe the risk

management processes, if any, they applied to the project they chose. This account was

allowed to proceed without interruption, except for clarification, in order to create a

context for my understanding of the project from the perspective of the respondents, and

to guard against my own biases and preconceptions. This account also helped to activate

the respondent’s memory of the project as a context for the following questions, and to

establish a rapport (Klein et al., 1989).

Elicitation of specific incidents. I then asked respondents to describe specific

incidents from the project that were challenging from a risk perspective, and which might

have been dealt with differently by a novice manager. This approach has been used in a

number of critical methods studies (Klein et al., 1989; Klemp & McClelland, 1986) and

was chosen to meet the joint aims of both describing the specific risk management

processes occurring during each incident, and identifying tacit knowledge that may

distinguish more experienced managers. If necessary, I asked prompting questions to elicit

situations that illustrate aspects of risk assessment, risk response planning, risk monitoring

or contingent action. I used a structured series of decision point probe questions to elicit

information about the situational cues surrounding the incident, the strategies and options

considered, the factors or triggers that determined one response rather than another, detail

about the action taken and the consequences of the action, and why this situation could

have been difficult for novices. Examples of the probe questions are shown in Table A.1

in Appendix A.

Exploration of typicality. Once respondents had exhausted their recall of key

incidents for the specific project chosen, I asked whether those incidents and the actions

and consequences were typical of other projects in their experience. I used follow-up

questions to identify any key differences between projects identified. Respondents

66

typically chose to discuss another one or two projects to provide contrasting information

that in their view helped to explain possible differences in context and project structure

that may have influenced their risk management procedures in the first project they

discussed.

Collection of demographic data. Demographic information was collected

detailing the respondents’ age group, gender, ethnicity, education and experience. The

scale of projects the respondents typically managed (i.e. duration, effort in terms of

person-months, team size) was also recorded.

3.3.6. Refinement of interview protocol

The initial interview protocol was tested with three IT project managers.

Transcriptions of these first three interviews suggested one major area of change,

discussed in the next paragraph, and some minor gaps and oversights in my interview

technique, especially in one or two of the follow-up areas. I expanded the follow-up

prompts in the protocol to include specific reminders about two areas – what actions the

project managers actually took in response to any risks identified before the project

started, and where and how the project managers had learned their risk management skills.

The gaps were not sufficient to require the exclusion of these interviews from the sample,

and I sought information by email from these project managers to complete the interview

records in these areas.

The major area of change highlighted by these first three interviews was in my

original intention to focus solely on software package implementation projects. It was very

clear that, in the view of these first three respondents, project managers working for

vendor firms worked on a very wide range of projects, and while most of their projects

included package implementation, this was typically embedded in a much larger

infrastructure project involving major hardware, network and operating system upgrades

along with the migration of existing applications and installation of usually highly

customized package applications. Many of the more recent vendor projects included a

high degree of web development work associated with the implementation of, for

example, a supply chain or customer relationship management package. In the view of

these project managers, it was difficult to separate projects into categories such as

software package implementation, custom development, systems integration or

infrastructure, because most projects involved aspects of all of these categories. For

67

example, many projects involving package implementation also included very extensive

hardware installation and upgrade work. These projects were often referred to broadly as

infrastructure projects because they involved a complete revamping of an organization’s

IT infrastructure in conjunction with the move to a major package system. Similarly, most

package implementation projects included extensive customization work, which was often

the major time and cost component of the project.

As a result of these issues raised during the testing of the interview protocol, I

decided to allow the project managers themselves to be the judge of which projects they

wished to discuss. I recognized that providing only loose guidelines to respondents about

the types of projects to discuss would mean that the results would reflect the respondents’

interpretations about which projects would be most useful for my purpose. However, all

of the respondents were aware that my main area of interest was in risk management for

software package implementation, and with the exception of three respondents at the end

of the data collection period who were specifically chosen for their in-house project

management experience, all of the project managers either worked for vendor firms or for

consulting firms that were involved in major package implementation work. Thus I

expected that the majority of projects that my respondents chose to discuss would involve

issues relevant to risk management in IT projects and would fall into a broad category of

IT projects that include aspects of package implementation. In fact, as discussed earlier in

Section 3.3.3, respondents had typically thought of two or three projects involving aspects

of risk management that they thought would be interesting, and 39 (65%) fell into the

broad category of projects involving aspects of software package implementation.

A natural break occurred in the scheduling of interviews after 14 interviews had

been completed and I reviewed the protocol again at this stage against the concepts that

were emerging from initial data analysis, in order to determine a more confirmatory focus

for the remaining interviews. As discussed in Section 3.3.1 on sample selection, I

expanded the sample to include respondents from in-house IT departments, and

respondents with pre-sales and project executive experience. With these new respondents,

while I continued to use the same interview protocol as a framework, I focused

specifically on aspects of risk management from the perspective of the roles they played in

the projects. I also continued to interview more project managers from vendor firms, for

whom I made only minor changes to the protocol. Firstly, I focused more specifically

again on the relationship between any risk analysis completed in the pre-sales stage, often

68

done by the sales team with little or no involvement of the project manager, and the plans

made and actions taken by the project manager once he or she took over the project.

Secondly, I explored in more depth the extent to which problems which arose during the

course of the project might have been anticipated with in-depth risk analysis at the start.

These choices reflect the influence of my own perspective and interpretations on the

direction of the study and clearly shaped the data collected.

3.3.7. Reflections on interview process

Ideally, each interview would have been scheduled so that the transcript and initial

interview summary could be completed before the next interview. This would have

maximized the opportunity for improvement of interview technique, and might have

allowed for an earlier focus on some key issues. However, in practice, in order to

maximize the opportunities for interviews, I had to make appointments at the

convenience of the respondents to fit in with the demands of their own work schedules.

The result was that after the first three interviews, there was a delay during which I was

able to complete the transcripts and make the refinements noted above. This period was

followed by an intense six-week period, during which another eleven interviews were

completed. At the end of this period, I was able to process all the transcripts and

summaries for this set of interviews, which thus formed the group of initial exploratory

interviews. The remaining eleven interviews were spread out over another 3 months.

I processed all transcripts myself, and while this was very time-consuming, it was

also valuable in three ways. Firstly, it made me aware of some shortcomings in my

interview questioning style and provided me with the opportunity to reflect on and

improve my interview technique. Secondly, I found that listening to the interviews again,

and transcribing them enabled me to begin the thought process about the themes and

categories that were emerging, and to examine possible ideas and concepts in a very

preliminary manner as I worked at the transcripts. The transcribing process made me

much more familiar with the content of each interview, and meant that the later work of

closely reading each interview transcript and identifying themes and sections of words that

illustrated various ideas and concepts was much easier. Finally, processing the

transcriptions myself, immediately after the interview, enabled me to enforce a self

discipline to keep on top of the rapidly accumulating data, and also ensured that I quickly

identified any minor omissions that may have occurred during the course of the interview.

For example, in allowing the respondents to tell the story of their project in their own

69

words, I sometimes found that they would omit details such as the overall duration or

team size, as these details were not important from their perspective. However, from my

perspective as researcher, these details are particularly useful in gaining an overall

understanding of the projects, and when comparing different projects. The prompt

processing of the transcripts also enabled me to quickly follow up on any missing

information while the interview was still fresh in the respondent’s mind.

3.4. Analysis and coding processes

All interviews were analyzed with a qualitative content analysis procedure. The

general analysis process followed three key stages (Miles & Huberman, 1994; Wolcott,

1994): description (i.e. summarizing what happens); coding and analysis (i.e. systematically

identifying key factors and relationships); and interpretation (i.e. drawing my conclusions as

a researcher from the data). An overview of these three key stages of the analysis process

is given below. Sections 3.4.1 and 3.4.2 describe the detailed coding carried out for the

various research questions under investigation.

Description: The first descriptive stage of the analysis simply involved

summarizing the transcripts of each respondent on a project-by-project basis. This first

stage was completed as soon as possible after the interview and the transcript had been

completed. Each project described by an interviewee was summarized in a similar way.

Firstly, basic details about the type of project, size and duration were extracted, and a

brief timeline was prepared. Then, any risk management processes applied at the start of

the project were summarized. Detailed coding of risk factors, strategies and constraints

was deferred at this stage, as the aim of this descriptive part of the analysis process was

to gain an overall familiarity with the transcript and the project manager’s description

and explanation of the course of each project. However, a brief memo was created for

each project identifying passages describing any specific problem situations that the

respondent had to address during the course of the project. Analysis of these problem

situations formed the basis for reporting on Research Questions 4 and 5, as described in

Section 3.4.3. Finally, respondents typically gave several more general comments about

their views on risk management issues and strategies to address risks in IT projects, and

about how they learned their risk management skills. These comments were included at

the end of the interview summary.

70

Analysis: A qualitative data analysis software package, NVivo version 2.0, was

used to manage the detailed coding and analysis process. NVivo provides several different

tools to manage and code qualitative data, and after some experimentation, I set up the

structure illustrated in Figure 3.1 and discussed below to manage the research project and

to provide a number of different options for analysis.

Fig. 3.1: NVivo coding structure

It became clear from the summary of the transcripts that I needed to be able to

work with two different units of analysis. The project managers themselves formed one

level of analysis, but I also wanted to be able to compare their responses on a project-by-

project basis. Thus I set up two separate classifications of the transcripts. Firstly, each

transcript was coded in its entirety to its corresponding project manager ‘case’, as shown

in Figure 3.1. This allowed for an analysis of responses by project manager, with

breakdowns according to various project manager attributes, such as experience and

project management role. Secondly, each transcript was formatted into sections to identify

the different projects discussed and these sections were coded to corresponding project

‘cases’, thus providing for cross-project analysis.

Transcript PM 1

Strategies

ProjectCase Group

PM1 Project 1

PM1 Project 2

PM1 General

Transcript Sections

Project stage:At start; during

Risk Factors

Project Manager Case Group

PM 1 PM 2

ProblemSituations

Perspective:PM, executive, presales

Routine, Troubled

Codesapplied

Transcript PM 1

Strategies

ProjectCase Group

PM1 Project 1

PM1 Project 2

PM1 General

Transcript Sections

Project stage:At start; during

Risk Factors

Project Manager Case Group

PM 1 PM 2

ProblemSituations

Perspective:PM, executive, presales

Routine, Troubled

Transcript PM 1 Transcript PM 1

StrategiesStrategies

ProjectCase Group

PM1 Project 1

PM1 Project 2

PM1 General

ProjectCase Group

PM1 Project 1

PM1 Project 2

PM1 General

Transcript Sections Transcript Sections

Project stage:At start; duringProject stage:At start; during

Risk FactorsRisk Factors

Project Manager Case Group

PM 1 PM 2

Project Manager Case Group

PM 1 PM 2

ProblemSituationsProblemSituations

Perspective:PM, executive, presalesPerspective:PM, executive, presales

Routine, TroubledRoutine, Troubled

Codesapplied

71

The transcript summaries also highlighted the need to be able to set up various

cross-case comparisons between different categories of project managers and projects. I

used the ‘attribute’ function in NVivo to capture demographic details for the project

managers, and to record a number of different attributes of the projects, including the

type of client (for example, commercial or government); the type of project from the

respondent’s perspective (for example, vendor-client or in-house); the nature of the

project (for example, customized package, infrastructure including package, or custom);

the respondent’s firm for this project (since some respondents described a project they

had managed for a previous employer) ; the location of the project; and the respondent’s

role (for example project manager, pre-sales manager, project executive, or manager

appointed to ‘rescue’ a ‘troubled’ project).

Once the overall structure of the research project had been created, I focused on

the detailed coding of individual transcripts. Several passes of coding were made in order

to address the various research questions. For each question I began by developing an

overarching coding framework, and then further refined this framework as coding

progressed. I also regularly reviewed the coding to ensure that codes were applied

consistently at different stages of coding. For example, when I had finished coding all

transcripts for a research question, I would select one or two of the documents that I had

coded at the start, and recode them without reference to my original coding, in order to

check whether there had been any change in my application of codes during the period of

coding. As discussed in Section 3.4.1, Research Question 1 required the most intensive

coding and this coding review enabled me to identify any subtle shifts in my interpretation

or application of codes over time. A reconsideration of these codings helped me to

further develop the framework to better reflect the categories that were emerging from

the data. The resulting on-going development of the coding framework meant that I had

to process all the transcripts several times for each research question before I was satisfied

that I had achieved a fully consistent application of the coding.

The partitioning of the transcripts into sections for each project, and for each

respondent’s general comments, enabled me to examine the codes both at the project

manager level and also to identify those factors mentioned in specific projects and those

mentioned only in general comments, as Figure 3.1 illustrates. Even more, by coding the

passages from the project sections according to whether they related to situations at the

start of the project or during the course of the project, I could examine risks and

72

problems that were considered at different stages. This proved particularly useful in the

analysis for Research Question1 on risk factors. The next pass of coding was to identify

strategies used to address the risks already identified and again I could examine these by

project manager or by project, or by project stage. Finally, I broke down the project

sections even further to identify those passages that were related to each risk or problem

situation so that I could examine the decision-making processes in each of these

situations.

Interpretation: The interpretation stage of the analysis process set the

groundwork for the discussion of results reported in subsequent chapters. An underlying

aim of the data analysis process was to understand the domain characteristics as a

framework for interpreting practitioner performance (Potter, Roth, Woods, & Elm, 2000).

Domain characteristics, such as the level of complexity of the projects, the various

demands of key players during the course of the projects, variability between projects, and

complicating factors and limitations particular to each project, provided a framework for

attempting to interpret the expertise, knowledge, strategies and errors of the project

manager practitioners. I considered the situations described by the interviewees both from

a practitioner focus and a domain focus to get complementary perspectives. At this stage,

I drew on the detailed work that I had carried out during the description and analysis

stages in order to consider questions such as 'What complexities in the domain triggered

the choice of particular strategies?’ and ‘What key aspects of the domain limited the choice

of actions?’ Consideration of questions such as these enabled me to develop my own

interpretations of the data against the context of existing theories, particularly in the areas

of risk management and decision making. At this stage of the overall process, I was

particularly concerned with drawing together my own perspectives on the elements of the

risk management process discussed by my respondents, and the key characteristics of the

projects they described. My aim was to use what Patton (2002 pg 55-58) has termed

inductive analysis and creative synthesis, building on a detailed cross-case comparison

between the projects and also between individual project managers, in order to uncover

important patterns, themes, and interrelationships, and to discuss these findings in the

context of existing decision making and risk management theories.

The next three sub-sections describe the detailed coding process undertaken for

the five research questions under investigation.

73

3.4.1. Research Question 1: Key risk factors

An initial framework for the coding of risk categories for Research Question 1

was identified from the literature reviews of risk management, as recommended by Miles

and Huberman (1994). Where no existing categories seemed to fully capture the

respondent’s intent, a further close reading and analysis of the interview transcripts was

used to identify new categories (Ryan & Bernard, 2000). In particular, the list compiled by

Schmidt et al. (2001) of 53 risk factors in 14 different themes was taken as a starting point

since this is the most recent and arguably the most comprehensive list of risk factors in

the literature. This list was supplemented with risk factors not mentioned by the panels of

experts surveyed in the Schmidt study, but identified in previous studies (Barki et al., 1993;

Boehm, 1991; Moynihan, 1996; Sumner, 2000). Schmidt et al.’s list also includes extensive

explanation of their interpretation of each category, which made it easier to determine

whether a particular factor could reasonably be placed into a particular category. Where a

risk factor identified by a project manager did not correspond to any of the items on

Schmidt’s list, I searched other studies for an appropriate category. Where none appeared,

I created a new category. This process resulted in a final list of 98 risk factors, grouped

under 17 different themes.

Initially, I encountered some difficulties in coding these risk factors for two

reasons. Firstly, I found some of the distinctions drawn by previous researchers too fine-

grained to apply consistently. For example, Schmidt et al. (2001) made a distinction

between the factor “Personnel: Lack of required knowledge and skills” and the factor

“Staffing: Insufficient or inappropriate staff”. The personnel factor was defined as a “lack

of required knowledge and skills in the project personnel” while the staffing factor was

defined as “not enough people or people with wrong skills/insufficient skills assigned to

the project”. I found that I had used these codes interchangeably, and frequently had

applied both of them to the same passage of text. Similarly, I found it difficult to

distinguish between the factor “Project Management: Lack of effective project

management methodology”, which was defined as “the team employs no change control,

no project planning, or other necessary skills or processes” and the factor “Project

Management: Poor or non-existent control”, defined as “no sign-offs, no project tracking

methodology, unaware of overall project status, ‘lost in the woods’”. I resolved these

coding problems when I developed a second coding framework, described later in this

section.

74

The second problem I encountered with coding was my own reluctance to add

new categories to an already extensive list. However, I found when reviewing my coding

that many of the factors that seemed wrongly coded were the ones that I had tried to fit

into an existing risk category whose description did not really capture my understanding

of the project manager’s comments. Once I overcame my reluctance to add new

categories, the coding went smoothly. For example, a number of project managers

identified the issue of team morale as a key risk factor, and a key problem that occurred

during the course of their projects. Initially I considered coding this factor into Schmidt et

al.’s risk category “Personnel: Poor team relationships”. The description given by Schmidt

et al. is “Strains existing in the team due to such things as burnout or conflicting egos and

attitudes.” Initially, it seemed to me that this could conceivably encompass strains due to

low morale and motivation, especially when these have been caused by problems in the

progress of the project or by excessive overtime required to meet deadlines. However, my

respondents quite clearly saw the issue of morale and motivation as one quite separate

from issues relating to conflicting egos and attitudes, and they also assigned a very high

level of importance to the need to maintain morale as a risk mitigation strategy, and so I

decided that a separate risk item was needed to capture these differences. My respondents

also made distinctions between risks that related to their own firms and their own vendor

project teams, risks that related to their clients and their client project teams, and risks that

related to any third parties involved in the projects, and I needed to reflect these

distinctions in the coding. In total, I created 45 new factors to more closely reflect the

responses of the project managers that I interviewed. Sixteen of these factors were in the

Relationship Management and Project Management themes, while a further fifteen

specifically related to distinctions between vendor, third party and client risks.

At this stage the risk categories coding framework, comprising 98 different

factors, was so large and extensive that it was becoming unmanageable, and I reviewed all

the codes closely to see why so many were needed. As mentioned above, I had already

encountered difficulties in my own ability to consistently apply some of the finer coding

distinctions made by previous researchers. However, some of the new codes I had added

myself also seemed, on reflection, to be too fine-grained to be useful, and could be easily

amalgamated with closely related codes. Moreover, it had become clear from the code

review that my respondents regarded some risks as inseparable, even though previous

researchers had separated these risks into clearly distinct risk areas. For example, in the

funding and scheduling area, my respondents all regarded these risks as ‘two sides of the

75

same coin’. If a project was under-funded, then that would immediately be reflected in an

inadequate time and effort allowance on the schedule, and if the schedule set unachievable

deadlines, then the budget would automatically also be unachievable because the only way

to overcome the schedule problem was to add more resource, i.e. more staff, to the

project team. Similarly, respondents made little distinction between various aspects of

personnel and staffing problems. Instead they referred to “resources” as a risk area, using

this term to cover the whole broad area of whether sufficient numbers of staff with the

appropriate skill mix were assigned. While previous researchers have made fine

distinctions between risks relating to scope and risks relating to requirements, the project

managers in the present research tended to use the term ‘requirements’ to include issues

related to both overall scope definition and more detailed specific requirements definition.

It also became clear that these respondents, i.e. project managers from vendor firms,

managing vendor-driven projects, clearly distinguished between risks related to their own

firms, risks related to the client and risks that related to any third parties involved in the

projects. This distinction was not well catered for in the original framework derived from

the literature.

As a result of this close review of the risk factors coding framework, I decided to

develop a new framework that would better reflect the themes that were emerging from

the particular set of project managers that I had interviewed. This final coding framework

still drew on the codes I had previously identified from the literature, but now I

condensed and amalgamated some factors and expanded categories in other areas in order

to better match my respondents’ perspectives. I used three overarching sets: vendor risks,

client risks, and third party risks. Within these three sets, I identified a total of 43 risk

factors in five broad themes: relationships, project management, environment, technology,

and solution. The full table of factors used in the final coding framework is shown in

Table 4.1 in Chapter 4.

I then recoded every transcript again, without reference to the original coding

framework. This detailed coding of all transcripts under two different frameworks

provided the opportunity to assess the extent to which codes had been applied

consistently. Although the codes at the most detailed level in each framework were in

some cases quite different, the NVivo software provides the facility to amalgamate codes

at higher levels within a coding hierarchy. The original framework, including the new

themes that I had added, consisted of 17 higher level themes, and typically I could map

76

each of these themes to one or two of the six themes developed in the second framework.

(The External Dependencies theme of the original framework was an exception since it

mapped to third party risks in all of the second framework categories.) I developed a

matrix table in NVivo to identify the intersection of coding references across the two

coding frameworks. I was then able to check the consistency of my coding by examining

the extent to which coding references occurred at the intersections expected from the

mapping of the original framework onto the new framework. I was also able to ‘drill

down’ on intersection points that displayed unexpected co-occurrences of coding to

investigate any anomalies. The majority (81%) of coding intersections were as expected.

Two thirds of the anomalies (12%) were spurious, due to overlapping caused by selection

of slightly larger “chunks” of text during the second coding process. The remaining

anomalies (7%) were all related to the coding problems with the original framework

discussed earlier, and after reviewing each of these I was satisfied that the second

framework had been consistently applied and better captured the intent of my

respondents.

Finally, in order to facilitate analysis of subsequent research questions, I

distinguished up to three sets of risk factors within each project. For all projects, I

identified the factors discussed by the project managers as risks or potential problems

when they first took over the project, and also those factors which became actual

problems during the course of the project. In addition, ten of the projects discussed by

the project managers were ‘troubled’ projects that the project managers had taken over or

‘rescued’ part way through the course of the project. For these ‘troubled’ projects, the

project managers identified both the existing problems that had occurred prior to their

taking over the project and the risks, or potential problems, they identified in their own

risk assessment when they took over. They also discussed actual problems that arose

during their management of the project.

3.4.2. Research Questions 2 and 3: Risk management practices and

strategies

Coding for Research Questions 2 and 3 proceeded much more quickly, both as a

result of the skills I had gained while working on Research Question 1, and because the

emerging categories relating to these questions were much fewer.

77

I used the summaries of risk management processes applied at the start of the

project, prepared in the Description stage of the analysis process, to report on Research

Question 2 (What risk management practices are employed?). This was not always

entirely straightforward, as many of the project managers described a relatively informal

process, and their actual practices were embedded both in their descriptions of the

progress of each project, and in general comments they made about their approach to

project management. Thus, I made my own interpretations in a number of cases to draw

out the underlying processes being applied by the project managers. These summaries of

risk management processes also contributed to reporting on Research Question 5 (Is there

a gap between risk management theory and practice?) as discussed fully in Section 3.4.3.

While the risk management literature provides some prescriptive advice on

appropriate strategies to address risks in IT projects (Research Question 3) most of the

recommended strategies are fairly high-level, for example Barki et al.’s (2001) suggestion

that high risk projects should have higher levels of planning and user participation. Thus

the broad themes gathered from the literature were used to give some guidance and the

transcripts were studied closely to extract all passages that related to specific strategies and

constraints discussed by the respondents. I then reviewed these passages to draw out links

between the various strategies described by the respondent and the nature of the project

and its context.

3.4.3. Research Questions 4 and 5: Tacit knowledge and gaps between

practice and theory

In the Description stage of the analysis process, a brief memo was created for

each project identifying passages describing any specific problem situations that the

respondent had to address during the course of the project, and these passages formed

the basis for reporting on Research Question 4 (What knowledge, tacit or explicit, do

experienced project managers use to address critical risk situations?) and Research

Question 5 (Is there a gap between risk management theory and practice and to what

extent can decision making theories explain any gaps?).

For Research Question 4, I deduced much of the respondents’ tacit knowledge

from an examination of the variances from prescribed practice that had been identified

for Research Questions 1 to 3. I used an iterative process to compare the actual practices

in response to context-specific problems with the commonly accepted and promoted

78

procedures recommended in the academic literature, in order to identify any practices that

seemed to characterize tacit knowledge about aspects of addressing critical risk situations.

In addition, I carefully scrutinized each description of a problem situation to identify any

environmental or situational pointers that managers attended to as they dealt with the

particular problem facing them. These signals, together with contextual detail relating to

the project circumstances, were of particular interest as they are likely to represent key

tacit knowledge that the managers have learned about what to watch for during the

progress of their projects.

Research Question 5 drew together the findings on risk management practices,

strategies, and tacit knowledge identified in the previous four research questions, and I

first reviewed the comparisons of these findings with the risk management processes

recommended in the IT risk management literature. Since some gaps and variances were

revealed, I firstly re-examined the decision processes described by respondents during the

specific problem situations to determine the extent of fit with the rational risk

management process, which proposes that the decision-maker should evaluate all possible

options for the decision concurrently and choose the best one. Under the rational

approach, although a situation assessment (which is the major focus of the RPD approach

discussed in the following paragraph) is important, there is greater focus on the

consideration of all possible options for the decision.

I then examined each problem situation from the perspective of the Recognition

Primed Decision (RPD) model, using a framework based on the RPD model described

in Chapter 2 and the responses to the decision point probe questions outlined in Table

A.1 in Appendix A. The analysis followed the approach described by Klein (Klein et al.,

1989), and is discussed in detail next.

Under the RPD model of decision-making, the major focus of the decision-

maker’s attention is situation assessment. Recognition of the situation leads the decision-

maker to a set of goals, cues, expectancies and actions. Decisions about what action to

take are made using serial evaluation of options rather than concurrent evaluation. That is,

options for the decision are evaluated one at a time, or serially, and as soon as a suitable

option is identified no further options are considered. In addition, the RPD model

suggests that decision-makers spend more time on assessment of the situation than on

evaluation of options. In order to investigate the extent of the applicability of the RPD

model, I scrutinized the specific incidents described by interviewees for evidence of goals,

79

cues, expectancies and courses of action, and I assessed the decision points described in

these incidents to determine whether concurrent or serial evaluation was primarily used,

and whether the deliberation was about the situation (situation assessment) or about the

action (option evaluation).

3.5. Researcher’s perspective

As noted in Section 3.2 above, the present study was an interpretive and

exploratory investigation and analysis of IT project managers’ risk management processes

and their tacit knowledge about how to respond to particular problem situations that arise

during the course of their projects. It is particularly important to be clear about the

perspective from which the data were examined, as my perspective as researcher and my

own personal background clearly influence the interpretations that I have drawn from my

investigation. My focus on the projects discussed was that they were not only structures

for implementing new information systems within a client organization, but also

structures for communication, collaboration and social interaction between the players

from all parties involved i.e. client, vendor, and in some cases other third parties. My own

experience as a client project manager was helpful in establishing a rapport with my

respondents and in enabling me to see things from their perspective. However, I was

aware that my experience could lead me to make unwarranted assumptions about the

issues being discussed by respondents. Thus I tried to maintain sensitivity to my

respondents’ views, using the probing questions described in the interview protocol as a

guide to remind me to seek out and confirm details that might contradict my own

assumptions. I found the critical decision method of focusing on a specific project, and

encouraging the respondent to ‘tell the story’ of the project, particularly helpful in

ensuring that I set aside my own preconceptions about how projects are generally run.

In order to minimize the extent to which my own initial biases might influence my

interpretations, I found it useful to contrast the two major decision making theories

(rational and naturalistic) discussed in Chapter 2, and move iteratively between them,

examining the results first in the light of one theory, and then from the second

perspective. This theoretical triangulation (Patton, 2002 pg 562) ensured that I maintained

a balanced approach using contrasting theoretical perspectives when drawing out

interpretations of the situations described by respondents, and helped to uncover any

implicit assumptions I had made about the data I was interpreting. I was also able to use

the contrasting perspectives of different types of respondent as a more concrete form of

80

triangulation. For example, the eight outsourced projects described from the perspective

of the in-house client project manager provided a useful comparison with the descriptions

of vendor projects carried out for clients, and described from viewpoint of the vendor

project managers. In addition, I used a meticulous approach, described in Section 3.4.1,

to ensuring coding consistency in order to reduce any systematic biases introduced by my

own perspective.

It is from this basis that I have attempted to interpret my findings in light of two

current risk management and decision making theories, in order to develop understanding

and gain insight into any divergence between these theories and the practices described by

respondents. Following the example of recent qualitative researchers (Davidson, 2002;

Nelson, Nadkarni, Narayan, & Ghods, 2000; Trauth & Jessup, 2000; Walsham & Sahay,

1999) I present, in Chapter 7, a self-evaluation of my research against criteria appropriate

for this type of study.

3.6. Ethical considerations

Confidentiality, by way of disguise of the organizations and participants’ names,

was offered to all participating organizations and interviewees. All participants were

assured that their participation was voluntary, and that they could withdraw from the

study at any time they chose, without penalty. The organizations, and participants within

them, were fully informed, via a consent package, (see Appendix B) of the nature of the

research, and what their involvement in it entailed for them, prior to commencement of

the study, and were offered copies of a final summary report.

3.7. Summary

In Chapter 3, I have set out in detail the methodology used to investigate the five

research questions developed in Chapter 2. I have discussed the chosen research

approach, the critical decision interview method, and shown why it is an appropriate

method for the investigation of the research questions. I have described the research

procedures followed, and discussed the sample selection, instrumentation, and data

collection. I have explained the analysis and coding approach in detail, and reviewed my

perspective as researcher and the steps I have taken to minimize my biases.

The project managers interviewed and the projects they described met the original

aims of providing an “information-rich” set of data, covering a very wide range, in terms

81

of size and type of project management organization, size and type of client, and type of

project work being undertaken. This wide variety of project experiences provided the

opportunity to examine strategies and risk management approaches in a number of very

diverse contexts.

In the following chapters (4 to 6), I discuss the results obtained from this rich data

set for the research questions under investigation. In these discussions, I have retained

confidentiality of the respondents and their projects. However, when providing

quotations from transcripts, I have indicated the project manager concerned with a letter,

from A to Y, and the particular project discussed by a project manager is identified with

the project manager’s letter followed by a number. Thus, for example a quotation from

the second project discussed by project manager K is labelled [K2], while a quotation

from respondent Y’s general comments is simply labelled [Y]. Conclusions and

implications drawn from these results are presented in Chapter 7.

82

C h a p t e r 4

4RESULTS AND DISCUSSION OF RESEARCH QUESTION 1: KEY RISK FACTORS

4.1. Introduction

Research Question 1 asked: What key risks do project managers attend to in

software package implementation projects, and how do these compare with the risk

factors identified in the literature for software packages as well as for custom

development? Earlier in Section 3.4.1, I described the coding process adopted to

investigate this question. A risk factor framework was drawn initially from codes identified

from the literature and was further developed with amalgamation of some factors, and

expansions in other areas in order to closely reflect the themes and categories that were

emerging from the particular set of respondents that I had interviewed. This risk factor

framework was used to code the interview transcripts. In this chapter, I report on and

discuss the analysis of that coding.

In Section 4.2, I describe the risk factors identified in the present study, and report

the numbers of respondents identifying each factor. These risk factors are contrasted with

lists of factors from previous research in Section 4.3, where I identify risk factors from

previous literature not mentioned by the present respondents (Section 4.3.1), and new risk

factors identified in the current study (Section 4.3.2). I discuss the results of this

comparison with previous research in Section 4.3.3. Next, in Section 4.4, I present a

ranking of the risks mentioned by study respondents, as measured by the number of

respondents identifying each factor. I compare the rankings of importance of risks

reported in previous literature with the frequencies that risks were mentioned in the

present study (Section 4.4.1), and discuss this comparison in Section 4.4.2. Finally, in

Section 4.5, I examine risks identified at different stages of the projects, and compare

potential problems identified at the start of each project with the actual problems that

arose in that project (Section 4.5.1). In Section 4.5.2, I look at those problems that

occurred unexpectedly in projects, and in Section 4.5.3 I investigate the key problems that

arose in ‘troubled’ projects. I present a brief summary of the results and discussion in

Section 4.6.

83

4.2. Risk factors

I used three overarching sets of factors: vendor risks, client risks, and third party

risks. Within these three sets, I identified a total of 43 risk factors in five broad themes:

relationships, project management, environment, technology, and solution. The risk

factors identified from the interviews with respondents are summarized in Table 4.1.

Table 4.1: Summarized risk factors

Set

Theme Vendor Third Party Client

Relationships - Team morale - Internal negotiations - Top management

support

- Cooperation - Expectation - Trust - Top management

support - Users - IT department - Bad news

Project Management - Staffing resources - Change management - Schedule & budget - Documentation

- Staffing resources - Deliverables control

- Staffing resources - Sign-off control - Readiness - Project management

Environment - Reputation - Overseas head office - Competition - Legal & credit risk - Contract terms &

conditions

- Non-local 3rd party - Multiple 3rd parties

- Multiple sites/countries

- Organization culture - Multiple departments - Business changes

Technology - Customization - Newness - Complexity

- Integration & compatibility

- Data conversion - Technical environment

Solution - Development choice - Requirements

- Deliverables - Requirements - Functionality

The number of project managers identifying each risk, either in a project or in

their general comments, is shown in Table 4.2 next, which also includes detailed

descriptions of each factor in the coding framework. Although, traditionally, risks have

been considered to be potential problems that could impact adversely on the project, the

vendor project managers in the present study approached risk assessment and risk

management with a broader focus, in that they also considered the risks to their own firms

that could arise from poor performance on the project or from difficulties with the client.

This broader perspective is reflected in the risks categorised in the Environment-Vendor

set in Table 4.2, and is discussed further in Section 4.3.2 below.

84

Table 4.2: Number of respondents (out of 25) identifying risk factors

Theme Set Factor Description No. Relationships Vendor Team morale Risk of deteriorating team morale affecting project performance 13 Internal negotiations Risks associated with internal competition for staff; and with need to influence

other internal teams and other sections of the vendor firm e.g. for customization of package requirements

5

Top management support Risks associated with getting necessary support from vendor top management for increased schedule/budget; for getting support to influence clients and third parties

5

Third Party Cooperation Risks associated with getting required level of cooperation from third parties 6 Client Expectations Risks associated with addressing expectations of clients, including expectations

set during pre-sales; expectations about client contributions to project work; expectations about deliverables

18

Trust Risks associated with deterioration in or lack of trust between client and vendor. Sometimes related to difficult customers

14

Top management support Risks associated with lack of, or unwillingness of, client top management to support and drive through change, and manage internal conflict, within the client organization

7

Users Risks associated with user skills - too much or too little; users' willingness to participate

8

IT department Risks associated with client IT department skills - too much or too little; Client IT department's willingness to participate

7

‘Bad news’ Risks associated with unwillingness of client users and IT department to pass bad news upwards. Includes reluctance to keep client senior management informed, especially where problems could relate to client staff's own poor performance

10

85

Table 4.2: Number of respondents (out of 25) identifying risk factors continued.

Theme Set Factor Description No.

Project Management Vendor Staffing All problems with vendor staffing – not enough, wrong skills, staff turnover 17

Change management Risks for vendor associated with managing client change requests, both managing too tightly (hence alienating clients) and managing too loosely (hence losing control)

18

Schedule and budget management Risks associated with keeping to preset schedule and budget, including failure to develop work breakdown structure; failure to adequately monitor progress on tasks; and blow-outs caused by unexpected problems, beyond the pre-set contingencies

22

Documentation management Risks associated with vendor failure to adequately document project meetings, sign-offs, agreed changes and 'trade-offs' (add a new item in exchange for deleting unrequired item). Includes risks from over-aggressively using documentation to control project

12

Third Party Staffing All problems with third party staffing – not enough, wrong skills, staff turnover 4

Deliverables control Risks associated with failure of third parties to meet deliverables deadlines, or to meet quality requirements. Includes risks such as failure to ensure third parties develop adequate work breakdown structure and failure to monitor third party progress

7

Client Staffing All problems with client staffing – not enough, wrong skills, staff turnover 5

Sign-off control Risks associated with reluctance or failure of client to adequately test and sign-off on deliverables i.e. client fails to meet its deadlines in terms of testing when required and then signing off on satisfactory test completion. Can be a deliberate ploy on part of client to delay payment, or as a result of fear of accepting in case something turns up later on

7

Readiness Risks associated with client's readiness to implement. Includes site preparation, user training, client's ability to meet their deadlines on e.g. data conversion etc

5

Project management Risks associated with client's internal project management structure, including failure to appoint client project manager, failure to give client project manager adequate authority over client staff, incompetence of client project manager

7

86

Table 4.2: Number of respondents (out of 25) identifying risk factors continued.

Theme Set Factor Description No.

Environment Vendor Reputation Risks associated with possible damage to vendor reputation as a result of project

outcomes such as bad client relationships; bad (over time, over budget, poor quality) project performance; or 'walking away' from difficult projects

8

Overseas head office For large multi-national vendor companies, risks associated with differences between non-local HQ and local branch in terms of what should be done, or how it should be done

4

Competition Risks associated with external competition and its consequences, particularly the impacts on the pre-sales team such as over-selling and under-bidding to get projects

14

Legal & credit risk Risks associated with possible financial damage to vendor as a result of 'walking away' from difficult projects, or as a result of client financial instability, or refusal to make progress payments

12

Contract terms & conditions Pre-sales has agreed to a contract with harsh terms and conditions, in the client's favour. There is little room for negotiation. Often typical of government contracts

11

Third Party Non-local third party Risks associated with dealing with a non-local third party; or with the local branch of an overseas based third party

9

Multiple third parties Risks associated with the need to co-ordinate and integrate several different third parties

5

Client Multiple sites; sites in multiple countries Risks associated with delivering projects across multiple sites. Risk is compounded if sites are spread across several countries. Includes risks associated with country variations

13

Organization culture Risks associated with differing organizational cultures. Includes issues related to e.g. dealing with government versus commercial projects, and with differences in country culture

9

Multiple departments Risks associated with delivering projects across multiple departments within one site, including interdepartmental conflict, and issues relating to unwillingness of any one department to accept responsibility

6

Business changes Risks associated with financial viability of client, and with possible major business changes such as takeovers

6

87

Table 4.2: Number of respondents (out of 25) identifying risk factors continued.

Theme Set Factor Description No. Technology Vendor Customization of product Risks associated with degree of required customization (from none to complete

development). Includes risks associated with issues of package integrity across other clients if major changes are required, and issues of on-going support

10

Newness of product Risks associated with newness of package, and associated bugs 13 Complexity of product Risks associated with the number of different modules or functionalities to be

implemented - the greater the complexity the greater the risk 9

Third Party Integration & compatibility Risks associated with integration and compatibility of third party products both with vendor product, client technology, and other third party products

4

Client Data conversion Risks associated with data conversion from client legacy systems 9 Technical environment Risks associated with client technical environment, including particular

combinations of h/w and o/s; availability of testing environment and specific technical requirements set by client IT department

7

Solution Vendor Development choice Presales commits to wrong development strategy e.g. package instead of custom,

or over-sells modules not really required, or fails to identify modules that are required, or fails to identify the degree of customization required

3

Understanding of requirements Vendor has inadequate grasp of requirements, e.g. specification is at too high a level, or unclear, or has omissions or is based on unfounded assumptions

14

Third Party Deliverables Risks associated with the third party's ability to deliver what it has contracted to deliver

10

Client Understanding of requirements Clients are unclear as to what their requirements are or have significant internal differences about requirements

17

Required functionality Risks associated with the match between product and client's requirements and degree of change required from client to utilize the provided functionality

7

88

4.3. Comparison of risk factors with previous research

The initial framework was derived from risk factors identified in the literature,

primarily from the list compiled by Schmidt et al. (2001), and so there is substantial

commonality between the set of factors shown in Table 4.2 and Schmidt’s list. Table 4.3

presents a comparison of the factors used in the present study, and factors identified in

three previous studies (Moynihan, 1996; Schmidt et al., 2001; Sumner, 2000). Schmidt et

al.’s study is the most recent and the comprehensive, and included comparisons against

previous lists developed by Boehm (1991), Barki et al. (1993) and Moynihan, and so I

have used the Schmidt list as the benchmark for comparison purposes. However, I have

included Moynihan’s list because four of his factors appear to have no corresponding

factor in the Schmidt list, and because his respondents were all working for external

clients rather than on in-house projects, and so are likely to have a similar perspective to

the respondents from vendor firms in the present study. I have also included Sumner’s

list, which was not referred to in Schmidt et al.’s study, because Sumner’s list is the only

study to investigate risk factors in one category of package implementation project,

namely ERP projects and the factors she identified may be very pertinent here.

There are many small variations between studies in definitions of factors, and so I

have reported factors that seem, based on a close reading of the descriptions presented in

each study, to be the closest match to the definitions I have developed in the present

study. Some factors from previous studies have been broken down into several different

factors in the present study, and these previous study factors are identified with the

addition of ‘(part)’. Some factors from previous studies only encapsulate part of the

definition of the present study factor, and these cases the previous study factor is shown

with the addition of ‘(partial match)’. Those factors identified in the present study that

appear to have no match or only a partial match with any previous factor are shown in

bold. Those factors that do not have a match with any factor from the Schmidt list, but

which can be matched with a factor from the Moynihan or Sumner lists are shown in

italics.

89

Table 4.3: Comparison of Risk Factors Identified in Present and Previous Studies

Present study Schmidt et al. (2001) Moynihan (1996) [M]/ Sumner (2000) [S] RELATIONSHIPS Vendor Team morale: Risk of deteriorating team morale affecting project

performance

Internal negotiations: Risks associated with internal competition for staff; and with need to influence other internal teams and other sections of the vendor firm e.g. for customization of package requirements

Top management support: Risks associated with getting necessary support from vendor top management for increased schedule/budget; for getting support to influence clients and third parties

Lack of top management commitment (partial match but doesn’t distinguish vendor aspect)

3rd Party Cooperation: Risks associated with getting required level of

cooperation from third parties Improper definition of roles and responsibilities Lack of control over consultants, vendors and sub-contractors (part)

Client Expectations: Risks associated with addressing expectations of

clients, including expectations set during pre-sales; expectations about client contributions to project work; expectations about deliverables

Failure to manage end-user expectations Ineffective communications [S]

Trust: Risks associated with deterioration in or lack of trust between client and vendor. Sometimes related to difficult customers

Top management support: Risks associated with lack of, or unwillingness of, client top management to support and drive through change, and manage internal conflict, within the client organization

Lack of top management commitment Lack of client responsibility, ownership and buy-in

Existence of project sponsor [M] Level of client enthusiasm (part) [M] Lack of senior management support [S] Lack of a champion [S]

Users: Risks associated with user skills - too much or too little; users' willingness to participate

Failure to gain user commitment Lack of adequate user involvement Lack of cooperation from users (part) Growing sophistication of users Lack of appropriate user experience (part)

IT competence of users [M] Working through users vs. working through IT department (part) [M] Level of client enthusiasm (part) [M] Lack of sensitivity to user resistance [S]

90

Table 4.3: Comparison of Risk Factors Identified in Present and Previous Studies continued.

Present study Schmidt et al. (2001) Moynihan (1996) [M]/ Sumner (2000) [S] RELATIONSHIPS Client IT department: Risks associated with client IT department skills - too

much or too little; Client IT department's willingness to participate Working through users vs. working through IT

department (part) [M] Insufficient training and re-skilling (of IT department staff) [S]

‘Bad news’: Risks associated with unwillingness of client users and IT department to pass bad news upwards. Includes reluctance to keep client senior management informed, especially where problems could relate to client staff's own poor performance

PROJECT MANAGEMENT Vendor Staffing: All problems with vendor staffing – not enough, wrong

skills, staff turnover Lack of effective project management skills (part) Lack of required knowledge/skills (part) Lack of “people skills” in project manager Poor team relationships Insufficient/ inappropriate staffing Staffing volatility Lack of available skilled personnel

Lack of business analysts with business and technology knowledge [S] Lack of ability to recruit and retain qualified ERP systems developers [S]

Change management: Risks for vendor associated with managing client change requests, both managing too tightly (hence alienating clients) and managing too loosely (hence losing control)

Not managing change properly Lack of effective project management methodology (part) Changing scope/objectives Lack of frozen requirements

Schedule and budget management: Risks associated with keeping to preset schedule and budget, including failure to develop work breakdown structure; failure to adequately monitor progress on tasks; and blow-outs caused by unexpected problems, beyond the pre-set contingencies

Lack of effective project management methodology (part) Poor or nonexistent control (part) Under-funding of development Bad estimation Artificial deadlines Poor risk management Lack of effective development process

91

Table 4.3: Comparison of Risk Factors Identified in Present and Previous Studies continued.

Present study Schmidt et al. (2001) Moynihan (1996) [M]/ Sumner (2000) [S] PROJECT MANAGEMENT Vendor Documentation management: Risks associated with vendor failure

to adequately document project meetings, sign-offs, agreed changes and 'trade-offs' (add a new item in exchange for deleting unrequired item). Includes risks from over-aggressively using documentation to control project

Lack of effective project management skills (part) Lack of effective project management methodology (part) Poor or nonexistent control (part)

3rd Party Staffing: All problems with third party staffing – not enough, wrong

skills, staff turnover Excessive use of outside consultants Failure to mix internal and external expertise effectively

[S] Deliverables control: Risks associated with failure of third parties to

meet deliverables deadlines, or to meet quality requirements. Includes risks such as failure to ensure third parties develop adequate work breakdown structure and failure to monitor third party progress

Poor or non-existent control (part) Lack of control over consultants, vendors and sub-contractors (part)

Client Staffing: All problems with client staffing – not enough, wrong skills,

staff turnover Lack of appropriate user experience (part) Insufficient internal expertise [S]

Sign-off control: Risks associated with reluctance or failure of client to adequately test and sign-off on deliverables i.e. client fails to meet its deadlines in terms of testing when required and then signing off on satisfactory test completion. Can be a deliberate ploy on part of client to delay payment, or as a result of fear of accepting in case something turns up later on

Lack of cooperation from users (part) Poor or non-existent control (part)

Readiness: Risks associated with client's readiness to implement. Includes site preparation, user training, client's ability to meet their deadlines on e.g. data conversion etc

Client willingness/capability to handle implementation [M] Insufficient training of end-users [S]

Project management: Risks associated with client's internal project management structure, including failure to appoint client project manager, failure to give client project manager adequate authority over client staff, incompetence of client project manager

Lack of proper (client) project management [S] Lack of full-time commitment to project management and project activities [S]

92

Table 4.3: Comparison of Risk Factors Identified in Present and Previous Studies continued.

Present study Schmidt et al. (2001) Moynihan (1996) [M]/ Sumner (2000) [S] ENVIRONMENT Vendor Reputation: Risks associated with possible damage to vendor reputation

as a result of project outcomes such as bad client relationships; bad (over time, over budget, poor quality) project performance; or 'walking away' from difficult projects

Overseas head office: For large multi-national vendor companies, risks associated with differences between non-local HQ and local branch in terms of what should be done, or how it should be done

Competition: Risks associated with external competition and its consequences, particularly the impacts on the pre-sales team such as over-selling and under-bidding to get projects

Legal and credit risk: Risks associated with possible financial damage to vendor as a result of 'walking away' from difficult projects, or as a result of client financial instability, or refusal to make progress payments

Contract terms and conditions: Pre-sales has agreed to a contract with harsh terms and conditions, in the client's favour. There is little room for negotiation. Often typical of government contracts

Third Party Non-local third party: Risks associated with dealing with a non-local

third party; or with the local branch of an overseas based third party

Multiple third parties: Risks associated with the need to co-ordinate and integrate several different third parties

Managing multiple relationships with stakeholders Multi-vendor projects

Co-ordination complexity of the project [M] Main source of control of project (developer vs. client vs. third parties) [M]

Client Multiple sites; sites in multiple countries: Risks associated with

delivering projects across multiple sites. Risk is compounded if sites are spread across several countries. Includes risks associated with country variations

Knowledge of country/culture/ language (partial match) [M]

Organization culture: Risks associated with differing organizational cultures, reflecting ‘the way we do things’. Includes issues related to e.g. dealing with government versus commercial projects, and with differences in country culture

93

Table 4.3: Comparison of Risk Factors Identified in Present and Previous Studies continued.

Present study Schmidt et al. (2001) Moynihan (1996) [M]/ Sumner (2000) [S] ENVIRONMENT Client Multiple departments: Risks associated with delivering projects

across multiple departments within one site, including interdepartmental conflict, and issues relating to unwillingness of any one department to accept responsibility

Conflict between user departments Failure to identify all stakeholders Number of organizational units involved

Need to satisfy multiple groups of users [M]

Business changes: Risks associated with financial viability of client, and with possible major business changes such as takeover

Climate of change in the business creates instability Unstable corporate environment Change in ownership of senior management

Stability of client’s business environment [M]

TECHNOLOGY Vendor Customization of product: Risks associated with degree of required

customization (from none to complete development). Includes risks associated with issues of package integrity across other clients if major changes are required, and issues of on-going support

Failure to adhere to standardized specifications which the software supports [S]

Newness of product: Risks associated with newness of package, and associated bugs

Introduction of new technology Previous experience of application [M] Maturity of technology [M]

Complexity of product: Risks associated with the number of different modules or functionalities to be implemented - the greater the complexity the greater the risk

Logical complexity of application [M]

3rd Party Integration and compatibility: Risks associated with integration and

compatibility of third party products both with vendor product, client technology, and other third party products

Need to integrate/interface with other systems (part) [M]

Client Data conversion: Risks associated with data conversion from client

legacy systems Need to integrate/interface with other systems (part) [M]

Failure to follow an enterprise-wide design which supports data integration [S]

Technical environment: Risks associated with client technical environment, including particular combinations of h/w and o/s; availability of testing environment and specific technical requirements set by client IT department

Stability of technical architecture Familiarity with platform/environment/methods [M] Freedom of choice of platform/development environment [M] Attempting to build bridges to legacy applications [S]

94

Table 4.3: Comparison of Risk Factors Identified in Present and Previous Studies continued.

Present study Schmidt et al. (2001) Moynihan (1996) [M]/ Sumner (2000) [S] SOLUTION Vendor Development choice: Presales commits to wrong development

strategy e.g. package instead of custom, or over-sells modules not really required, or fails to identify modules that are required, or fails to identify the degree of customization required

Choosing the wrong development strategy Lack of effective development methodology Trying new development methodology

Ease of validation of solution (e.g. prototyping) [M]

Understanding of requirements: Vendor has inadequate grasp of requirements, e.g. specification is at too high a level, or unclear, or has omissions or is based on unfounded assumptions

Scope creep (due to not understanding true work effort required) Misunderstanding the requirements New and/or unfamiliar subject matter for vendor Lack of required knowledge/skills of business

Knowledge of client’s business sector [M]

3rd Party Deliverables: Risks associated with the third party's ability to deliver

what it has contracted to deliver External dependencies not met

Client Understanding of requirements: Clients are unclear as to what their

requirements are or have significant internal differences about requirements

Unclear/misunderstood scope/objectives (due to client ‘fuzziness’) New and/or unfamiliar subject matter for client

Client’s knowledge of requirements [M] Failure to emphasize reporting [S]

Required functionality: Risks associated with the match between product and client's requirements and degree of change required from client to utilize the provided functionality

Mismatch between company culture and required business process changes

Level of change to be experienced by user [M] Failure to redesign business processes [S]

FACTORS NOT IDENTIFIED IN PRESENT STUDY Projects intended to fail Failure to get project plan approval Project not based on sound business case Under-funding of maintenance “All or nothing” budget, leading to under-funding in later years “Pre-emption” of project by higher priority project No planning or inadequate planning

Criticality/reversibility of the roll-out [M]

95

4.3.1. Risk factors in previous literature not mentioned in this study

The risk factors identified in the present study include most of the factors identified

in previous studies with the exceptions of seven factors from the list developed by Schmidt

et al. (2001), and one factor from Moynihan’s (1996) list. The seven risk factors noted in

Schmidt’s study and not mentioned by respondents in this study are:

− Projects intended to fail;

− Failure to get project plan approval;

− Project not based on sound business case;

− Under-funding of maintenance;

− “All or nothing” budget leading to under-funding in later years;

− “Pre-emption” of project by higher priority project;

− No planning or inadequate planning.

Schmidt et al. set out to derive a fully comprehensive set of risks, while in the present

study I was seeking to identify those risks that actually caught project managers’ attention,

rather than asking respondents to list every single risk they could think of. Thus it is to be

expected that some of the Schmidt factors were not mentioned by present study

respondents. However, none of these seven risks featured in the 29 factors reported by

Schmidt et al. as being highly ranked by any of their three different ranking panels, so it is not

surprising that these factors were not among those identified by present respondents as being

significant risks in their projects.

The one factor mentioned in Moynihan’s study and not in the present study was

criticality/reversibility of roll-out of new system. This factor also appears to be missing from Schmidt

et al.’s list. It is somewhat surprising that this factor was not identified in the present study

since, as noted above, Moynihan’s respondents might be expected to have a similar

perspective to the project managers interviewed in the present research. However,

Moynihan’s respondents were all reporting software-intensive application development projects,

which deal with systems that are essentially untested in a live environment until

implementation, while most of the present study’s respondents were reporting projects with a

software package focus, where major issues of functionality are likely to have already been

tested on beta release sites. Thus the present study respondents may have seen less need to

96

be concerned about being able to reverse the roll-out of the new system, since the type of

major problems that would require such a drastic action should have already been addressed

in testing at beta release sites.

4.3.2. New risk factors identified in this study

There are 20 factors, discussed in detail below, which seem to have no or only partial

counterparts among factors reported in the Schmidt et al. (2001) list. Seven of these factors

can be matched fully or partially with factors on the Moynihan (1996) or Sumner (2000) lists.

These 20 factors are particularly interesting in view of the comprehensive nature of Schmidt

et al.’s study. The most noticeable gap is in the area of the distinctions drawn by the present

study respondents between risks associated with their client, risks associated with their own

firm, and risks associated with third parties. This distinction has led to subtle differences in

emphasis in a number of factors.

a) Environment factors: The largest subset of risks not previously mentioned relates to

the environment in which the project managers are operating, with five factors related to

vendor environment, one to third party environment, and two to client environment (one of

which has a partial match with a Moynihan factor).

Four of the vendor environment factors, risks to the vendor firm’s reputation, risks from the

vendor’s competitors, legal and credit risk, and risks from contract terms and conditions, are clearly specific

to vendor organizations operating for profit in a commercial environment, and indicate that

the vendor project managers in the present study had a wider focus when considering risk

than simply the risks that could impact on the successful outcome of the project. These

vendor managers also had to consider the possible risks for their own firms from

involvement in the project. Other previous studies typically have focused on software

development projects (Barki et al., 1993; Boehm, 1991; Schmidt et al., 2001), and while these

studies have not explicitly distinguished between in-house and vendor-driven projects, it

seems likely that most of their respondents were in fact involved in in-house development

since they were drawn from major corporations and companies, and not specifically from IT

vendor firms, as in the present study. It is perhaps not surprising that these earlier studies did

not identify these vendor-related risks since, with the exception of risks from contract terms and

conditions, they were not mentioned in the in-house projects of the present study. However,

managers of three in-house out-sourced projects did express concern about possible risks to

97

the client relating to contract terms and conditions if these were not carefully negotiated with

the vendor. Moynihan’s (1996) research did involve projects conducted for external clients,

and while he does not specifically mention risks that could correspond to these vendor

profitability-related factors, there are two items, each mentioned by one respondent, in his

‘other constructs’ category (The client’s premises are in good shape and in a nice district; They look like

they can afford to pay) that suggest that at least one of his respondents had some concerns

about the possible credit risks to his firm.

The final new risk reported in the vendor environment subset was overseas head office.

This factor was specifically mentioned in relation to problems encountered by project

managers working in the local Hong Kong branches of three of the four multi-national

vendor companies. These managers referred to difficulties they encountered when the

ultimate source of control and decision-making about their product and their project was

located overseas, typically in the United States. These difficulties were of two kinds. Firstly,

project managers encountered difficulties in getting required levels of product support from a

distant head office.

[C2] “Fighting politics is another thing as a PM - internal and external. So we [have been] talking about all the customer side, but internal politics is about how to deal with the developers [in the U.S.], how to deal with your management to support you to fight for resources to fix the bugs. … So you have no other choice but to jump on the backs of the developers and the support, but they’re so far away, how can they respond to every single request that you have?”

[G1] “The product was developed by our US team and at that time there were 2 teams, one from Asia-Pacific and the other from Europe, and for some reason there’s a sense of competition between Asia and Europe … So this is a difference in priority, so that we need to manage [it] and also our US team they were also pulled by different regions, AP and Europe, so it was a struggle.”

Secondly, managers also frequently found it difficult, particularly on large Asia-Pacific

projects, to balance the demands of their own local, regional and head-quarters managers as

well as mediating between the aims of the local, regional and headquarters client managers.

These tensions are aptly illustrated by the following quotation:

[U2] “In this case, I worked with the customer representative located in Hong Kong but he represents Asia-Pacific, and also his boss is actually located in Singapore, she was the head of Asia-Pacific, and then I also need to be responsible to the customer headquarters, the big guys. And I also have to report to [our own firm’s] project manager in United States, she reports to the customer US headquarters big guys, and I also have to take care of the interests of [our firm] in Hong Kong, and I also have to take care of the interests of all the parties in all the different locations. … And finally I choose to support the customer Asia-Pacific people and try to be the middleman between them and their

98

US headquarters and also my [own firm’s] project manager in the US. I have to take sides in order to contain the risk.”

This concern with a non-local locus of control for the vendor firm has a counterpart

in the new third party environment factor, non-local third parties. While Schmidt et al. (2001)

identify risks associated with third parties, including multi-vendor projects, external dependencies not

met, and lack of control over consultants, vendors, and subcontractors, they do not distinguish the

compounding effect of working with an overseas subcontractor or with the local branch of

an overseas third party.

[G2] “This legacy system was developed by a 3rd party vendor, so we need to work with this 3rd party vendor. And they actually are located in UK. Even before the start of the project we anticipated there could be some problem …”

[E2] “And that vendor is not in HK, so we’re working in an American time zone, so our team spends a lot of time at night on conference calls.”

[R3] “The subcontractor who sold us the product was the HQ office in London, but the office in HK lacked the skills in implementing this product and in supporting the product.”

The first of the two client environment factors, multiple sites or sites in multiple countries,

also reflects difficulties working with geographically distant parties. It is a little surprising that

the multiple sites aspect of this factor was not mentioned by respondents in Schmidt et al.’s

study, because it might be expected that managers from the large organizations contacted for

this study would have had experiences in project implementations across several branches of

their firms. In the present study, for example, one project encountering this problem [A1]

involved a system implementation in 28 branches of the client firm across Canada. The sites in

multiple countries aspect of this factor is simply a compounding of the multiple sites problem.

Implementing at many sites is difficult enough, but working with sites in many countries

increases the logistical problems and adds the extra issues of language and culture.

Moynihan’s (1996) construct knowledge of country/culture/language, mentioned by two of his

respondents, covers the language and culture part of this factor, and thus provides a partial

match, but does not appear to include the logistical problems that also arise with a multi-site,

multi-country implementation. Respondents described their experiences with projects in

mainland China, Taiwan, Singapore, India, Malaysia, Japan, Canada, South Africa, United

Kingdom, and Australia as well as Hong Kong, so it is not surprising that this factor was

regarded as a key risk by 13 of the 25 respondents in the present study. For example:

99

[H3] “The actual content providers were in Japan, and we weren’t able to have face-to-face meetings with them until we got everything set up over there in Japan, so we had to go and deal with that on a voice basis just over the phone, and that’s very difficult to get people to have a comfortable feeling about their ability to do things that they want to see, unless they can visualize it, it’s very hard to describe, and to describe things in Japanese over the phone, it’s awkward.”

[G1] “Since the AP region looks after about 52 countries in the region, so we need to produce a product that we can deploy in multiple countries, but since in multiple countries they have different customs, rules, regulations, so it was very difficult … Also we had simplified Chinese version, traditional Chinese, Korean, Japanese, Thai.”

The final new factor in the client environment subset, organization culture, encompasses

three aspects of ‘the way we do things round here’ (Schein, 1992, pg. 9). Firstly, respondents

referred to issues related to different organizational cultures within Hong Kong, particularly

differences between government and commercial clients.

[T1] “The people were an on-going difficulty. It seems people in government have a different mentality from commercial organizations…. With a semi-government organization it’s always difficult to get the team moving because of the nature of government organizations and the unwillingness of government staff to make decisions or to accept responsibility. So I needed to alert my management of this risk…”

Once again, this organizational culture risk was compounded if the client was located

in another country, and this was particularly so for projects in mainland China, where firms

were perceived as having quite a different way of doing business as illustrated by the

following quotations, which exemplify the comments of all four respondents who had

experience with mainland China projects:

[F] “In HK if you ask people to work overtime or on Sundays it’s no big deal, but in China it’s not that easy.”

[L2] “This project, [in China], the major risk is actually a different way of doing things. In China, the way they like to run IT project is, the systems people just think of something they want to do and they go and do it. They won’t plan, they won’t organize, they won’t even communicate well to let people know that I am making this change to this system. … [so] their way of doing things, their culture may not appreciate the same kind of project disciplines that you want. To me that was the first risk.”

And finally, one in-house manager spoke of the prevailing culture within his

government organization which made it extremely difficult to ever declare a project finished.

[B2] “In [our organization], the user departments don’t care, they just keep changing the requirements, and if we can’t do it for them, they say OK we accept the system partially, and then because we have endless resources we can keep on doing for them as long as my boss agrees, and if not then their boss will talk to my boss! We’re always changing things. Even now, the [B2] system was

100

rolled out in 1998, and even now annually we are doing - we have 2 staff supporting it doing program modification for them - they enjoy this kind of service!”

b) Relationship factors: The second subset of new risks concerns the key relationships

the project managers need to maintain during the course of their projects, with three factors

related to vendor relationships (one with a partial match to a Schmidt factor), and three

concerning client relationships (one of which has matches with Moynihan and Sumner

factors).

One of the three vendor relationship factors, vendor top management support, could be

seen as a partial match of the factor lack of top management commitment to the project reported by

Schmidt et al. However, managers in the present study made a distinction between the need

for a client top management sponsor, and the need for top management support within their

own vendor organization. In particular, they noted the importance of getting the necessary

approval for increased budget expenditure from their own project executives when they were

asked to take on a project that had been under-bid by the pre-sales team, and also in getting

assistance when necessary to influence clients, third parties and other sections of the vendor

firm itself. The second vendor relationship factor, internal negotiations, is closely related to the

top management support factor, in that project managers reported risks associated with

internal competition for a limited pool of staff, and also difficulties in influencing other

sections of the firm, for example, for customization of package requirements. These internal

competition risks were aggravated if the manager did not have strong backing in the internal

negotiations from his or her project executive.

[X] “One issue can be that the firm is running several different projects that are competing for the same internal resource.”

[W1] “Project team is very important, and then initially I have to fight for the resources. And once the project team is set up, then I try to stabilize it so that it won’t change. This is a very important issue.”

The third vendor relationship factor, team morale, was frequently highlighted, with 13

of the 25 respondents identifying it as an issue. This factor was identified as an issue at all

stages of projects. At the start of projects, managers were concerned to ensure that the team

was well motivated:

[T2] “We had to set the team expectation right from the start - let them know the timeline is set and can’t be changed, and that we must achieve it because the project is very high profile for [our

101

firm] in the Asia Pacific region. The whole management has a close involvement and interest in this project. The team was very important to the success. I had to constantly monitor and we worked a lot of O/T.”

It was also a problem that could arise during projects, as illustrated by this quotation

from a project that ran into substantial difficulties part way through:

[D1] “So all these tensions they were trying to cope with, and it really nearly broke the team, not because they couldn’t do the work, but just that if you’re normally coping with 10% stress level, they were coping with 60%, which meant that any time we put pressure on them from a project perspective they nearly broke.”

And for managers taking over ‘troubled’ projects, one of their most pressing

concerns was the potential effect of the previous poor project performance on the morale of

the team:

[N1] “Morale was an issue when I took over. We had people that were owed 2 months time off in lieu of payment for OT work, and it was necessary to carry on working them 6 or 7 days a week, and 12 hours a day at least. So morale was low because these guys were working hard, they hadn’t been compensated, to be told, you’ve got 2 months off in lieu of OT, is unrealistic in [our firm] because you never get the chance to take that, and what are you going to do with 2 months off anyway? That’s not going to compensate you.”

While the team morale factor was seen as an issue of building and maintaining good

relationships within the vendor team, managers were also concerned with the need to build

effective working relationships between the vendor team and the client staff, and thus two of

the three new client relationship factors, trust and ‘bad news’, both concern the management of

vendor-client relationships.

The need to develop a strong and trusting relationship with the client management

and users, and the problems that could arise if trust was not established from an early stage,

figured prominently in the respondents’ concerns, with 14 of the 25 respondents identifying

this factor. This aspect of building relationships with the client appears to have no

counterpart at all in Schmidt’s list, perhaps because in-house projects typically begin from a

starting point of some common ground, since all players are working for the same employer.

Representatives of vendors, on the other hand, are sometimes viewed with some suspicion,

as this vendor project manager described: [L2] “You have to constantly show them… you don’t just

come in, get the money, and go.” One in-house manager in the present study with considerable

experience in managing out-sourced projects also identified the importance of establishing a

basis of trust with the vendor to ensure a good working relationship. The key emphasis

102

placed by the respondents on this factor is further illustrated by the way their descriptions of

the risk are intertwined with descriptions of the strategies they used to address the risk. This

intertwining of risk perception and risk prevention strategies is discussed further in Chapter 5

where I discuss the strategies employed by project managers to address risks in IT projects.

[C1] “I have to go around, kind of lobbying or chatting with people to make them feel comfortable that I am not the guy that will create problems for you guys, so don’t do that for me! That’s the way I kind of convey that message around. So relationship building for project is very important - not just this project, any project.”

[U3] “The first thing you have to do… is they want to see something immediately so they can start building trust on you.”

[F] “If you have a better personal relationship with them they can make the project much easier when they negotiate. Once they get to trust you, they will respect you more. They will trust you more, even though things might not be so good for them they will trust you.”

[I] “You explain your method, and that’s where that communication is, and then they have that trust. Because as a contractor you have to build confidence on their side also.”

The ability of the project managers to build a trusting relationship was seen as crucial

in mitigating the ‘bad news’ factor. Ten managers referred to this factor, which they saw as

risks associated with the unwillingness of users and the client IT department staff to keep

their senior management informed of possible problems in the project, particularly where

these problems could be construed as indicative of poor performance or poor decision

making on the part of the client staff.

[N1] “…they tend to not want to be the bearer of bad news. To be the bearer of bad news implies you’re not doing your job.”

[D2] “…he didn’t want to go upsetting the Chairman or anyone else …”

Most of the ‘bad news’ references related to problems that arose during the projects,

rather than potential problems identified at the start. These problems were seen as difficult to

address because the obvious solution, the vendor project manager delivering the bad news

himself or herself, was seen as likely to destroy any trust that had previously been established

with the client staff.

[K1] “…they said they were in trouble - they say to me, not to the boss, and then as a vendor, you would not go over them and talk to their boss. That is more like a respect, and a relationship problem.”

In-house managers also experienced this problem:

103

[B2] “… the working level dare not to talk to the senior quite often. I work with [government departments] and they’re all similar - the junior ones won’t speak up to the senior ones.”

The third client-related relationship factor, client IT department, had no match with any

Schmidt factors, but did have matches with Moynihan’s factor working through users vs. working

through IT department, and with Sumner’s factor insufficient training and re-skilling of IT department

staff. Concerns about the skill levels of the client IT department, and about their willingness

to cooperate, were highlighted by seven of the respondents in the present study, and were

mentioned by half of Moynihan’s 14 respondents, and in four of Sumner’s cases. Schmidt’s

list does include general factors related to the lack of required knowledge and skills in the

project personnel. However the distinction made by respondents in the present study

between client IT staff skills and willingness to cooperate, and vendor team skills, together with

the identification of IT department skills as a separate concern by both Moynihan’s and

Sumner’s respondents, justifies the inclusion of a separate factor for this item.

c) Project management factors: The two new risk factors in the project management

subset, client project management and client readiness are closely related and concern issues about

the ability of the clients to manage their side of the project. Neither of these factors is

mentioned in Schmidt’s list, but both have partial matches with Moynihan and Sumner

factors, and this reflects the greater degree of vendor orientation in these two studies. While

the presence of a strong and competent project manager on the client side was welcomed by

respondents, weakness and incompetence in client project management was seen as a significant

and difficult risk to deal with.

[O2] “So my role becomes, make sure everybody [on the client team] knows what we’re doing, knows where we’re going, and what we need to achieve … nobody was doing that, it wasn’t there, the people supposed to be doing that weren’t really doing it.”

The extent to which the client organization was ready and prepared for the new

system, i.e. client readiness, was seen as a risk that was easy to overlook, and one that was

important for the vendor project manager to check personally, rather than simply relying on

assurances from the client’s senior management.

[A] “The readiness of the organization - we think we have the best project manager, the best team, … they could go in there and do anything, they know the technology inside out! The money’s there, everything, but some organizations are just not ready, in terms of culture, in terms of infrastructure, in terms of even personnel. I think a lot of the time, thinking about some of my projects, if I had walked the floor earlier, if I had gone out to the field, to the client, to the location … I should have.”

104

d) Technology factors: The final subset of new risks comprises four technology factors.

Two of these factors, customization and complexity, are risks associated with the package

proposed by the vendor for the project, while integration and compatibility related to problems

that could arise with interfaces to any third party products proposed as part of the solution,

and data conversion related to risks associated with the need to convert or extract data from

client legacy systems. These are all issues that have no direct counterpart in the Schmidt list

but which are identified by one or both of Moynihan and Sumner. All four of these risks may

be more critical for vendors working in a package implementation environment, rather than

on custom development in-house projects, since vendor staff will have less familiarity with

the detail of the client technical environment. This may explain why these factors were not

identified in Schmidt et al.’s study. However, two of these problems were noted in in-house

projects in the present study, although they were mentioned by only one manager each. A

third party integration problem was reported by the manager of one in-house custom

development project, and the manager of an in-house outsourced project noted that they

encountered problems in data conversion from a legacy system.

The customization factor corresponds to some extent to Sumner’s factor, failure to adhere

to standardized specifications which the software supports. While Sumner refers to the importance of

using an ERP package’s built-in ‘best practices’ the present respondents expected that some

customization would always be required, although they were concerned to limit this as much

as possible.

[T1] “There was some customization. We would try to leverage the standard product as much as possible and persuade the client to accept the standard solutions in order to best provide for on-going support but it was not clear how much customization would still be required.”

This factor also includes risks associated with package integrity across other clients if

major changes are required. For example:

[N1] “And because we were dealing with a package a great deal of work had to be done in order to do that [meet the non-standard requirement] and still protect the integrity of the package.”

These customization problems were sometimes seen as a result of over-optimism at

the pre-sales stage. The pre-sales team could make promises about customization in order to

win the project, which then had to be delivered, as this project manager found:

105

[N1] “You couldn’t take any standard package off the shelf and try to fit it to this one. But that’s we had proposed and that’s we had sold, and that’s what we then had difficulty delivering.”

Indeed, one of the pre-sales respondents acknowledged that the pressure to win the

sale can outweigh concerns about the possible resulting risks for the implementation project

manager:

[Q1] “Some of the features they would like to get require additional customization, the product can do that, no problem, but it will require more effort. So in response during the sales cycle, some-one ask me, can it be done, I say yes it can be done. But during the project when the customer ask for this feature, this capability, our manager turn him down because it involve huge customization, so that was a little bit misleading, but back then we have to win the deal - I told the truth but I didn’t explain in detail how it can be done and the effort required. … so it takes the [manager]’s effort to talk him out of that, to be out of scope.”

The complexity of product factor can be matched with Moynihan’s logical complexity of

application, and in the present study has been used to code respondents’ references to risks

associated with the number of different modules to be implemented. This factor includes

some project managers’ concerns about implementing combinations of modules that had not

been tried before, as the following quotation illustrates.

[C2] “They selected a combination of products that is not as stable as most of the other customers, because it’s a newer area of our product suite. It’s financials, ERP, supply chain, manufacturing. It’s the manufacturing part that is more problematic because of the complexity of the design and these were not factored into the project.”

Finally, both the third party integration factor and the client data conversion factor relate

to respondents’ concerns in Moynihan’s study with the need to integrate/interface with other

systems. Sumner also refers to lack of integration and risks associated with attempting to build bridges

to legacy systems.

In general, while the area of technology was recognized as a source of several risks,

the respondents in the present study viewed these risks as typically relatively easier to

manage:

[E] “There’s always technical risk areas … that’s actually the stuff that staff find interesting. Technically they like doing all that sort of work. So we don’t see that as big risk area.”

106

4.3.3. Discussion of new risk factors identified in this study

As described in detail in the previous section, twenty new risk factors for IT projects,

not distinguished in previous research, have been identified in the present study. These are

summarized below in table 4.4.

Earlier in Section 2.2.3, I reviewed the descriptive IT risk factors literature and noted

that little is known about the extent to which the various risk taxonomies reported in that

literature are applicable in all IT project situations. The new risk factors identified in Table

4.4 suggest three specific situations may warrant special attention, with additional risks that

project managers in these situations must address.

Table 4.4: New risk factors identified in the present study

Set

Theme Vendor Third Party Client Relationships - Team morale

- Internal negotiations - Top management support

- Trust - IT department - Bad news

Project Management

- Readiness - Project management

Environment - Reputation - Overseas head office - Competition - Legal & credit risk - Contract terms &

conditions

- Non-local 3rd party - Multiple sites/countries

- Organization culture

Technology - Customization - Complexity

- Integration & compatibility

- Data conversion - Technical environment

Firstly, vendor-driven projects have a unique set of additional risks. Managers of

these projects need to maintain an awareness of the commercial environment their firms are

operating in, particularly with respect to their firm’s reputation in the marketplace, and also in

terms of possible threats from their competitors. The contractual nature of the relationship

between the vendor firm and its client can also be a source of risks for the manager trying to

implement the project. The project managers in the present study were particularly aware of

the need to develop and maintain good working relationships with their clients in order to

achieve a successful conclusion of their projects, and to pave the way for future business

between their firm and the client. The separation between the vendor firm and its clients also

means that vendor project managers are likely to know less about their clients’ technical

environment and capabilities, and less about the detailed design and operation of the clients’

107

existing systems. This lack of knowledge increases risk in areas related to fitting the new

system into the clients’ existing operations, both in terms of understanding issues such as

data conversion from legacy systems, and in terms of understanding the capabilities and

general readiness of the client organization for the implementation of a new system. Finally,

project managers of vendor-driven projects have an additional layer of management to deal

with, i.e. their own internal management, and must be aware of the potential threat to their

projects from internal competition for limited resources.

Secondly, implementers of software package systems have additional risks specific to

the nature of their product. While the general ‘wisdom’ with package implementation is that

firms should change their business processes to match the package and to take advantage of

the claimed ‘best practice’ embedded in the package, the present study respondents’

experience suggests that some degree of customization is almost inevitable in any package

implementation. This need for customization is associated with risks both in terms of

understanding the extent of the required changes, and in terms of managing the impact on

the integrity of the package. Again, the project managers in the present study also reported

risks associated with their own internal management in terms of negotiating the required

support to make requested customization changes. A further risk arose with the complexity

of system requested by some clients. Many of the large package systems now available have

so many different possible combinations and configurations that it is unlikely that all possible

variations have been fully tested. Thus a project involving a complex and unusual mix of

package modules is likely to present additional risk.

The final IT project situation needing special attention identified in the present study

arises out of the combination of the respondents’ location in Hong Kong, and the complex

nature of many of the projects. The key location of Hong Kong in south-east Asia meant

that several respondents in the present study had experience working on multiple country

implementations, and as a result, identified a further new risk associated with the logistical

and cultural demands of this type of multi-location project. Other studies have worked with

respondents from in-house IT departments in large organizations, and these respondents are

most likely to have been focused on internal projects, rather than involved with multi-

country implementations. However, it is surprising that the issue of implementing at multiple

sites within a single country has not been raised before, since project managers working on

108

projects with large organizations might be expected to have experience with implementing

systems in different branches of the same firm.

A second aspect of the location of the present study in Hong Kong is the greater

incidence of projects involving overseas third parties or an overseas vendor head office.

Again, the remote location of key participants in the project introduces logistical risks that

have not been identified by previous researchers who, with the exception of Schmidt et al.,

have tended to focus on in-house development in major western countries such as the

U.S.A., Canada, or the U.K. One of Schmidt et al.’s panels was based in Hong Kong, but

this panel did not mention working with non-local third parties as a risk (or if they did, it was

reported or coded differently). The risks identified by the nine Hong Kong panellists seem

more congruent with risks reported in other studies for in-house development projects, so

these Hong Kong panellists may have had less experience than present study respondents

with complex implementations involving products from overseas third parties.

4.4. Frequency with which risk factors were mentioned

The large number of new risk factors identified in the present study is interesting, but

if these new factors were only mentioned by relatively few of the respondents, compared to

other factors, then they might be considered only of minor importance. While respondents in

the present study were not asked to rank factors in order of importance, the critical incident

technique used in the interviews was specifically designed to tease out those risks that

respondents thought important enough to attend to, rather than seeking every possible risk

that they might consider. Indeed, as discussed later in Chapter 5 on the results for Research

Question 2, respondents tended to focus their attention on those issues they saw as critical to

the success of their projects. Thus by counting the number of respondents identifying a risk

factor as needing attention, a ranking of those risks deemed by respondents to be important

enough to attend to can be inferred, and is shown in Table 4.5.

Table 4.5 includes eight of the new factors (shown in bold print) identified earlier in Section

4.3.2. Three of the four new vendor environment factors, risks from competitors, legal and credit

risk, and risks from contract terms and conditions and one new client environment factor, multiple

sites/multiple countries, featured highly in the concerns of the present study’s respondents

(ranked 6th equal, 12th equal, 14th, and 9th equal, respectively). Similarly, three of the four

completely new relationship factors, client trust (6th equal), vendor team morale (9th equal), and

109

client ‘bad news’ (15th equal) were frequently mentioned as issues of concern. Finally, one of the

four new technology factors, customization, ranked 15th equal, was also an issue for these

respondents. Thus, almost half (8 out of 17) of the highest ranked factors in the present

study have not been mentioned in previous studies. These new high ranked risk factors relate

to the specific environment of vendor-driven projects, to the relationships that project

managers of vendor-driven projects need to establish and maintain with their customers, and

to particular aspects of the (typically package-based) technology being implemented in these

types of projects. This finding reinforces the importance, foreshadowed earlier in Section

2.2.3, for practitioners of ensuring that any risk factor check list used is indeed appropriate

for the particular project situation.

Table 4.5: Top 17 Risks (mentioned by at least ten respondents)

Risk Factor Rank (no. respondents out of 25) Schedule & budget management 1(22) Client expectations 2 (18) Change management 2 (18) Client understanding of requirements 4 (17) Vendor staffing 4 (17) Client trust 6 (14) Vendor understanding of requirements 6 (14) Vendor competition 6 (14) Vendor team morale 9 (13) Newness of technology 9 (13) Client multiple sites/countries 9 (13) Vendor legal & credit risk 12 (12) Vendor documentation management 12 (12) Vendor contract terms & conditions 14 (11) Third party deliverables 15 (10) Customization 15 (10) ‘Bad news’ 15 (10)

4.4.1. Comparison of risk factor rankings with previous research

Schmidt and his colleagues analyzed importance ratings from their Delphi survey

(Schmidt et al., 2001) of experienced software project managers in a separate paper (Keil et

al., 1998), where they identified a common set of 11 factors considered by respondents to be

most deserving of a project manager’s attention. A direct comparison between rankings in

the present study and in the Schmidt/Keil study is not possible because there is no one-to-

one correspondence between the factors. However, a partial comparison can be made, and it

is useful to look at how the present study respondents ranked those factors that are the

110

closest match to the top eleven factors reported by Schmidt and Keil. This comparison is

shown in Table 4.6. Where several of the Schmidt/Keil factors were mapped to one factor

from the present study, I have identified the closest matching factor with the addition of

‘(part)’. Where two or more factors together from the present study provide a match to a

Schmidt/Keil factor, both are shown and I have reported the total number of respondents

mentioning either of the possible matching factors.

As can be seen from Table 4.6, seven of the top eleven Schmidt/Keil factors were

mentioned by at least half of respondents in the present study, and only one, conflict between

user departments, was mentioned by less than a third of the current respondents. Thus,

although present respondents may have been reporting from a different perspective to that

of respondents in the earlier study, there is a considerable level of agreement about the core

key risks that need attention.

Table 4.6: Comparison of risk rankings: Schmidt/Keil studies and present study

Schmidt/Keil Factor Schmidt/Keil rank (1 is highest)

Closest matching factor from present study

Present study rank (no. respondents out of 25)

Lack of top management commitment to the project

1 Vendor top management support Client top management support (part)

22 (8)

Failure to gain user commitment 2 Client users (part) 22 (8) Misunderstanding the requirements

3 Vendor understanding of requirements (part)

6 (14)

Lack of adequate user involvement

4 Client users (part) 22 (8)

Failure to manage end-user expectations

5 Client expectations 2 (18)

Changing scope/objectives 6 Change management (part) 2 (18) Lack of required knowledge/skills in the project personnel

7 Vendor staffing (part) 4 (17)

Lack of frozen requirements 8 Change management (part) 2 (18) Introduction of new technology 9 Newness of product 9 (13) Insufficient/inappropriate staffing 10 Vendor staffing (part) 4 (17) Conflict between user departments

11 Client multiple departments (part) 32 (6)

111

4.4.2. Discussion of comparison of risk factor rankings

Although, for the most part, current respondents pay most attention to the same core

risks that have been identified in previous studies, two results are surprising: Schmidt/Keil’s

number one factor, lack of top management support, does not appear among the top seventeen

factors of the present study; and the highest ranked factor in the present study, schedule and

budget management, does not have any corresponding factor in Schmidt/Keil’s top eleven.

However, the present study findings do closely support Boehm’s (1991) study, which is often

cited as a seminal study on risk management in IT projects. Boehm’s article does not

mention lack of top management support in his top ten factors, but does report unrealistic schedules

and budgets as the second highest risk after personnel shortfalls, which was ranked 4th in the

present study.

A closer examination of the type of respondents and the target reader audience in the

previous studies gives some clues about the differences in the studies that may lead to such

different perspectives on the importance of key risk factors. Boehm’s article was written from

a practitioner perspective, and his risk factor list was based on his own extensive project

management experience together with a survey of “several experienced project managers”.

Thus Boehm’s study may be aimed more at the operational level of project management. In

contrast, although detail of the current positions of Schmidt et al.’s respondents is not given,

we do know that all three panels had high levels of work experience (average 16 years) and

were nominated by senior executives in their organizations as the most experienced and

successful managers. In the present study, I found that such managers were typically

operating at the project executive level, and thus might be expected to have a more strategic

management focus.

With the exception of the three project executives, I took care in the present study to

specifically select respondents who were hands-on project managers, actually working in the

field. These project managers tended to have a more operational perspective on their

projects. The project executive respondents, who had both in-house and vendor project

executive experience, clearly outlined their different perspectives as executives, as this

comment shows:

[D] “[The project executive has] to deal with and to shield the project managers from politics in organizations, from organizational conflict. Project managers should be focused on, I’ve got these

112

activities, I’ve got to get them done, I’ve got to get them completed within the time and budget that’s been set for me, and to the requisite quality. [Project executives] should be dealing with all the things that might deflect the project managers from [their task].”

In the view of these three project executives, issues of ensuring adequate top

management support for a project, and the possible associated internal political manoeuvring,

fall in the project executive’s domain, at the initiation stage of the project. The project

managers should be immediately escalating any ‘political’ issues that might require top

management intervention to their senior executives. In contrast, they saw issues of schedule

and budget management as pivotal to the project manager’s work.

The current respondents were focused by the interview technique on what they

themselves actually did during their projects to address and manage risk. It may well be the

case that, while these project managers were aware of the need for top management support,

they saw it as an issue that fell outside their locus of control and therefore not one that

ranked highly in their perspective. This finding is consistent with March and Shapira’s (1987)

finding, already noted earlier in Section 2.4, that managers do not regard issues outside their

control as risks. On the other hand, respondents were very focused on anything that

threatened their ability to meet the required budget and schedule, and as discussed later in

Chapter 5 on risk management strategies, they viewed strategies to closely monitor and

control performance against schedule and budget as essential to successful project

management.

4.5. Anticipated and unanticipated problems, and ‘troubled’ projects

The distinction made during coding between potential problems identified at the start

of projects and actual problems that arose during the project provides the opportunity to

explore in greater depth whether any particular risks are more likely to cause projects to go

off track. Three different aspects are interesting. Firstly, which problems are still likely to

occur, even when they have been anticipated, and presumably mitigated, at the start of the

project? Secondly, which problems are most likely to be overlooked at the start, and then to

arise unexpectedly at some stage during the project? And finally, what are the key problems

that caused some projects to become ‘troubled’ projects? To investigate these questions, I

looked at the risks identified at the project level, rather than risks mentioned at the project

manager level. Sixty projects were discussed by the respondents, but some of these were only

mentioned briefly to illustrate a particular issue or to contrast with a previous project

113

discussed by that respondent. Generally, those projects that were only described briefly went

relatively smoothly, and so it is to be expected that they would reveal little in the way of

major problems. In total, 39 of the projects were discussed in depth, and these 39 projects

were used for the analysis that follows.

4.5.1. Anticipated problems

Table 4.7 shows the top seven problems most often anticipated by present study

respondents at the start of their projects, and the number of projects where the problem still

arose in spite of any mitigating action taken by the project manager. As might be expected,

the factors most typically anticipated at the start of projects also featured highly in the

importance rankings shown in Table 4.5. The only new factor (not in Table 4.5) appearing in

Table 4.7 is client organization culture, and this factor was actually the highest ranked of the

factors not shown in Table 4.5, being noted by 9 respondents.

Schedule and budget management caused the most problems even after mitigating

action had been taken, and in all eight of the projects where this factor still arose the

managers believed that they had been asked to take on an overly ambitious project schedule,

committed to by the pre-sales team. This highlights a particular issue that has not been

mentioned before in the literature, namely the tension between pre-sales and implementation

stages of vendor-driven projects. Project managers viewed the desire of sales team to win the

bid, and also to remain within their own limits in terms of costs of pre-sales work, as an

underlying cause of many subsequent problems in IT implementation projects.

Table 4.7: Problems anticipated, and number still arising

Risk factor No. of projects in which problem was anticipated

(out of total 39)

No. of projects in which problem still arose despite

mitigating action Schedule & budget management 24 8 Vendor staffing 16 5 Newness of technology 12 5 Multiple sites; multiple countries 11 1 Vendor understanding of requirements 9 0 Client expectations 8 3 Client organization culture 7 4

The best way to avoid underestimation of project budgets and schedules is to do the

pre-sales requirements analysis as thoroughly as possible, but this is costly, particularly if the

bid is not successful. Even if a thorough pre-sales analysis is completed, the final bid may

114

need to be cut in order to make it competitive. Indeed, one respondent saw this risk as

endemic in the industry, and a key problem for IT professionals:

[A] “I can say that almost across the board for IT personnel we oversell, we promise more than we can really do. We try to get the business a lot of the time - of course we are vendors, we provide services, but even for in-house MIS organizations, we over-promise. IT people do not give a real picture to the users about the project challenges, and I think that’s a major, major problem.”

The smallest firms might be expected to be most at risk here, since they were most

likely to be very ‘hungry’ to win a bid, as exemplified by this quote from a project manager of

a small firm describing one of the early projects for his firm:

[H1] “We did a formal risk assessment, and there was an incredible amount of risk. We saw that, we knew that going into it. But at the same time, it was one of our first projects … and they were willing to pay good money, and we felt that we could handle it.”

However, six of the eight projects where schedule or budget continued to be a

problem were large firms, and as described in Chapter 5, these firms have well established

procedures aimed at minimizing potential problems flowing on from the pre-sales stage.

This apparent failure of mitigation actions taken at the pre-sales stage to prevent the

occurrence of schedule and budget problems will be explored further in Section 5.2 on risk

management practices.

4.5.2. Unanticipated problems

Table 4.8 shows the top seven unanticipated problems that actually occurred at some

stage in the projects discussed by respondents.

Table 4.8: Unanticipated problems arising during the project

Risk factor No. of projects in which problem occurred

unexpectedly (out of total 39)

Client trust 12 Client expectations 9 Vendor team morale 8 Change management 7 Client understanding of requirements 6 Client project management 5 Client ‘bad news’ 5

Tables 4.7 and 4.8 between them contain all but one of the top eleven factors ranked

as most important in Table 4.5. The one factor mentioned in the top eleven in table 4.5 and

115

not included in Tables 4.7 and 4.8 is vendor competition, and this factor was mentioned mainly in

respondents’ general comments, rather than in relation to particular projects. (Nine of the

fourteen respondents mentioning vendor competition did so in their general comments). Hence

it does not show as a major issue in the project-by-project analysis.

With the exception of client expectations, the set of problems that occur unexpectedly in

projects is quite different from those that arise in spite of being anticipated and mitigated.

While the most important risk ranked by project managers shown in Table 4.5, schedule and

budget management, is one that is also most likely to be anticipated in a project, the next three

most important risks in Table 4.5, client expectations, change management, and client understanding of

requirements do not feature in the most often anticipated problems but are all highly likely to

occur unexpectedly. These unforeseen problems are more likely to be client-related rather

than vendor-related, and are more likely to involve relationship issues.

Clearly, practitioners would be well advised to consider and pay attention to

relationship issues both with their client and within their own team at the start of the project,

and to take steps to ensure those relationships are well-founded right from the start. As I

noted earlier in the discussion on new risk factors in Section 4.3.2, respondents in the present

study were keenly aware of the need to develop sound working relationships with clients.

While this awareness resulted in strategies to manage relationships, discussed later in Section

5.3 on strategies used by project managers, it appears that respondent rarely anticipated that

such relationship-related problems might actually occur. It is possible that these client

relationship risks are not so often anticipated because they are perhaps less tangible, and

therefore more difficult to quantify or assess in a risk management exercise. Thus project

managers may tend to disregard them or place less weight on them than on other, more

visible and more easily measured risks such as how much slack has been allowed in the

schedule, or how familiar the team is with the proposed technology.

The client expectations factor features prominently in all three tables (4.5, 4.7 and 4.8),

and was also a key factor in ‘troubled’ projects, as discussed below in the next sub-section.

The successful management of client expectations was seen by a number of respondents as

one of the key areas for managing risk, and I also discuss this further in Chapter 5.

116

4.5.3. Problems causing projects to become ‘troubled’

The ten ‘troubled’ projects are of particular interest, since they represent projects that came

close to failure, and the assessments of these projects by the highly experienced managers

brought in to rescue them may give significant clues about the critical factors that can make

the difference between success and failure. I discuss the assessment of these projects by their

managers in depth in Section 5.5, when analyzing the decision-making processes of project

managers in managing risk, so here I simply summarize the key problems identified by these

experts during their initial assessment when they took over their ‘troubled’ projects. Note

that all of the ‘troubled’ projects were considered by their ‘rescuers’ to have reached a

successful conclusion, even though typically they were very much over schedule and budget.

For these managers, the definition of ‘success’ appeared to be that the systems are ‘live’ and

in use.

Table 4.9 shows the top six problems that managers of ‘troubled’ projects considered had

occurred to send these projects off the rails prior to their taking over.

Table 4.9: Problems responsible for ‘derailing’ troubled projects

Risk factor No. of projects in which problem occurred

(out of 10 troubled projects) Schedule & budget management 8 Vendor staffing 8 Vendor understanding of requirements 4 Client expectations 4 Vendor team morale 4 Change management 4

While the factors identified by these ‘troubled’ project managers are shown as

separate factors as shown in Table 4.9, the process of identifying and coding each factor

separately actually obscures the view of the project managers that in fact these factors were

highly inter-related. Poor schedule and budget management was described as a key factor in

causing problems in these projects, and in five of the ten projects this was attributed to an

original under-estimation or under-bidding on the part of the pre-sales team:

[L1] “They significantly underbid. The estimate was wrong [because] the complexity of requirements was not fully understood.”

117

The situation was often compounded by serious deficiencies in the estimation of the

vendor staff skills required to complete the project.

[N1] “It was grossly under-sized … The original plan called for just one guy with skills and knowledge of that package. We ended up by having about 6 or 7 people, brought in from overseas.”

The failure of the vendor staff to fully understand the client requirements at the pre-

sales stage also contributed to both the underestimation of schedule/budget and of required

skills.

[N1] “The other key point to cause it to be troubled was the wrong solution - we tried to put a package solution in where a new development - a roll-your-own solution would have been more appropriate…. You couldn’t take any standard package off the shelf and try to fit it to [these requirements].”

This often led to difficulties in controlling change requests from the client, since,

from the client’s perspective these were not change requests but legitimate requirements of

the project.

[N1] “The requirements were not so much changing but being reinterpreted at times. … So you could say there may not have been additional requirements, but clarification of how the requirements should be dealt with.”

As the complexity of the requirements became clearer, typically performance against

schedule would deteriorate, setting up a spiral of declining team morale, which would cause

further performance deficiencies. Underlying all of this was the failure of the pre-sales team

and the original manager to set and control client expectations at a reasonable level.

Finally, it is interesting to note that the first three factors identified in Table 4.9 as key

problems in ‘troubled’ projects rank first, second and fifth of most often anticipated

problems in Table 4.7, while the fourth to sixth factors in Table 4.9 rank second, third, and

fourth of most frequent unexpected problems in Table 4.8. As I discuss in Chapter 5, the key

strategies recommended by present study respondents for managing risk in their projects

revolve around these key risks in Tables 4.7 to 4.9.

4.6. Summary of discussion: Research Question 1 – Key risk factors

In this chapter, I have presented the results from the investigation into Research

Question 1, which asked: What key risks do project managers attend to in software package

118

implementation projects, and how do these compare with the risk factors identified in the

literature for software packages as well as for custom development?

I have identified a number of new risks, summarized in Table 4.4, that should be

considered particularly in vendor-driven package focused projects, and I have suggested that

the differing views in the literature about the importance of certain risk factors may be related

to the management level of respondents being studied. Further study is needed in this area to

examine the extent to which risk perspectives vary across management levels.

Six key risks, schedule and budget management, vendor staffing, vendor understanding of

requirements, vendor team morale, change management, and client expectations have been highlighted as

critical factors causing major problems in ‘troubled’ projects. Three of these risks featured

highly among those typically anticipated at the start of projects, while the remaining three

occurred frequently as unexpected problems in projects. Control of these six key risks was

considered by respondents to be crucial for successful management of risk in their projects.

Next, in Chapter 5, I present and discuss the results for Research Questions 2, and 3,

which relate to the project managers’ risk management practices and strategies in managing

the risks they identified in their projects.

119

C h a p t e r 5

5RESULTS AND DISCUSSION OF RESEARCH QUESTIONS 2 AND 3: RISK MANAGEMENT PRACTICES AND STRATEGIES

5.1. Introduction

Research Questions 2 and 3 involved the investigation of the risk management

practices applied by respondents at various stages in their projects, and the strategies they

used to address specific risks, both to mitigate risks at the start of projects and to address

problems that arose during their projects. The respondents’ reports of key risk factors they

attended to and standard risk management practices they employed were to some extent

intertwined with the strategies they described to manage risks, and thus a detailed and careful

teasing out of managers’ descriptions of projects and their general comments was required in

order to reveal the underlying practices and strategies.

The coding and analysis processes to address these questions were described in

Chapter 3. In this chapter I report on the results of that investigation, discussing Research

Question 2 on risk management practices in Section 5.2, and Research Question 3 regarding

respondents’ risk management strategies in Section 5.3. Finally, Section 5.4 completes this

chapter with a brief conclusion.

5.2. Research Question 2: Risk management practices

Research Question 2 asked: What risk management practices do project managers

employ when managing software package implementation projects, and how do these

compare with prescriptions in the literature? In order to answer this question I used

summaries of risk management processes prepared during the Description stage of analysis,

as described in Section 3.4.2. In the following sub-sections I report on the analysis and

interpretation of these summaries. I first describe the pre-project risk management practices

reported by respondents, and then review the risk management practices employed at various

stages of the project implementation – at project start-up, during the course of the project,

and at the end of the project. Finally, I compare these practices with prescriptions from the

literature and discuss some of the issues arising from these practices in particular contexts.

120

5.2.1. Pre-project risk management practices

Vendor respondents. All of the vendor respondents, with the exception of two

discussed below from the same small ‘boutique’ software house, indicated that their firms

used some form of pre-project risk assessment that was usually carried out at the pre-sales

stage. The practice described by all these respondents was similar, although the degree of

formality was less in the smaller firms. The risk assessments typically involved a detailed

questionnaire or checklist, consisting of a number of questions covering the broad areas of

schedule and budget estimation, requirements and the match with the proposed solution,

technical issues, sub-contractor issues, and contract terms and conditions. The risk

assessment checklists made little, and in some cases, no reference to potential risks in areas of

relationship management and communication, which is interesting given that, as discussed in

Chapter 5, these are the key areas that often result in unanticipated problems in the course of

the project. Respondents typically expressed some surprise when communication or

relationship problems arose during the course of a project and it seemed that such problems

were difficult to identify at the early stages of the project, when both sides were eager to

negotiate a deal and as such perhaps on their ‘best behaviour’.

[A] “It’s almost like a marriage, it starts off very, very happy, everybody has a rosy picture about what’s coming down the pipe …”

The responses to the risk questions rarely took the form of an explicit estimate of

impact and probability for a risk item. Instead, potential problems were usually assessed

either on a yes/no basis indicating whether or not they applied, or with an estimate of

whether the risk was a low, medium or high item with no differentiation between size of

impact and likelihood of occurrence. Some firms completed a formal risk response planning

exercise at the pre-sales stage, identifying mitigating actions for the highest priority risks.

However, as discussed in Section 5.2.2 below, there seemed to be little follow-up by the

implementation team on the execution of specified mitigating actions. For most firms, the

key action taken at this stage was the addition of contingency time and dollars to the schedule

and budget estimates.

In the larger firms, the pre-sales risk assessment was carried out by a separate quality

assurance team, independent of both the pre-sales and the project implementation teams.

This separation of duties was intended to guard against any conflict of interest that could

121

arise from the desire of the pre-sales team to win the bid, or from over-enthusiasm of the

implementation team for a new and interesting project. Mid-size firms typically did not have

a dedicated quality assurance section, but relied on an independent review by another project

manager not involved with the project in order to get an objective assessment of risk on a

project proposal. The smaller firms often did not have sufficient staff for a completely

independent review, and respondents from these smaller firms were clearly aware of the risk

of underestimation of schedule and budget resulting from this potential conflict of interest:

[H1] “You really want to separate the people who are doing the risk assessment from the people who are in any way involved or have too much on the line for the project itself. It’s very hard to do that, especially in a small company where everyone basically wants the project at that point.”

At the extreme end of this conflict of interests were the two respondents from the

very small ‘boutique’ firm mentioned at the start of this section, who did no formal risk

assessment at the pre-sales stage. One of these two respondents was on the pre-sales team,

while the other was a project manager. Both alluded to the pressures of having to pare down

their bid in order to win the deal. For them, risk management was a costly luxury:

[Q (pre-sales)] “No, we leave [the formal risk assessment] for the project managers, but we have a rough consensus. We’re more worried about the risk of losing the deal rather than the risk for the project”

[K (PM)] “I guess for the norm these days, there is not that much risk management in software implementation projects. It’s not a good thing, but the basic reason stems from money.”

Even in this firm, however, there was an implicit attention to risk. Typically the

project manager likely to be assigned to the project would be consulted, and careful attention

was paid to management of the customer’s expectations, in order to ensure the customer

understood what was achievable on a low budget:

[Q] “We as pre-sales people, we do have concern about what we propose, whether it’s going to be feasible and reasonable.”

[K] “We would try to work on the scope and make sure the customer understands that for that amount of money you would not be able to do what you want to do. Keep nagging to get them to understand what is feasible and what is not feasible and why it is not feasible.”

In-house respondents. The three in-house respondents reported a more mixed

approach to pre-project risk assessment. The in-house project manager from a large

commercial firm indicated that his firm typically did not carry out risk assessment for their in-

house projects, simply regarding the management of risk as part of project management in

122

general. For out-sourced projects, his firm did assess risk in terms of evaluating the vendors,

but not in terms of the project itself:

[P] “When we select a vendor, we evaluate their company profile, all their track record, and so on - you can say it’s part of the risk management. But we don’t put the risk as one of the main topics or one of the main areas we need to concentrate on for in-house, or even in the out-sourced projects. We do it because we’ve got to do the project management and then the risk is automatically included as part of the project management.”

The two project managers from government organizations also regarded risk

management as something that simply followed from the project management methodology

adopted in their organization. These project managers both used the PRINCE methodology

(the UK government standard for IT project management) for managing projects, and while

one manager described a rigorous risk assessment using the PRINCE methodology at the

project proposal stage in order to gain approval for an in-house project, the other manager,

who specialized in managing outsourced projects, relied on detailed requirements and

schedule specification to mitigate against the risk of poor vendor performance:

[V1] “At the start, we didn’t have the formal term of risk assessment, but we knew that delay and quality of work were usually the problems of that kind of work, and that’s why in the RFP stage, we clearly spelled out the contents that were required, and we also set the milestones for delivery. And …after the deliverable of one milestone has been accepted by [us] they can proceed onto the next milestone. That kind of milestone serves as a safeguard of the quality of the product.”

5.2.2. Risk management practices at project start-up stage

It was surprisingly difficult to find evidence that respondents either revisited the pre-

sales risk checklist, or made their own independent risk assessment when they actually started

the project work. Instead, with the exception of two types of risk discussed next, managers

appeared to rely on the contingencies already built into the schedule by the pre-sales team,

rather than taking pro-active actions to mitigate any potential problems. Indeed as one

manager commented there seemed to be a substantial gap between the risk assessment plan

and the execution of risk mitigation actions:

[C] “The risk assessment process is not strong enough. I’ve gone through quite a lot of proposals before, and there’s always a section for risk mitigation, so you identify what you see about this project as potential problems, and what you would do … But when it comes to executing that mitigation … but there’s no specific requirement that somebody really has to go through each one of these mitigation steps …”

123

Two risks, schedule and technical, did appear to alert the respondents to take further

mitigating action, and the strategies described to address these risks are discussed in Section

5.3.2.

Managers of ‘troubled’ projects were the exception here, as they typically did a very

thorough analysis of their ‘troubled’ projects, and what had happened to date. Interestingly,

these managers did not describe such a thorough approach for those projects that they ran

from the beginning. However, in their ‘troubled’ projects the managers carried out their own

in-depth investigations of the project documentation and the present project situation,

although they still made no reference to any original pre-sales risk assessment that may have

been done. The investigations described by these ‘troubled’ project managers, for the most

part, took a rational approach of prioritizing the most critical tasks based on the probability

and impact of failure to complete the task on time, followed by a evaluation of options for

mitigating or eliminating the risk of missing the required deadlines. These differences in

approach to risk assessment at the project start-up stage will be considered in more detail in

the discussion on Research Question 5 in Chapter 7.

5.2.3. Risk management practices during the course of the project

While the small and mid-sized firms relied on the project manager’s skills for risk

management during the course of the project, the larger firms had more formal processes in

this regard, with regular checks of project status being conducted on a one or three monthly

basis by the quality assurance team. However, these project ‘health’ checks seemed to focus

mainly on whether the project had so far run according to the targeted schedule and budget,

rather than looking ahead at what potential problems could still arise during the remaining

part of the project. And in the opinion of one project manager, the ‘health’ checks occurred

too late in the project to be of real value in preventing problems arising:

[C] “But that [‘health’ check] is 3 months after the project starts, and a lot of time, they already have a lot of work done, or if the direction is wrong, they’ve already gone in the wrong direction.”

5.2.4. Risk management practices after project completion

Most respondents reported some form of post-project review or evaluation, but

typically this review appeared to be intended for a senior executive audience, rather than to

spread information about the experience gained among other project management staff

124

within the firm. Overall, there appeared to be little attention paid to the value to be gained

from encouraging more experience-sharing among project management staff, and this seems

to be an opportunity that firms could easily take advantage of.

5.2.5. Comparison of risk management practices with prescribed processes

The general risk assessment processes carried out in the pre-sales stage conformed

closely to the practices recommended in the prescriptive IT risk management literature, as

reviewed earlier in Section 2.2.3 and illustrated by Figure 2.1. However, as discussed later in

Section 5.3 on strategies applied by respondents during their projects, there was little

evidence of active execution of risk management plans by managers during the implementation

stage of projects.

In the following sub-sections, I compare the practices detailed in the preceding

section with each of the four major processes in Figure 2.1, namely risk identification; risk

assessment; risk response planning; and risk monitoring. I then discuss two aspects of the

literature, the contingency approach to risk management and post-project reviews, that were

not clearly reflected in the practice described by respondents, followed by some areas of

practice described by respondents that are not well covered by the literature prescriptions, in

particular, separation of risk assessment duties and handover from pre-sales to

implementation.

Risk identification: The respondents described a rational approach to identifying

risks for a particular project proposal, using some form of formal or informal checklist to

ensure that all key risk areas have been considered. However, this approach appeared to be

more strongly established among the vendor IT firms than in the in-house organizations,

with in-house respondents reporting a more mixed approach, paying more attention to risk

when out-sourcing than when planning an in-house project. The ‘softer’ risk areas of client

relationship management and communication were not well covered in the checklists, and

while it is tempting to recommend that firms pay more attention to these areas in their risk

assessment processes, potential problems in these areas were typically not apparent at this

early stage of the projects.

Risk assessment: Risks identified were assessed and ranked. The ranking processes

used to prioritize the risks required an implicit assessment of the importance of the risk, but

125

since there was typically little or no requirement to explicitly consider separately the impact

and probability of occurrence, it is possible that, as Pablo (1999) observed, the pre-sales team

focused more on the impact than on the likelihood of the risk when making their

assessments. Further research is needed into this aspect of risk assessment in IT projects.

Risk response planning: The risk response planning phase of the pre-sales risk

assessment process focused almost exclusively on the addition of contingency to the

proposed schedule and budget. This differs from the range of possible risk responses

recommended in the literature, which includes taking pro-active actions to eliminate or

reduce risks that have been identified, as well as planning contingent actions to address risks

that might arise, and that cannot be easily mitigated in advance (Boehm, 1991; Powell &

Klein, 1996). Many of the risks identified at the pre-sales stage related to high levels of

uncertainty in terms of new technologies, unclear requirements, or complexity of the project.

High uncertainty in these areas can be mitigated by further study or research at the pre-sales

stage, but as noted previously in Section 5.5 such detailed study is costly. Both vendor and in-

house respondents reported that their firms were reluctant to invest too much at the pre-

project stage when there was a risk that, for vendors, their bid might not be successful, and

for in-house departments, that the project proposal might not be approved. Even when

respondents noted that a particular risk had been identified that was very amenable to

reduction by further detailed investigation, for example, unclear requirements or unfamiliar

technology, the response to this risk was typically to defer further investigation to the

implementation stage by adding more time into the proposed schedule, thus minimizing

costs at pre-project stage.

Risk monitoring: Risk monitoring practices during the course of project

implementation were less formalized, especially for the small and mid-sized firms, which

tended to rely exclusively on the respondents’ detailed project management skills and general

situation awareness to alert them to any problems that might arise and to ensure successful

containment of risks during the course of the project. Larger firms did use more formal

checkpoints to review project progress, but respondents suggested these were not effective in

ensuring a project stayed on track since they typically occurred too late in the project, and

also tended to be backward-looking, at what had already happened, rather than forward-

looking, at what problems might still occur. With the exception of specific technical issues,

respondents did not appear to use the pre-sales risk assessment to warn them about specific

126

potential problems that they should watch out for. Instead, managers focused their attention

on a detailed analysis and monitoring of the proposed schedule and work breakdown

structure to ensure time and budget targets were met. Since the most typical form of risk

mitigation at the pre-sales stage was the addition of extra time and budget as a contingency

measure, it could be argued that there was little else for the project manager to do in terms of

risk monitoring, so long as the schedule and budget already included adequate contingency

allowances.

Contingency approach to risk management: As noted earlier in Section 2.2.3,

some writers in the prescriptive risk management strand of the literature (see, for example,

Barki et al., 2001; Davis, 1982; McFarlan, 1981) have advocated the use of a contingency

approach to risk management, where the type and level of risk management approach, and in

particular the development methodology, is matched to the level of risk exposure identified

in the project. In the present study, respondents commented that the size of contingency

added to the schedule and budget was clearly determined by the degree of risk assessed for a

given project. Respondents from the larger firms, with their more formal reviews of project

progress, also noted that high risk projects were subject to more frequent scrutiny by the

quality assurance team. However, I found no evidence that respondents varied their risk

management approach or their development methodology depending on the level of risk

exposure identified in their projects. Generally, as discussed next in Section 5.3 on strategies,

respondents applied the same management strategies for all their projects, regardless of how

risky those projects were perceived to be.

Post-project reviews: While general evaluation and review of project performance is

typically recommended in the project management literature as the final stage in a project’s

life cycle (PMI Standards Committee, 1996), specific review of the risk management

performance of projects has not been identified as an important activity. In the literature on

tacit knowledge, however, as discussed earlier in Section 2.3.4, informal experience sharing

through story-telling, ‘war stories’, and mentoring is recommended as a way of capturing and

sharing tacit knowledge (Nonaka, 1994; Swap et al., 2001). While most respondents reported

some form of formal post-project reviews typically reported to senior management, they did

not mention any requirement to specifically evaluate issues related to the management of risk

in the project. Most of the respondents, when asked about less formal opportunities within

their firms for experience sharing and knowledge transfer among project manager peers,

127

replied that they had not had such opportunities themselves, although some indicated that

they were trying to initiate mentoring relationships with junior staff. One of the largest firms

was establishing a more formal mentoring system to improve knowledge transfer to new

project managers, but it was too early to assess the success of this approach.

This lack of specific attention to the opportunity to capture tacit knowledge from

risk management experiences, both in the project management literature and in practice,

highlights an area worthy of further investigation. The collective experience of project

managers within a firm is potentially a very valuable source of tacit knowledge in terms of

training staff about possible problems they may encounter in their projects, and in identifying

strategies to address these problems. Even though these organizations had such extensive

stores of knowledge on hand, only a few firms appeared to recognize the importance of

encouraging this experience-sharing and knowledge transfer among their staff.

Separation of risk assessment from pre-sales and implementation teams: All of

the vendor respondents were clearly aware of the potential risk for projects that arises from

the risk assessment process itself i.e. a conflict of interest can arise if those doing the risk

assessment have some kind of vested interest in the project, whether because they are

anxious to meet sales targets, or because they are keen to actually work on a particular

project. Such a conflict of interest can lead to a serious underestimation of risk on a project

or a reluctance to add a realistic level of contingency to the proposed schedule and budget.

Surprisingly, this risk has not been highlighted in the literature as an area of concern. The

typical mitigation for this risk, separation of duties, is a standard approach to addressing risks

in many other management areas (for example, to reduce risk of employee theft or fraud) so

perhaps it has been regarded as so obvious that it does not warrant specific attention.

However, respondents were both aware of and concerned about the problems that could

arise from having risk assessments completed by staff who potentially had an emotional

commitment to the project, and strategies to address this risk will be discussed later in

Section 5.3.1.

Hand-over from pre-sales to implementation: There appeared to be a significant

gap, which has also not been addressed in the literature, in the hand-over from the pre-sales

team to the implementation project manager, with little evidence that project managers

utilized or checked the risk assessment work done during the pre-sales stage when preparing

128

the detailed plans for the project implementation. Many of the respondents did not regard

risk assessment as a separate and on-going project activity. Instead, it appeared to be

something done once at pre-sales stage and then almost forgotten about. This is perhaps not

so surprising when set against the context of the typical project management training

literature (see, for example, PMI Standards Committee, 1996; Yeates & Cadle, 1996). Risk

management is presented in literature such as this in a separate section as a process that must

be completed at the beginning of a project. While the risk monitoring process is identified as

something that should continue throughout the project, the remaining sections focus on

operational aspects of managing the project and rarely refer again to on-going risk

monitoring.

The separation of risk assessment duties described in the previous sub-section, while

addressing the conflict of interest risk, actually compounds the problem of ensuring on-going

risk monitoring. The implementation project manager has not been responsible for

identifying the risks and planning responses, so it is likely that s/he feels little ownership of

the risk management plan. Moreover, since the typical risk response is to add contingencies,

rather than plan pro-active actions, there might seem to be little else to be done unless and

until a particular risk surfaces as an actual problem. If such a problem does occur, then the

contingent action – more time and/or budget – has already been planned for and can simply

be applied.

To some extent this lack of attention to specific risks identified during the pre-project

assessment process may have arisen because once the project had started, respondents

regarded risk management as an implicit part of their day-to-day project management

strategy, rather than as a separate activity required by the firm’s senior executive. As I discuss

further in Section 5.3 next, managers typically employed a few broad strategies designed to

address a very wide range of risks, and thus implicitly they may be well prepared for whatever

problems arise, even though they have not taken explicit steps to manage the specific risks

identified in the pre-sales risk assessment.

5.3. Research Question 3: Risk management strategies

Research Question 3 asked: What strategies do project managers use to address risks

in IT projects – both to prevent them causing problems in the first place and to deal with the

problems if they do arise (i.e. if the risks identified as potential problems turn into actual

129

problems)? As set out in Section 3.4, in the Description stage of analysis I extracted passages

describing specific problem situations addressed by respondents during their projects. In

total, as shown in Table 5.1, I identified and analyzed 94 situations, in 47 different projects.

Four of these situations described pre-sales strategies, 23 occurred in ‘troubled’ projects, and

the remaining 67 related to projects that the respondents had managed from inception. In 45

of the situations, project managers described actions they took when they started work on

the project, either at the pre-sales stage, at project start-up, or when they took over as a

rescuer of a ‘troubled’ project. The remaining 49 situations related to actions taken to address

problems that arose during the progress of the project.

Table 5.1: Problem situations

Pre-sales projects

Projects managed from inception

‘Troubled’ projects TOTALS

Initial assessment situations

4 (4%) 27 (29%) 14 (15%) 45 (48%)

Problem arising situations

40 (43%) 9 (9%) 49 (52%)

TOTALS 4 (4%) 67 (72%) 23 (24%) 94

In addition to the specific situations described by respondents, I also scrutinized their

general comments for descriptions of strategies they applied to manage their projects. These

general comments particularly related to pre-sales and start-up strategies, as respondents had

generally developed heuristic strategies which they applied to all their projects.

Next, in Section 5.3.1, I discuss the strategies described by respondents to address

risks emanating from the risk management practices used at the pre-sales stage. Then, in

Section 5.3.2, I develop a general framework categorizing the strategies used by respondents

to manage their projects into four groups, linked to the broad risk themes discussed in

Chapter 5. Using this framework, I discuss specific strategies applied by respondents in each

of the four groups in Sections 5.3.3 to 5.3.6. Finally, in Section 5.3.7, I compare the

framework and the strategies applied by respondents with recommendations from the

literature on risk management.

5.3.1. Strategies to manage risks emanating from pre-sales practices

The pre-sales stage of vendor-driven IT projects has attracted little attention in the

literature, and yet in the view of the critical incidents described by present respondents this

130

stage is potentially a source of substantial risk for the implementation project manager. Two

issues in the pre-sales stage were identified by respondents. The first pre-sales issue

mentioned was the tension between the need of the pre-sales team to win the sale and the

necessity to keep the customer’s expectations to a level and within the scope that could be

achieved and delivered. The risk here is of over-selling by the pre-sales team: an often subtle

encouraging of the customer to expect a higher level of delivery than is achievable or even

budgeted for in the cost/time/effort estimates.

[E1] “There’s definitely an issue in that before a project … you give a sample, and you give your best, you give your absolute best because you’re selling. And then when you’re in the project you don’t really need [that] level … but the expectation has been set and we’ve set it, and we need to now live up to that. [We’re in] this funny circle of having to deliver something we promised irrespective of it being necessary.”

Respondents identified two strategies to deal with this problem. Firstly, the pre-sales

team were encouraged to seek input from the implementation staff on the proposed solution,

budget and schedule. While this input usually helped to ensure that the bid was well-founded,

the implementation team often had limited time to be involved in pre-sales work since they

were working on other projects. Thus they could only contribute a superficial assessment of

the proposal, and frequently missed potential problem areas. Further, some respondents

described situations in which the implementation staff got caught up in the excitement and

interest of a new project, particularly if it involved interesting technical challenges. In these

situations the desire of the implementation team to be able to work on the project

outweighed their objectivity:

[H1] “We wanted the project, we wanted to do it, we thought - a lot of it was rosy coloured glasses - we thought, we can make this work.”

The second strategy used to manage potential over-promising at pre-sales stage was

to use an independent risk assessment of the project proposal, as described above in Section

5.2.1. This independent assessment was considered by respondents to be helpful in guarding

against the more obvious excesses of over-selling. However, as noted in Section 5.5.1, even

in the largest firms where the quality assurance and risk assessment process was very well

established, this independent assessment was not always successful in ensuring that an

achievable schedule and budget was set. Respondents attributed this failure to the second

major issue at pre-sales stage, which was the tension between the need to accurately assess

the client’s requirements and the need to minimize the cost of pre-sales activities.

131

[E1] “We definitely have a decision internally about how much we’re willing to spend on the pre-sale and in this instance a lot more spent on the pre-sale would have saved us a lot on the project. But we didn’t do that because we took our standard approach - this is enough detail for the pre-sale.”

If requirements were misunderstood or inadequately defined at the pre-sales stage,

then the schedule and budget would be under-estimated.

[F] “Normally when we bid on a project, we need to calculate our estimated costs and we need to keep the price very competitive and try to minimize all the effort. But the world is not that easy and once we start a project …, when we study more about the requirements we find out that it is getting much bigger than what we assumed, so that’s why the resources for this kind of project may be not enough.”

Resolution of this second issue was seen by most respondents as a judgment call on

the part of the senior management, since decisions on whether to approve proposed project

bids were made at this level.

[X] “The decision to take on the contract is made by the senior management, who consider not only the project itself but also the bigger picture of the internal resource etc. available at that time.”

However two of the small to mid-sized firms took a more pro-active approach.

These firms employed a tactic of separating the requirements specification part of pre-sales

work into a stand-alone, chargeable consulting activity. Thus potential clients were

encouraged to employ the vendor in the first instance simply to develop a complete

requirements specification in preparation for calling for quotations for the project. This

strategy enabled these firms to gain revenue from the selling process itself and to influence

the development of the requirements specification to align it with their products. The process

of developing the requirements specification also enabled the firms to build a strong

relationship with the client which often worked in their favour during the subsequent bidding

process for the implementation project.

[X] “The advantage is [we] get to influence the scope and specifications, and also establish a relationship and reputation with the customer which often gives us an advantage when the contract is awarded.”

Both firms reported better customer relationships and ultimately improved project

outcomes as a result of this approach:

[H] “So by the time we start doing any real work on the system we’ve already got 3 levels of their organization involved in it, and we’ve got buy-in from all of them.”

132

The success reported by these two firms with this pre-project partnering approach

provides support for previous findings from Jiang, Klein et al. (2002), who investigated the

impact of pre-project partnering on subsequent project performance and reported that pre-

project activities led to more effective project team performance.

5.3.2. General categorization of risk management strategies

When focusing on the start-up and implementation stages of their projects,

respondents generally described broad strategies that they applied to cover a wide range of

risks, and as mentioned earlier in Section 5.3.2, their project management strategies were

intertwined with their risk perceptions and frequently indicated an implicit assessment of

particular risks. Thus, in order to determine which risks a particular strategy was being

applied to, I cross-referenced the situation description passages with the risk framework

coding described in Chapter 5, and also considered the wider context of the projects being

discussed. Then, as shown in Table 5.2, I grouped the strategies identified from the situations

and from the respondents’ general comments into categories relating to the areas of risk that

they were intended, either explicitly or implicitly, to address. As can be seen from Table 5.2,

the strategies fell into four categories, for the most part directed at quite different sets of risk.

Strategies in the first category, Control, were typically applied in those situations that

the project managers saw to be directly within their locus of control, and were mainly

described in situations and contexts relating to the risks grouped in the Project Management

risk theme, namely vendor staffing resources, change management, schedule and budget, documentation and

client sign-off control. Control of these risks was seen as being at the heart of a project manager’s

skill set, and as discussed below in Section 5.3.3, strong management of the schedule and

budget was seen as the foundation for project success. The Environment risk, multiple

sites/countries, was also partly addressed by control strategies, since close management of

logistical issues was seen to be paramount in addressing this risk.

The second strategy category, Negotiation, describes strategies used in situations seen

by the respondents as only partly within their locus of control. Success in these situations was

dependent not only on the project manager’s skills but also on the willingness of other parties

to work together towards a successful outcome. These situations were typically the ‘soft’

situations involving inter-personal interactions and relationship issues. Such situations could

not be addressed by a simple application of a control strategy as they required co-operation

133

from other parties, and such co-operation had to be negotiated and carefully managed. Change

management appeared in both the Control and the Negotiation strategy categories because while

respondents clearly considered it important to exercise very close control over change

requests, they also recognized the inevitability of requirements change and the need to

negotiate with the client a satisfactory solution for requested changes:

[V2] “There must be changes and we need to accommodate that”

Thus the change management risk was addressed both with strong Control strategies and

also with careful Negotiation strategies as discussed below in Sections 5.3.3 and 5.3.4.

Table 5.2: Strategies and risks they addressed

Strategy type

Broad risk theme

Control Negotiation Research Monitoring

Project management

- Vendor staffing resources - Change management - Schedule & budget - 3rd party deliverables control - Client sign-off control

- Change management - Client project management - Client staffing resources - 3rd party staffing resources

Relationships – external (client & third party)

- Expectation - Trust - Bad news - Cooperation

- Top management support

Relationships – internal (vendor)

- Internal support - Team morale

Technology & Solution

- All technology risks - Unclear requirements and/or functionality

- Development choice

Environment - Multiple sites/ countries

- Non-local and multiple 3rd parties

- Client readiness - Multiple sites/ countries - Organization culture -Vendor reputation - Competition - Contract terms & conditions - Client business changes

The third category of strategies, Research, like the Control strategies, was applied in

situations that the respondents appeared to regard fully within their control, namely situations

134

that required further information in order to determine what action to take. In particular,

these strategies were applied when technological risks such as newness or compatibility had been

identified, and also when there was a lack of vendor understanding of the client

requirements. In some ways these situations were the ones that the managers felt most

confident about, because if problems arose in these areas they could be resolved simply by

getting more information. If the new information revealed that the problem was insoluble,

then at least they had a definitive answer and could either look for a different solution, or

escalate the problem upwards for resolution at executive level.

Finally, for a large number of risks, respondents appeared to use no mitigating

strategies at all, and instead adopted a Monitoring strategy. Most of these risks fell into the

Environment risk theme. Environmental risks were potential problems that respondents

were aware of that formed the context of the project, and that could influence how they

approached a particular project situation, but they were not risks that managers actively

attempted to address or change. They formed part of the environment of the project and

provided situational cues that coloured the respondents’ approach to managing the project.

The Monitoring strategy was also used to address one risk each from the Relationship and

Technology/Solution risk themes, top management support and development choice respectively.

Respondents appeared to view these two risks as non-negotiable parts of the project

environment that they had little control or influence over, and, just like contract terms and

conditions, something to be accepted and managed. As noted earlier in Section 5.4.2, neither

of these two risks was ranked highly by project managers.

Table 5.3 below shows the numbers of specific situations describing the use of the

different strategies. Note that the specific situations did not include specific descriptions of

the Monitoring strategy. Evidence for this strategy came from the analysis of respondents’

general comments, as well as a close reading of the cues they took account of when applying

Control, Negotiation, or Research strategies.

Table 5.3: Strategies used in initial assessment and problem arising situations

Initial assessment situations

Problem arising situations

TOTALS

Control 26 (29%) 12 (13%) 38 (42%) Negotiation 8 (9%) 35 (39%) 43 (48%) Research 7 (8%) 2 (2%) 9 (10%) TOTALS 41 (46%) 49 (54%) 90

135

As discussed below, Control strategies tended to dominate in the initial assessment

stages of projects, while Negotiation strategies were more prominent when problems arose

during the course of projects. In the following sub-sections, I describe the strategies in detail.

5.3.3. Control strategies

The Control strategies described by respondents were typically proactive actions that

were initiated at the beginning of the projects, and continued to be applied throughout the

projects in order to maintain control and meet targets. The key Control strategy, described in

26 of the 41 initial assessment situations, and in 12 of the 49 problem-arising situations, was

an exhaustive analysis and control of the work breakdown structure. This strategy was relied

on by respondents to generally mitigate the top five most commonly anticipated risks shown

previously in Table 5.7 (schedule and budget, vendor staffing, newness of technology, multiple

sites/countries and vendor understanding of requirements). Respondents paid particular attention to

the development of the project plan and work breakdown structure as the lynch-pin of their

strategy.

Work breakdown structure: The typical Control approach described by respondents

at the start of their projects was to focus intensively on the work breakdown structure (WBS)

of the project, with the aim of developing a very detailed task breakdown. This focus had the

implicit side effect of helping the manager to develop a detailed understanding of the client

requirements and the proposed solution and technology for the project. The required vendor

staff skill-set was established from this analysis and technical and logistical requirements were

revealed. Hence the project manager established an in-depth understanding and control of

the project and its potential risk areas without actually assessing risks explicitly.

The risk that most managers were concerned about at this start-up stage of their

projects was an over-ambitious schedule, and where this risk was identified, managers would

quickly move to mitigate the risk by alerting their senior executive and attempting to

negotiate internally for more resources. Here, managers’ motives appeared to be related both

to protecting themselves in the event that the timeline targets were not achieved, as well as to

concern about the project itself going over time or budget. Respondents from vendor firms

were aware that if they made a strong enough case they could sometimes negotiate for a

particular project to be run on a lesser profit margin than the standard one applied in their

firm, thus allowing them to draw in more staff resources to meet difficult schedules.

136

However, more often, respondents recognized that commercial realities presented limitations

on the size of budget and schedule and acknowledged the expectation that they would use

their project management skills to meet the required targets.

[N] “The project management aspect starts in, and that’s to manage that [schedule] risk in the project.”

While development of a WBS is a very basic part of any formal project management

training course, respondents felt that such courses did not provide the understanding of how

such structures should be developed, in terms of the level of detail required. Respondents

reported a rule of thumb of aiming to break down tasks to units of one week, because that

was the maximum practical controllable unit:

[A] “A good project plan should have the WBS broken down to a point where every task should not be more than 40 hours. Because once you go beyond that, you just cannot manage it… If it’s just one man-week, you can manage that, and you have … a quantifiable, manageable piece of work that this person cannot [lie] to me … don’t wriggle, give me the story! Only by doing that you are able to manage the project properly.”

Progress control. Once the WBS was well-established, the second Control strategy

involved very close monitoring and control of each task’s progress to identify any slippage.

Respondents regarded the first two to four weeks as critical for establishing early and

effective control of the project and identified the management and control of the WBS as the

single most crucial activity. Problems that arose in the projects even after being anticipated

when the original WBS was developed were most often the top five listed in Table 5.7 in the

previous chapter (schedule and budget, vendor staffing, newness of technology, multiple sites/countries,

vendor understanding of requirements). These problems were seen to be almost always caused by

underlying inadequacies in the original schedule and budget. Even if the immediate problem

was a shortfall in vendor staffing, or unexpected problems with technology, or logistical

difficulties with multiple sites, or more complicated requirements than originally assumed, the

respondents saw the fundamental issue as one of insufficient time or budget. With more time

(man-hours), and hence more budget, these issues would not be problems.

This view of the inter-relatedness of project problems helps to explain the

respondents’ reliance on detailed project planning and monitoring as the key risk prevention

strategy. If most of the key problems have their source in inadequate schedule and budget,

then they will also immediately show an impact in this area. Thus by focusing on

137

performance against a detailed work breakdown structure managers both maximize their

ability to control their projects and also ensure the earliest possible warning of any problems

arising.

As with the development of the WBS, while monitoring and control of progress is

also a very basic component of standard project management courses, respondents felt that

such courses typically did not fully convey how this monitoring and control should be done.

In their view, the recommendation of such courses that the project team report on progress

on a weekly basis was only the starting point. At each reporting session, any tasks still in

progress were seen as requiring very particular attention. Respondents described a process of

probing deeply into weekly progress reports to uncover exactly how team members had

arrived at their estimates of task completion, and to ensure that they could explain and justify

their claims of a certain percentage completion of a particular task.

[A] “You ask [your team] how far are they in a task and they say, 80%. So what made you say it’s 80%, why not 60% why not 85%? People always give you the easy answer, they want to walk away because you are putting them under the spotlight. So you say, when do you think you’ll have that remaining 20% done - oh by the end of the week, and you can check. … Why do you think you can do the rest in 4 days?”

Managers recognized that this strategy was often difficult to apply as it could make

team members uncomfortable, but they were emphatic that failure to persevere was likely to

result in unidentified problems. Manager A. described his initial reluctance to probe so

insistently and explained the need to develop interpersonal skills that would allow him to get

the information he required without giving offence:

[A] “So a lot of time, when someone gives you an easy answer, you take the easy answer and let the difficult question go away. That’s how I was when I was younger … But we can say this in a very courteous manner, and before you ask the question, you ask yourself, would a mature adult ask such a question? Would a responsible person on that particular task be able to give you a good answer? If you say yes, you ask the question!”

Managers with experience of working with sub-contractors on projects highlighted

the importance of ensuring that they were subject to the same detailed reporting

requirements:

[N] “Whether it’s your own development staff, or whether it’s sub-contractors, and whether it’s development or you’re running through the test cycle, you don’t just accept somebody’s word for it. If you’re monitoring a building and someone says we’re at the 50th floor, you can look up and see that.

138

But if you never showed up on site, you wouldn’t know. You have to inspect, you have to say show me, and you use your own yardstick to assess with.”

Change control. While close management of the WBS was generally seen as the

most fundamental Control strategy for the management of a project and its risks, managers

also recognized that, in order to achieve the schedule and budget targets underpinning the

WBS, clients’ change requests had to be tightly controlled. This typically involved application

of change control clauses in the contract and while use of these clauses appears on the

surface to be a simple Control strategy, respondents were always aware of the underlying

interaction and negotiation with the clients. Hence change control strategies are discussed

further in the next section on Negotiation strategies. In two types of situations, however,

respondents clearly saw change control procedures as a means of enforcing control in their

projects.

Firstly, when faced with particularly difficult schedules and budgets, respondents

typically took a Control approach to the use of the change control clauses in the contract in

order to ensure that they met their targets. Indeed, two managers reported that, with astute

use of these clauses, they were able to negotiate additional time or budget and thus alleviate

some of the deadline pressure in their projects. These managers reported that this approach

enabled them to turn very tight projects into more profitable ones for their vendor firms:

[N2] “We actually got 25% more revenue through change control.”

[S1] “Enhancements helped the gross profit because the additional time and money helps to re-baseline the cost and schedule.”

Secondly, in-house managers of outsourced contracts welcomed the enforcement of

change control clauses on their users by the vendor firm. These managers had long and

frustrating experience of having to acquiesce to continuous user requirements changes on in-

house projects, with little or no support from senior management to limit or control these

requests.

[P1] “For the in-house development users usually don’t spend extra effort on specifying. Because they always think they can change it later, so the user requirement is always more difficult to control. … You see if they really want to change it they can go up to the big boss and come down (to us). Because in-house we can’t use the contract to ask them to pay more.”

[B] “I’ve tried many ways. Sometimes I try to ask the users to document their requirements. And then we design based on this, but at the end they still want to change it, and I have to give up!

139

There’s nothing we can do, because we are making systems for the user. If the user does not want it, then the system is useless.”

On out-sourced projects, however, these managers supported the external

contractors in enforcing a disciplined change control approach on the users, and noted that

this experience had a positive spin-off in subsequent projects:

[B] “So sometimes we’re going to outsource the project, then they have difficulty, because the external provider will charge [us]. So they have a lot of discussion, and the [users] don’t clearly spell out the requirements in the tender document, but the external providers want the requirements spelled clearly. So for one of the projects I was only an IT advisor, and the users asked me, is it in the contract for the implementation stage, if we add some reports, the contractor says [we] have to pay, and the user comes to me and says, is this reasonable? And I say, if it’s not in the tender then you have to pay. Now they’re very cautious about the requirements in the tender.”

Documentation. There was general agreement that, however the issue of change

control was approached, documentation was essential. Even when the contract change

control terms were waived, respondents emphasized that it was still important to record

client change requests that had been accommodated in case disputes arose at a later stage.

This strategy was viewed as a very basic part of project management discipline, yet as

indicated by four of the managers of ‘troubled’ projects, failure to document agreed changes

could lead to major difficulties later in the project.

[P2] “He [previous project manager] did not follow the change request rules. Number one rule is whenever there’s change in price, change in resource, change in timeline, you have to document it and that’s then agreed. But it wasn’t there. I found out later on there’s been trading [substituting a new requirement for an old one that is no longer needed], verbal commitments, no confirmation on documentation. This turned out to be really bad … [a requirement] was traded for something else, and I cannot produce a document to tell them, you already traded that for that one we’ve already given you, but no document, no agreement, nothing on that, and I’m stuck with something that hasn’t been delivered …”

Some managers personally controlled documentation such as meeting minutes in

order to position themselves to advantage if disputes arose later:

[W1] “And then on minutes, I will try to do it myself, because that will benefit, although I will have additional work, but for minutes, once I write the minutes, of course I will try to put it on my side, to protect me. Although there will be more paperwork, but it is worthwhile to do it.”

Finally in this section, if a schedule/budget problem arose during a project, the

managers typically first attempted to correct the problem with one of the following two

reactive Control strategies.

140

Increasing available man-hours within the schedule time-line. Once a

schedule/budget problem arose during a project, the first defence was always a Control

strategy of increasing the over-time hours worked by the project team. Since team members

were usually paid on salary, extending their hours of work effectively increased the man-

hours available to meet the schedule without increasing costs at all. Of course regular use of

this approach had a negative impact on staff morale, and I discuss the managers’ strategies

for maintaining morale in the next section on Negotiation strategies.

Rescheduling and problem isolation. A second Control strategy for dealing with

schedule/budget problems involved isolating the issue causing the problems, and re-

scheduling remaining work so that progress could continue to be made. This approach

allowed work to resolve the problem to carry on in parallel with other tasks so that the

problem did not impact on the overall schedule.

[I2] “In terms of expectations for completion of the project, I saw these items are not critical issues for now, so let’s move them down so they do not paralyze the whole project - we must keep it going forward. Now you’ve contained the problem, you’re addressing the regular production issues, now how will we address that problem?”

When Control strategies were not sufficient to correct schedule/budget problems,

managers turned to Negotiation strategies as discussed next.

5.3.4. Negotiation strategies

Negotiation strategies were discussed by respondents in eight of the 41 initial

assessment situations and in 35 of the 49 problem-arising situations (see Table 5.3). The three

key external Negotiation strategies, change control, trust and relationship building, and

managing client expectations, were applied at the start of projects to proactively control client

expectations and to establish a good working relationship with the client, and throughout the

project both to maintain the relationship and to attempt to correct a situation when things

were going wrong. Internally, managers used similar skills to negotiate the support they

needed in order achieve their targets, and applied strong team building skills to build and

maintain commitment within their teams.

While change control, relationship building, and expectation management strategies

were always initiated at the start of projects, most managers mentioned Negotiation strategies

in the context of addressing problems that arose during their projects. As discussed

141

previously in Section 5.5, problems that arose during the course of the projects fell into two

categories, those that had been anticipated but still occurred in spite of any mitigating actions

and those that were unanticipated. General communication and negotiation skills

underpinned the respondents’ approaches in all these situations, and particularly in

addressing problems that arose as a result of unexpected difficulties with external

relationships with clients and third parties, and also where the respondents encountered

problems gaining required levels of support internally.

In addition to the three key Negotiation strategies mentioned in the previous

paragraph, three other Negotiation strategies were identified. Firstly the strategy of balancing

cost, schedule and scope with the client was used as a second line of defence if Control

strategies had failed to address the schedule/budget inadequacies underlying problems that

were anticipated and still arose. Secondly, where the unexpected problems related to staff

morale, various team building strategies were applied. Thirdly, an escalation strategy was the

fall-back position if the manager had exhausted all other strategies in his/her repertoire. I

discuss all six of these strategies in more detail next.

Change control. There was general agreement among respondents that it was

important to establish limits or boundaries on the customer expectations from an early stage

in the project, and their approach typically involved application of change control clauses in

contracts. However, managers differed in their views on the best way to approach change

control. Some managers believed that it was important to start out with an insistence on

applying strict change control procedures from the very beginning of the project. These

managers felt that an initial ‘softer’ stance could result in a loss of control that was difficult to

recover from:

[C] “The contract interpretation has to be really firm and I think from day one you have to train up the customer that this is the way the contract should be handled. You cannot be too soft at day one because once you pass the first two months as a soft person, it’s very difficult in the later part to come back and say I’m very firm, because you cannot change your personality at a later stage.”

On the surface this approach appears to be an example of the application of a Control

strategy rather than a Negotiation strategy. However, even managers like C. were aware that

they were building and managing a relationship with their client, and that this involved

maintaining a delicate balance, as C. went on to explain:

142

[C] “So you have to keep yourself a little ‘all-rounded’ in how you deal with the relationship. You can’t be one way too harsh, or you don’t have room to go back and talk about something else with them, so you have to maintain a balance on that.”

Other managers argued for an initially more cooperative approach, especially where

requested changes could be accommodated without much extra effort. This way they

established an atmosphere of trust and support with their clients which they found invaluable

if the project encountered difficulties at later stages.

[S1] “Customer relations are very critical for change management - it’s important that they can see the performance of the team so they can understand. … In the early stages, they could see the team working on site day and night working on the project. The client could see they were very dedicated and committed, so [later on] the client was more willing to pay extra for changes.”

[H] “Part of it is goodwill … sometimes we have to charge them extra, but by that time we’ve already built up some goodwill … We’re not going to do so [many changes] that we’re going to lose money on the project, but we factor that in so that by the time we come to sit down with them and start saying, you know, you can see that we’re doing extra here - you know it, we know it, your team knows it, so let’s be reasonable.”

I probed more deeply with some of the respondents on the question of the type of

approach to use for change control, and found that, as C and H explain below, they tried to

assess their clients at the pre-sales stage in order to help determine the best approach for

change control:

[C] “So based on your previous pre-sales meetings you should have an understanding on the way you can work with the external party.”

[H] “If you see from the very beginning that they can be very unreasonable, and you know that it’s going to be a tense project later on, then what you start, psychologically, you get them in a position so that every time they ask for something, you’re very clear and very specific - you’re asking for this, this is not included, so that later when you want to be flexible you can. But if you start too flexible to begin with, with the troublesome client then you’re lost, we’ve already been through that, so where you start is with a very tight rein on that kind of client, very focused and very meticulous about everything, so you’re training them, it’s a process of you’re training them.”

Thus, while the change control strategy was always applied to manage client

expectations about requirements changes, the manner in which it was applied was influenced

by cues derived from applying a Monitoring strategy to observe risks arising from the client’s

organizational culture and general approach to the project.

Trust and relationship building. Managers described various general trust and

relationship building strategies underlying their whole project management approach, which

143

seemed to be intended to serve as a general mitigation for all the ‘soft’ relationship risks that

were generally seen as hard to anticipate and difficult to address if they arose during the

progress of the project. Respondents described how they tried to view the project from the

customer’s perspective in order to understand how best to establish trust between the parties:

[D] “I always try and sit with the customer, even on internal projects, with the other person’s hat on for a while and think, if that were me, what am I looking for, what do I want. I try to pre-empt that. … Because how can you present a properly thought out case if you haven’t thought about their side of it. … I try to suggest to the client, you should think about this on your side of the contract, you want to have this kind of control. It’s easier then to deal with the problems as they arise … Sometimes it creates trust if they see me looking out for them.”

[U1] “You have to put yourself in the shoes of the customer.”

[L] “The client is part of my team … he’s not in the opposite position, on the outside, he’s part of my team.”

Other managers described using a ‘softly-softly’ approach at the start of their projects

in order to build a strong foundation for their relationship with their clients:

[I2] “If they say at the beginning of the project, this is your responsibility, and in my head I’m saying no, that’s your responsibility, but at the beginning of the project when you want to get things moving, sometimes you have to accept, shut up, and say thank you very much and move on. … You might be right, but better to say a simple thank you. As you move on and the strength flows back to you, then you can say, no I’m sorry, this is not right. When you came in, you’re in dangerous ground to have objections for everything. So you must plant the seeds for success, understand your end goal.”

In-house managers of out-sourced projects also noted the importance of building a

strong and trusting relationship with the out-sourcing partner:

[P] “Nowadays we are actively pursuing the outsourcing and the relationship between us and the contractor is crucial in having a successful implementation of the projects, and trust, and confidence.”

In order to win the confidence of the client’s staff, one project manager described

how he tried to ‘go the extra mile’ in his interactions with his customers:

[U1] “The customer never wants to take responsibility. So why don’t I do more, because I know I can do a better job than them? So I will fit in the homework to them and let them pass the homework to their boss and get the credits, and they will help you, throughout the whole project.”

When managers were faced with restoring client trust and confidence in ‘troubled’

projects they used a simple strategy of seeking out some issues that could be resolved quickly

and that would have visible results:

144

[I2] “Go for the low-hanging fruit, let’s have some quick wins”

Managing client expectations. One key aspect of the change control and

relationship building strategies was the managers’ underlying objective of managing the client

expectations. As noted in Chapter 5, client expectations featured as a key issue in both

anticipated and unanticipated problems and also in ‘troubled’ projects, and ranked second

only to schedule and budget management in respondents’ concerns. As such, it deserves

separate consideration here, even though, for the most part, managers did not identify

specific strategies targeted to manage client expectations. Rather, part of the purpose of the

change control and relationship building strategies mentioned above was to set the client

expectations at a level that could be met. Typically, managers faced an uphill battle here, since

most likely the pre-sales team had built a ‘rosy picture’ of the project:

[E1] “The expectation has been set and we’ve set it, and we need to now live up to that.”

[S1] “The main problem is at the kick-off - the parties have a dream and we need to make that realistic.”

Thus the implementation manager was faced with the task of guiding the client

through the realities of what could actually be achieved and a key part of this process

involved trying to recalibrate the client expectations from the start:

[I3] “You’ve got to realign their expectation back again - these are the focus items. … [It’s up to] you, the PM, to reset that expectation with the customer.”

Satisfaction with vendor or consultant performance is a key dimension of overall

success for customers when engaging an external firm (Gable, 1996), and this customer

satisfaction is a function of both the initial expectations and the degree to which those

expectations are met (Szymanski & Henard, 2001) If client expectations are unrealistic at the

start then managers realized that they must act quickly to bring them down to levels that

matched what could be achieved, particularly as the detail of requirements included and not

included in the original contract became clearer. However, managers found this to be a

difficult task:

[A1] “There are always exceptions that you have to pay more for. But how do you educate, manage the customer’s expectation? In general, we all fail to do that.”

One tactic mentioned several times, particularly by managers of ‘troubled’ projects,

was simply to be honest with the customers:

145

[I3] “Look there’s no use telling you what you want to hear … you need to … say this is what’s going to be happening. It’s not good but might as well bite it right now.”

[U3] “What are the issues the customer’s brought up for a long time that nobody responds to because there is no solution? You must use a very appropriate way to tell them immediately, don’t give them false hope.”

While such honesty might be initially unpalatable for the customer, it was seen to be

the first step towards re-aligning the customer expectations again so that even a ‘troubled’

project could be turned around and result in a satisfied customer, as U. explained, noting that

while he began his ‘troubled’ project with a very unhappy customer, the final outcome was

very beneficial for his firm:

[U3] “The customer is about to terminate the contract, and is very, very frustrated … Finally we brought the system into production, and the customer is very happy that we have saved the project. And in addition to that, they continue to give us variation orders and extra money on additional works and enhancements.”

Balancing cost, schedule and scope. As noted in the previous section, the first

approach to dealing with anticipated problems that arose in spite of mitigation was most

often to apply the Control strategy of increasing man-hours with extra overtime. If increasing

the available man-hours was not sufficient to address the problem, managers’ strategies

typically involved a negotiation to balance the conflicting demands of minimizing cost and

time and maximizing the deliverables:

[L] “Cost, schedule, and quality in terms of the specification and the scope, are the three areas that we as managers constantly have to balance off, because it is always a trade-off thing when you have only that amount of money to do this amount of work within this amount of timeframe.”

[F1] “But if we need to sacrifice [something] for the time, we will. I don’t mean the quality but maybe not using the best way to do it. Then we need to get this kind of trade-off.”

Typically, the managers would first attempt to negotiate with the client for a

reprioritization of time, budget, or quality in terms of what functionality could be delivered.

Where possible, as noted in the previous section, managers would try to use the change

control clauses of the contract to gain extra time to meet commitments. Often this involved

negotiating a postponement of less critical requirements to a second phase of the project, so

that all resources could be channelled into getting the essential core of the system running by

the schedule date.

146

The managers of three projects with tight and immoveable dead-lines found that the

fixed nature of the schedule gave them a strong advantage that made it easy to negotiate a

reduction in the requirements, or extra budget, when faced with difficult problems:

[P1] “When that [dead-line] was the number one priority then a lot of things could be compromised. …If I cannot meet [a requirement], then we can always leverage that [dead-line] for difficult decisions, where we’re stuck.”

On some occasions, managers saw themselves in a weak negotiating position with

their clients, either because the clients were very tough negotiators or because the managers

recognized some shortcoming in their own team’s performance, perhaps because of an over-

ambitious schedule set by the pre-sales team or an under-estimation of the resources required

to deal with the complexity of the project. In these situations, managers from the larger firms

would turn to internal negotiation with their own senior executive to attempt to obtain extra

resources, thus increasing their own budgeted costs and reducing their firm’s profit margin

on the project, while still meeting the schedule, budget, and deliverables commitments made

to the customer. These managers were aware that they could fall back on the fact that their

firms employed a large pool of permanent staff whose salaries were already accounted for in

the firms’ overall budgets. If things got very difficult, they could negotiate the assignment of

extra staff from this pool, even though it meant a cost overrun in their own projects, because

other more profitable projects could take up the slack.

[U1] “So when you talk about cost over-run in [my firm], and I believe that is also true in most of the sizable vendors, they’re more concerned on the cost over-run on the sub-contractor’s side where they need to pay real money to a third party to complete the work. And to a critical situation like this, they do not bother very much about how much additional internal resource you need to pull into the pool.”

Managers from the smaller firms faced more difficult negotiations with the client

since they did not have such buffers to fall back on, and failure to meet budget targets had a

very direct impact on their firms’ profitability. However, typically when faced with difficult

clients who were unwilling to negotiate any balancing in terms of cost, schedule, and scope,

these managers chose to accede to the client’s demands and accept reduced profits rather

than continue with protracted argument and disagreement:

[E1] “It was a very difficult negotiation. It was not what I see as negotiation on other projects. Every project involves negotiation, every project there’s a constant game of give and take. And there are always differences in expectation. This one, there were a lot of situations where we just said, you know the cost of argument is greater than the cost of doing it, and we’ll just do it.”

147

Even though contracts typically included exit clauses, these were rarely invoked, with

respondents viewing the potential damage to their reputation as an over-riding concern:

[E1] “ we calculated the cost of exit and the cost of exit is not only the fact that the project contract has exit parameters and costs, but also credibility … about our standing in the industry, not walking away from problems. … We don’t like a customer to end a project saying that was a hard experience.”

Escalation. While escalation to the respondents’ own managers was sometimes used

concurrently with other negotiation strategies in order to gain extra leverage, most often it

was seen as a last resort to be used when all other attempts to resolve an issue had failed:

[C] “Another customer, they could be very nasty, they could keep on arguing for a period of time. I have no choice but to escalate and then it’s up to the next level of management to decide whether they want to retain the customer relationship, or whether they want to stand firm…”

However, managers were generally wary of escalating problems to their client’s

executive, because they saw such a move as destructive and likely to seriously undermine the

relationship with client staff that they had put so much effort into building:

[K1] “…you would not go over them and talk to their boss. That is more like a respect, and a relationship problem.”

Team building. Managers were keenly aware of the difference a committed team

could make to the success of their project, and worked hard to negotiate internally for the

right mix of staff skills to meet the requirements. Most staff were employed on salary so that

overtime payments were not available, and managers had to rely on their own motivational

skills to motivate and encourage their staff to make the necessary effort to achieve results.

When tight schedules had been set, managers tried to set the team’s expectations from the

start that the project would be hard and challenging and would require long hours of work

from them:

[S1] “It’s important to maintain motivation throughout the project - you need to make sure you select the right team members at the start …That time schedule meant overtime was mandatory … I didn’t tell them that successful completion would result in a bonus - I had no money and no authority for that …I just sold them on the dream - they are the pioneers in e-business outsourcing in greater China.”

Once they had set expectations for long working hours, managers used a leadership-

by-example approach to maintain morale – if their staff had to work over-time the managers

themselves also worked similar hours:

148

[C] “They are not getting extra pay, they are not getting extra holiday, they are not getting extra anything, they are just under pressure because they know it is critical for the customer to have the system working so they are just working on their good faith. The way I get them is to let them know that I am also doing the same thing. I am not asking you to work like crazy while I’m just sleeping well there.”

Managers also sought to build commitment and enthusiasm by fostering a team spirit

and involving their staff in decisions about the project:

[N1] “I approached the morale issue by making them feel part of the team, not just a resource. I started weekly meetings … so if we had any issues, the issues were discussed as a group and resolved as a group. … That kind of stuff, just little pep talks, and we had displays up to show where we were, in one case a thermometer we had drawn out up on the wall, and we wanted to get up to 100%, and each day we would update that. … It made them feel I’m part of the team, I may be part of the problem but I’m also part of the solution.”

Managers who took over ‘troubled’ projects observed that team morale was often

low on a poor performing project and they typically tried to negotiate with their own senior

executive in order to provide some kind of incentives, most often in the form of

performance bonuses paid on completion of significant milestones in the project. In some

cases this required a very determined fight on the part of the manager, since the firms

typically had a strong no-overtime-payment policy:

[N1] “Morale was an issue when I took over. We had people that were owed 2 months time off in lieu of payment for overtime work, and it was necessary to carry on working them 6 or 7 days a week, and 12 hours a day at least. So morale was low because these guys were working hard, they hadn’t been compensated, to be told, you’ve got 2 months off in lieu of OT, is unrealistic in [our firm] because you never get the chance to take that … So I had a battle, but I won that battle … by getting approval to give them project awards at the end, and I sized those according to the OT debt that I had. And that was one thing that got them on my side, believing in me and trusting in me. For week after week, I’d tell them, it’s coming, and they’d start off each meeting with, is this going to be another ‘trust me’ meeting? I said, yes, and when I finally got the formal approval, I told them what was happening, and even though they had carried on working hard up to that point, they did it more enthusiastically after that, in a much happier mood.”

Sometimes, managers felt the need to protect key staff from outside pressures:

[D] “You need to be hard-headed … with people who are trying to put pressure on your team, whether that’s the client or your own firm.”

Managers were particularly protective of their key technical staff, who were often

seen as being poorly equipped with interpersonal skills to deal with the customer liaison side

of the project:

149

[I3] “You have to understand the mindset of a technical person. As a technical person, when they deliver stuff, if you criticize something, they feel personally criticized. And of course, why you know this, is because you’ve been in that situation, you’ve been in development also. So you have to understand - let me be that liaison person, that stop-gap, because right now that guy, the team, was getting blasted out of the water.”

A strong concern about the welfare of the project team permeated most of the

interviews. The care and attention that managers paid to ensuring they maintained a strong

team spirit was striking, especially given that team morale, or lack of it, has not been

identified in the literature previously as an important risk factor. But clearly the managers in

this study considered a strongly motivated team to be one of the keys to success, and they

worked hard to ward off any threat to their team’s well-being.

5.3.5. Research strategies

As noted in Section 5.2.2 above, respondents were specifically alert for technical risk

at the start of their projects and technical risks were discussed in seven of the 41 initial

assessment situations, and two of the 49 problem-arising situations. Technical risks, for

example, risk associated with new or unfamiliar technology, seemed to generate interest and

enthusiasm in the project staff.

[B] “For a technical problem, the project team will be aggressive to find out what is the solution.”

Managers typically tried to have some of their team make an early start, well ahead of

any task deadline, on researching and self-educating about the new technology in order to

identify any potential pitfalls:

[H2] “Because we knew that [new technology] was going to be a problem, we had a small team of two people work only on that problem from the very beginning.”

Indeed, this new technology risk was not one that raised many concerns:

[E] “There’s always technical risk areas … that’s actually the stuff that staff find interesting. Technically they like doing all that sort of work. So we don’t see that as big risk area …”

If an unexpected technical problem arose during the project, particularly any problem

that might involve a package customization, managers moved on to a Negotiation strategy,

with the aim of avoiding customization if at all possible, because of the implications for

package integrity across other customers. Thus they would first try to convince their client

150

that a ‘workaround’ solution would be satisfactory, or that the client should adapt to

whatever functionality was provided as standard in the package. Only as a last resort would

they propose customization because this would then typically involve them in internal

negotiation with their own developers to get the customization approved.

5.3.6. Monitoring strategies

While Monitoring strategies were not often explicitly described by respondents, there

was an ever-present undercurrent of situation awareness throughout their descriptions of

their work. Some managers noted that their more implicit approach to risk management was

embedded in their overall management, and involved a constant surveillance of the situation

for potential problems, as exemplified by this manager’s comment:

[D] “You have to keep an eye on the horizon all of the time to try and anticipate what’s going to come up. …It’s just that you naturally behave like that… it has to be ingrained in you that you naturally should be looking for those things all the time anyway.”

At the start of their projects, respondents described an often implicit assessment and

evaluation of the surroundings and context of their projects, which helped to alert them on

particular aspects of the Control strategy and the WBS that would require their attention. So

for example, if the environment of the project involved overseas third parties, while this did

not trigger off any new strategy, it did alert the project manager to be particularly vigilant in

monitoring and controlling the deliverables expected from that third party, and especially in

watching out for time zone differences that could cause additional and potentially unplanned

delays in meeting deadlines. Similarly, while managers were aware that difficult contract terms

and conditions could be a serious risk, they accepted whatever terms and conditions had

been agreed at the pre-sales stage, and then sought to use their overall skills to address any

problems that might arise as a result of these conditions:

[N2] “We said, OK, from a terms and conditions point of view this is a high risk project. What we have to do, is we have to accept it and realize that we’ve got these awkward terms and conditions, and manage it.”

Managers sometimes recognized there was no specific action they could take about

an environment risk. This did not mean that they ignored the risk. Rather, they monitored

the situation but took no preventative action and indeed often made no decision about how

to respond until and unless the situation actually arose:

151

[W] “Sometimes I just wait until it happens. … there are certain activities that are not controlled at the moment but it doesn’t mean I can forget them, I still have to pay attention and monitor them.”

Active monitoring was particularly important to help the managers determine the

level of support or resistance and general client readiness for the new system:

[A] “You have to walk the floor, get to know, size people up, see how they are.”

These kinds of observations, along with assessment of their clients’ general attitude

and organizational culture were essential cues in determining how to approach their

Negotiation strategies, as discussed in the previous section. However, equally important was

self-awareness and monitoring of their own firm’s position:

[I2] “What is your objective? Do you give up this so that you can get that objective? And that’s the chess playing, the negotiation.”

Throughout the progress of the project, managers were alluding to a constant

alertness about what was happening both within the project – with their clients, with third

parties, with their own team – and in the larger project context of the business environment

including such things as competition, business changes, and government regulatory changes.

[L1] “And I constantly look at how am I doing now, what are the things that could potentially turn things on the downside. There are two sets of things that I watch for. One is of course looking at the deadlines and the deliverables quality. The other thing that I constantly observe is how the team interact … and also client’s feedback, all these are things that constantly are good sources of information. While at the same time, things change, assumptions could be invalidated, people will turn over, business requirements could change … I keep telling myself, OK, you bring in project management discipline to make things happen, but don’t overlook the intangible side, the soft side of the project, meaning the morale issue, the motivation issue, and also potentially some external factors that are out of the team’s control. OK, you might be putting in a perfect project management plan from your perspective but there could still be chances that the government policy change, or the client raise a change request.”

5.3.7. Comparison of risk strategies with prescribed processes

The risk management strategies recommended in the literature revolve round first

identifying the specific risks for a particular project, and then planning and implementing

specific actions to address each risk, on a risk-by-risk basis (PMI Standards Committee,

1996). The planned actions may be to eliminate, mitigate or accept risks depending on their

priority, and contingent actions should also be planned to address problems that actually arise

in spite of any eliminating or mitigating actions that may have been taken. The approach

described by respondents in the present study is strikingly different from the typically

152

prescribed processes. Even though a detailed risk assessment had been done at the pre-sales

stage, managers did not report that they actively reviewed these assessments and they rarely

took any actions to eliminate or mitigate potential problems that had been identified, with

two exceptions discussed below. Rather, they relied on contingency time and budget built

into the project plan by their pre-sales team to enable them to address any problems that did

arise.

The first of the two exceptions referred to above arose when specific technical risks

were identified. These technical risks triggered an elimination response using the Research

strategy in order to resolve uncertainties about technical issues and produce solutions to any

problems uncovered. The second exception occurred if a manager’s review of the project

plan developed by the pre-sales team revealed unachievable budget or schedule targets. This

problem triggered mitigating actions to seek further time or money allowance from the

manager’s senior executive, and also to protect the manager from adverse criticism if indeed

the project failed to meet its schedule or budget targets. A further mitigating action of

enforcing a strict change control policy was also implemented when the manager faced

difficult schedule and budget deadlines.

More generally, however, managers did not attempt to plan specific responses to

each of the many possible risks that might be identified for a project, relying instead on their

own management skills to control the project’s progress and on constant situation evaluation

and assessment to give them sufficient warning to react when problems actually occurred.

Some managers attributed the lack of specific risk response planning to the need for firms to

survive in a fiercely competitive market, with customers unwilling to pay extra for a project

plan that included a risk management phase:

[K] “I guess for the norm these days, there is not that much risk management in software implementation projects. It’s not a good thing, but the basic reason stems from money. For most people, even for executive and senior managers, they are not that adaptive to thinking about risk management on software or intangible things, and that problem leads to, it is impossible to convince the client that you should pay extra time to manage the risk. … And so it just becomes a hidden rule that when you do the project plan you add up the contingency.”

Others argued that risk management was essentially a pre-sales activity, which if done

correctly would provide the correct budget and schedule to enable the project to run

smoothly:

153

[X] “Risk management is a pre-event exercise to prevent risks before they happen. It’s important that the pre-sales work is done to a high standard so that the initial timetable is realistically set. Scoping is the most important aspect of risk management. Eighty percent of the issues can be covered there.”

However, there also appeared to be reluctance on the part of the managers

themselves to attempt to make elaborate plans to address, particularly, the intangible risks

that were difficult to articulate and specify, and also difficult to quantify. For example, while

managers were clearly aware that a project involving an overseas third party was a higher risk,

whether that risk evolved into an actual problem depended on many uncertainties and

unknowns, such as the skills of the third party staff, their willingness and cooperativeness,

other work on the third party’s agenda, unknown technical problems that would only surface

at the stage of testing and integration of products, and logistical issues of import and delivery.

All of these uncertainties could fall together and produce a major impact on the project’s

progress, or none of them could happen at all. Where a specific technical problem amenable

to a Research strategy could be isolated, then this would be immediately addressed. Otherwise,

respondents described general Control, Negotiation, and Monitoring strategies that would enable

them to respond quickly to any or all of these problems if they arose, but that did not involve

any extra work or planning at the outset that would be wasted effort if everything ran

smoothly.

These key general strategies, Control, Negotiation, Research, and Monitoring, are a very

practical and ad hoc response to risk management in a highly uncertain, complex, and

changing environment under significant time and budget constraints. The Control strategies

enable the manager to quickly get to grips with the complexities and potential problems

inherent in the project, without necessarily specifically identifying individual risks. Where

initial Control strategies reveal specific, easily identifiable technical or requirements problems,

these can immediately be addressed with Research strategies. Control strategies also provide a

first line of defence against any issues that do arise, providing early identification of problems

and typically having general contingency built in to give time to resolve these problems. The

Negotiation strategies provide managers with tools for pre-empting potential relationship

problems with external and internal parties, by establishing positive and cordial working

relationships. If unexpected relationship breakdowns occur, or if other problems such as

functionality or technical difficulties arise, Negotiation strategies help managers to establish

compromises with their clients or with their own management in order to maintain the

154

forward progress of the project. Underpinning these three strategy groups is the Monitoring

strategy, which involves constant situation awareness and assessment to ensure early warning

of any problems on the horizon.

5.4. Summary of discussion: Research Questions 2 and 3

In this chapter, I have presented the results of the investigation into Research

Questions 2 and 3, which asked what risk management practices and strategies were being

applied by IT project managers, and how these compared with literature prescriptions.

The results and discussion for Research Question 2 on risk management practices

highlighted a very close match in practice at the pre-sales stage with risk management

prescriptions in the literature, although there was little evidence of the contingency approach

to risk management, which proposes that risk management and development methodologies

be matched to the level of risk exposure in a project. There was also a weak link in most

firms’ risk management practices in the hand-over from pre-sales to implementation teams,

with an apparent lack of follow-up on execution of the risk management plans prepared at

the pre-sales stage. This gap was potentially compounded by the practice in most firms of

separating the risk assessment duties from the pre-sales and implementation teams. Although

this separation of duties is commendable in terms of reducing potential conflict of interest

for the risk assessors, it removes responsibility for the risk management plan from the project

implementation manager, thus aggravating the problem of failure to execute risk mitigation

actions. Firms appeared not to capitalize on the store of tacit knowledge present within their

experienced project managers by using post-project reviews to encourage knowledge transfer

among staff.

Turning to Research Question 3, the investigation revealed sharp distinctions

between the risk management strategies applied at the pre-sales stage and those applied

during the implementation of the project. The key risks identified by respondents during the

pre-sales stage were related not so much to potential problems for the project itself, but

rather to the vendor firm’s own performance in two areas, namely the risk of over-selling or

over-promising in order to win the sale, and the risk of failing to fully understand the

requirements because of the desire to minimize pre-sales costs. Strategies to address these

two risks included the separation of risk assessment duties already alluded to and a pre-

155

project partnering approach to separate the requirements specification work into a stand-

alone chargeable activity.

The investigation into strategies applied during the project implementation

uncovered a substantial variation from the prescriptions in the literature, which recommends

a risk-by-risk approach to plan actions to address each of the risks identified for a project.

Respondents in the present study described strategies that can be summarized within four

broad categories of Control, Negotiation, Research, and Monitoring, which they applied generally to

all their projects, regardless of specific risks that had been identified. Under this view of risk

management for IT projects, the large number of potential risks (43 identified in this research

as discussed in Chapter 4) are consolidated into four major categories according to which

strategy is best applied to them. Thus most of the risks placed in the project management

theme in Chapter 4 fall into the category addressed by the Control strategy. Most of the risks

in the relationships theme are addressed by Negotiation strategies. Technology and solution

related risks are amenable to a Research strategy, and all the remaining risks fall under the

Monitoring strategy.

The application of these four general strategies is a key example of tacit knowledge

being applied by managers to modify the traditional prescriptions from the literature into a

practical and manageable approach, as discussed next in Chapter 6.

156

C h a p t e r 6

6RESULTS AND DISCUSSION OF RESEARCH QUESTIONS 4 AND 5: TACIT KNOWLEDGE AND GAPS BETWEEN PRACTICE AND THEORY

6.1. Introduction

In Research Questions 4 and 5, I draw together the findings of the first three

research questions, and examine these findings in the light of the theories discussed in

Chapter 2. Firstly, in the exploration of Research Question 4, drawing on Sternberg et al.’s

(Sternberg & Horvath, 1999) definition of tacit knowledge, I tease out the tacit knowledge

embedded in the findings of the first three questions, and show how respondents used this

tacit knowledge to extend the core explicit knowledge that they had gained through formal

education and training. Turning to Research Question 5, I consolidate the analysis from the

previous questions and examine the gaps identified between theory and the practice of these

respondents from the two contrasting theoretical perspectives discussed in Chapter 2 – the

rational decision-making approach underpinning literature prescriptions for IT project risk

management, and the NDM approach, which provides a descriptive framework for decision-

making in real-life settings.

The coding and analysis processes to address these questions were described in

Chapter 3. In this chapter I report on the results of that investigation, and discuss Research

Question 4 on project managers’ tacit knowledge in Section 6.2. In Section 6.3, I consider

Research Question 5, and discuss gaps identified between theory and the practice of these

respondents in the light of rational and naturalistic theories of decision-making. I complete

this chapter with a brief conclusion in Section 6.4.

6.2. Research Question 4: Project managers’ explicit and tacit knowledge

Research Question 4 asked: What knowledge (tacit or explicit) do experienced project

managers use in order to plan for and address critical risk situations that may arise during the

course of software package implementation, and in particular, what environmental and

situational clues do managers attend to when managing their projects?

157

Respondents were asked about their education and general project management

learning experiences during the interviews. These comments, together with the findings from

the first three research questions, and in particular the congruencies and variances observed

between the actual practices reported by managers and the recommendations in the literature,

inform the investigation of Research Question 4. Next, in Section 6.2.1, I discuss the explicit

knowledge applied by project managers, which was demonstrated in the high level of

conformance to prescribed practices at the pre-sales stage of risk assessment, and also

described by the respondents in their general comments about their education and learning

experiences. Respondents’ tacit knowledge about risk management practices, as discussed in

Section 6.2.2 below, is embedded in the variances from prescribed practices and strategies

discussed previously in Chapter 5, as well as being acknowledged by the respondents in their

comments about the limitations of their formal education. Finally, in Section 6.2.3, I

elaborate on the environmental and situational cues attended to by respondents, and I

contrast the key explicit and implicit cues used by managers to monitor their projects.

6.2.1. Explicit knowledge applied to project risk management

Most of the respondents (84%) had at least a bachelor’s degree, as detailed previously

in Chapter 3, and three of the four who had finished formal education at high school level

were over 50 years of age, and hence had commenced their careers at a time when specialist

computing and information technology degrees were not widely available. All of the

respondents made reference to gaining formal training in project management skills at some

stage during their careers. In some cases, they attended in-house training sessions on basic

project management skills, such as developing work breakdown structures, and in other cases

they sought training externally from organizations such as the Project Management Institute

(PMI). Respondents also referred to learning from the standard project management

methodologies and templates mandated by the firms they worked for, and in particular, as

noted previously in Section 5.2, most firms prescribed risk assessment processes and risk

checklists at the pre-sales stage of implementation projects that conformed closely with PMI

guidelines (PMI Standards Committee, 1996).

However, while respondents acknowledged the usefulness of such training, they were

also keen to point out its limitations, noting the role experience played in developing their

158

skills, and particularly the need for them to develop skills in addressing the ‘soft’ side of inter-

personal relationships with their own team and their clients:

[F] “I joined the PMI and I learned some skills from that area. In [our firm] we have some internal training once a year, how you manage the project, how you control the financials, how you read the financial status of the project. [But] it’s mainly the hard skills.”

[K] “It [PMI training] has some really good concepts in how you deal with situations, it aligns you with best practice ways of thinking, but at the same time it’s more like textbook versus real world. In terms of really executing those things, I guess you can write 10,000 words, but you cannot explain exactly what you need to do when you hit the situation. So it’s experience”.

As I have already described in Section 5.2, while respondents described a rigorous

pre-sales risk assessment process carried out at the pre-sales or project proposal stage, very

much in line with standard prescriptions from the literature, their approaches to the

management of risk at project start-up and during the course of the project relied much more

on their experience and tacit knowledge, and I discuss these aspects of their practice next.

6.2.2. Tacit knowledge applied to project risk management

As discussed previously in Section 2.3.5, tacit knowledge in the management field is

defined simply as knowledge acquired implicitly from everyday experience (Sternberg &

Horvath, 1999). Tacit knowledge has three key features. Firstly, it is typically acquired by

experience, observation, or trial and error, without systematic support from other people or

media such as books. Secondly, tacit knowledge tends to be procedural knowledge that

guides behaviour i.e. ‘knowing how’ rather than ‘knowing what’. And thirdly, it is knowledge

that has a direct practical outcome for the person acquiring it i.e. in the context of the present

study, it is knowledge that respondents have acquired that is directly applicable in the course

of their work. Since tacit knowledge is difficult (but not impossible) to articulate, to some

extent its existence must be deduced from a detailed examination of situations described by

respondents and a comparison of their descriptions of their actions with what they might

have been expected to do if they had applied only explicit knowledge gained, for example,

from formal training courses. Thus I have deduced respondents’ tacit knowledge about risk

management practices from the variances from prescribed practices and strategies, as well as

from respondents’ comments about the limitations of their formal education.

As noted in Section 5.3, respondents approached the control of their projects with

three general strategies (Control, Negotiation, and Research) underpinned by a constant Monitoring

159

strategy, rather than with specific strategies to address individual risks. This approach is a key

example of their tacit knowledge, since it meets the criteria outlined above. It is an approach

that managers have developed from their experience and that they use to guide their overall

risk management behaviour i.e. it is an example of ‘knowing how’ to manage the project,

rather than of ‘knowing what’ to manage. It is also not something that the respondents could

clearly articulate for themselves, and it only became clear from a close reading and analysis of

situation descriptions obtained from critical incident interviews. Although this general

approach deviates from typical prescriptive practice, it still encompasses the explicit

knowledge contained in formal sources, since managers demonstrated a very clear

understanding of the likely risks that could impact on their projects, and their approach to

each project was tempered by an awareness of the most likely risk areas for that project. For

example, in projects involving a sub-contractor, the respondents were aware that the general

Control strategy of developing and monitoring a detailed WBS should particularly be extended

to cover the sub-contractor’s activities, since failure of sub-contractors to deliver on time was

seen as a key risk.

Section 5.3 contains a number of other examples of tacit knowledge, and like the

general strategy approach, they tend to encompass or extend explicit knowledge about

prescribed practice. Some of the tacit knowledge contained in Section 5.3 is commonly

known in the community of practice, and yet receives little attention in the prescriptive

academic literature. For example, the Negotiation strategy, balancing cost, schedule, and scope,

is widely known amongst practitioners and even reported in some student textbooks, (see,

for example, Dennis & Haley Wixom, 2003 pg 60), but has not received attention in the

more formal body of knowledge on project management, discussed in Section 2.2.1 (see,

for example, PMI Standards Committee, 1996). As such, this strategy forms part of the

collective tacit knowledge of practitioners, as described in Section 2.3.4 – knowledge that is well

known by the members of the group i.e. project managers, but not formally recorded in the

explicit knowledge store.

Some tacit knowledge is encapsulated in rules of thumb, while other tacit knowledge

shows up more in a general understanding and situation awareness that underpins managers’

strategy descriptions. Section 5.3 already contains detailed descriptions of the strategies used

by respondents, including their tacit knowledge about how to apply these strategies, so here, in

Table 6.1 below, I simply summarize the main tacit knowledge items described in Section 5.3.

160

Table 6.1: Examples of tacit knowledge associated with risk management strategies

Strategy group

Strategy Tacit knowledge

General Manage projects with a few broad strategies, not with many risk-specific strategies

1. Many risks are difficult to anticipate or quantify so better to be prepared at a general level than attempt to manage on a risk-by-risk basis.

2. Avoid wasting time planning for potential problems that may never arise, while ensuring manager is well situated to address problems if they do happen.

Control Develop detailed WBS 1. Rule of thumb: break down WBS to tasks of one week duration for practical control.

2. Developing WBS to this level of detail will ensure manager has in-depth grasp of the requirements, proposed solution and technology, required skill-set, and potential problems without specifically evaluating these.

Control progress on WBS 1. Rule of thumb: probe for specific evidence on staff and sub-contractor claims about % completion of tasks.

2. Tight control of progress will provide best early warning of any problems actually arising.

Control the documentation 1. By retaining control of, for example, meeting minutes, managers can ensure wording is cast favourably towards the vendor as insurance in the event of later disputes.

Negotiation Apply change control but understand the need to maintain a balance in client relationships

1. Always view change control as a negotiation process rather than simple contract enforcement, even when applying change control clauses strictly.

2. Application of strict change control can be used to ease tight schedule and budget conditions.

3. Where a client is assessed as potentially difficult, early application of change control can be used to establish discipline and ‘train’ the client, but this approach should be used cautiously in order to ensure that goodwill is not undermined.

4. Indicating that a request falls within the change control scope, but then waiving the application of the clause can build goodwill, but such practices should always be carefully documented.

5. For in-house managers faced with departments that are accustomed to using internal politics to ensure that on-going change requests are met, using outsourcing and supporting change control enforced by the external provider can help to change the internal culture and improve understanding within internal departments of the need for careful requirements specification.

Trust & relationship building 1. Rule of thumb: mentally “stand in the customer’s shoes” to help determine how to approach any issues that arise.

2. Recognize when to give in rather than argue. This is often a useful tactic at the start of a project, but may also be necessary throughout a project with a particularly difficult client, especially if the only other option appears to be to exit the contract.

3. Find ways to enable client staff to look good in front of their own bosses.

Restoring confidence in ‘troubled’ projects

1. Identify two or three issues that can be solved quickly and solve them.

2. Be honest about unsolvable problems.

161

For most managers, the main source of their tacit knowledge was their own

experience, learning through trial and error. Only two specifically referred to working with

someone they regarded as a kind of mentor, or as someone they could learn from, and one

respondent noted that he had learned a lot about what not to do from working with one

particular manager early in his career. Some respondents were quite frank about the mistakes

they had made in the early stages of their careers:

[P2] “If I were doing it now, I’d do things differently. … Back then, I simply assumed the problem was technical and that sent me on the wrong track.”

[N1] “I did something very silly one day … but I turned round and said, ‘you’re right, I was wrong.’ If you’re wrong, admit that you’re wrong. You have to do that to some extent, you establish relations with the other person.”

Only one manager noted explicitly that she routinely engaged in self-reflection and

self-assessment in order to continually try to improve:

[L] “[I’ve learned from] practice … every time you finish a project you ask yourself what situations that I have faced in this project that I can do better, or I feel I do good. … And it’s a continuous learning, self reflection exercise. A lot of these are soft skills, I’m not trying to discredit project management discipline, all those tools and techniques, you still have to learn them. But when you are out in the field and you are facing all kinds of different situations, you just need to become more and more mature in terms of handling people.”

However the comments of respondents indicated that most of them did use self-

reflection implicitly as a means of fine-tuning their skills, and some commented that they had

found the interview conducted for this study both thought-provoking and useful in that it

directed them to focus and reflect explicitly on their practice.

6.2.3. Environmental and situational cues

The environmental and situational cues that respondents attended to are, for the

most part, encapsulated in the strategies they used to manage risk in their projects. The key

underpinning strategy in the managers’ toolkit was the Monitoring strategy, which involved a

constant scanning and awareness of the whole project context, in order to provide them with

the earliest possible warning of things that might be going wrong. Clearly this scanning is

based, at least to some extent, on the explicit knowledge of the types of risks that apply in IT

projects, as made clear by the wide range of risks that managers alluded to in their interviews.

But there was little evidence that they regularly reviewed the risk checklists prepared at the

pre-sales stage in order to explicitly consider whether any of the risks identified then were

162

about to kick in. Rather they vigilantly monitored the overall health of the project, and used

explicit and implicit cues to alert them to possible problems that could arise.

The key explicit cue described in the application of the Monitoring strategy derived

from the high level of attention paid to detail on the development and control of the work

breakdown structure, as discussed earlier in Section 5.3.3. This reflects the respondents’ view

that most problems are inter-related and the earliest warning of almost all problems will show

in an impact on task slippage on the WBS. Thus close monitoring of the WBS was regarded

as providing the best early warning not only for the more obvious potential problems in the

project management theme (as shown in Table 5.2) but also for other environmental,

technological, and relationship problems that, if they were beginning to surface, would

almost always quickly show some adverse effect on a task on the WBS.

When a slippage was observed, managers first focused on understanding more about

the task and its situation in order to identify the underlying cause of the problem, as

described by this manager of a ‘troubled’ project:

[L1] “When we start asking that question [why is the task slipping], then we go into what is the real reason for not being able to fulfil this task …so we find out, oh if it is a people problem … [or] if it is not a staffing issue, it is mainly we haven’t defined the requirements clearly, maybe we should add an additional task, and shuffle the tasks around.”

Once the cause of the problem was understood, the sequential application of

strategies described above was applied.

The other key aspect of the Monitoring strategy was revealed in the close attention

paid by managers to their clients, in terms of continuously assessing the clients’ expectations

about the project and extent of common ground between the clients and the project team.

Managers used a cue of trying to view each possible problem situation from the client

perspective, in order to find the best possible negotiating route through the situation.

Problems that had moved beyond the effectiveness of Control strategies were usually not

amenable to instant solutions, and the managers’ ability to negotiate a satisfactory agreement

about the resolution of such problems was dependent on how well they had understood the

clients’ attitudes and managed their expectations. Managers emphasized the need to involve

all levels of the client staff, in order to ensure that both the project team and the client staff

fully understood what was happening:

163

[H1] “I think you’re always better [off] as soon as you can getting down to the level of the real users who are really using the system and no matter what their hierarchy is and how they try to protect or control that, you always want to push the boundaries.”

Managers were often implicitly aware of an imminent problem before it had shown

any clear impact. Five managers used the word ‘smell’ to describe this process of sensing a

client-related problem before there was any explicit evidence that a problem was arising:

[V1] “At that stage, of course, we smelled something wrong at that very first deliverable and we talked to them, we had many, many discussions with them.”

[F2] “Actually we did anticipate this kind of, we smell this kind of thing before …”

While these managers were unable to articulate what exactly was cueing them to

recognize the problem situation, they described this warning sense in situations of client

unease where there were issues relating to the client staff either in their resistance to the

system or their unwillingness to take responsibility for issues on their side of the project, such

as decisions about particular functionalities. Managers typically used Negotiation strategies to

resolve these client-related issues and as noted above, relied on a second key cue of viewing

the situation from the customer’s perspective to help them understand the situation fully

before moving into a negotiation strategy.

The tacit knowledge and situational cues described in this section contribute

particularly to the discussion of decision-making processes, which are discussed in the next

section on Research Question 5.

6.3. Research Question 5: Gaps between risk management theory and

practice, and decision-making processes

Research Question 5 asked: What gaps are there between actual practice and the

rational risk management processes as recommended in the IT risk management literature,

and to what extent can decision making theories such as Naturalistic Decision Making

explain any such gaps?

Research Question 5 essentially draws together the findings from the first four

research questions. As noted in Chapters 4 and 5, there are a number of gaps between actual

practice and the rational risk management processes recommended in the IT literature. While

explicit knowledge formed the core of respondents’ practice, their tacit knowledge was

164

revealed in many of the details of their practice. In particular, as discussed in Sections 5.3 on

strategies and 6.2 on tacit knowledge, one major gap arises in the respondents’ descriptions

of the strategies they use to address risk in their projects. These strategies are broad general

strategies, intended to mitigate most of the key risks that might arise in a project, and are

applied with little attention to the likelihood of particular risks actually occurring. This

contrasts with the typically prescribed literature approach, which recommends a risk-by-risk

approach to risk management, with strategies put in place to address each specific high

ranked risk that has been identified.

To understand more about how managers deviated from the prescribed practice in

terms of their risk management during the progress of their projects, I re-examined the

specific problem situations extracted from the transcripts for the analysis of Research

Question 3, this time looking for evidence of respondents’ decision-making processes at

these critical stages of their projects. I looked firstly for evidence of a rational approach to

key situations, similar to Figure 2.1 discussed earlier in Section 2.2.2, where, in initial

assessment situations, managers showed evidence of progressing through risk identification,

risk assessment, and risk response planning stages, and for problem-arising situations,

managers may have described several different alternative strategies to address a problem and

evaluated these before choosing one option. I also examined the situations in the light of

NDM theory, extracting details of the respondents’ goals, cues and expectancies in the

situations they described and looking for evidence of emphasis on situation assessment and a

serial approach to the decision-making, as shown in the RPD model (Figure 2.2) discussed

earlier in Section 2.2.5.

As a result of this re-examination, I observed that the problem situations described

by respondents fell into three groups. The first group contained initial assessment situations

described by respondents at the start-up stage of projects that they had managed from the

beginning (called ‘routine’ projects in the following discussion), while the second group

comprised situations describing the initial assessments made by managers when taking over

‘troubled’ projects and planning a course of action to rescue them. The third group included

problem-arising situations in both routine and ‘troubled’ projects.

While the key decision described in all the situations related to choice of a suitable

strategy to address the situation, the examination of these situations revealed contrasting

165

views of the decision-making approaches of managers for the different types of situation.

Types of strategies described in these situations also varied, as shown in Table 6.2 below,

with the majority of initial assessment situations in both routine and ‘troubled’ projects

involving Control strategies, while the majority of problem arising situations in all projects

involved Negotiation strategies.

Table 6.2: Strategies and stages in routine and ‘troubled’ projects

Routine projects ‘Troubled’ projects Initial assessment

situations Problem arising

situations Initial assessment

situations Problem arising

situations Control 16 9 10 3 Negotiation 6 29 2 6 Research 5 2 2 - TOTALS 27 40 14 9

In the following three Sections, 6.3.1 to 6.3.3, I discuss these three groups of

situations, first outlining the approaches described by respondents for that type of situation,

and then examining the approach from rational and naturalistic perspectives.

6.3.1. Decision-making processes in initial assessment situations of routine

projects

The typical strategy applied by managers at the start of their routine projects, as

described in Sections 5.2.2 on start-up risk management practices and 5.3.3 on Control

strategies, was a detailed focus on the development of the work breakdown structure.

Managers concentrated their efforts on establishing an in-depth understanding of the project

situation, which had an implicit consequence of allowing them to develop control of

potential risk areas without actually assessing risks explicitly. If they identified unfamiliar

issues such as new technology or unclear requirements, they initiated a Research strategy, and

if they considered the schedule or budget to be inadequate they initiated an internal

Negotiation strategy to attempt to obtain extra resources. Otherwise, there was no evidence

that managers explicitly evaluated the project for possible risks or took specific risk

prevention or mitigation actions. Rather, they relied on the risk assessment and contingencies

already built into the schedule by the presales team to insure against possible problems arising

during the project.

166

Rational perspective. In terms of the rational risk management processes described

earlier in Figure 2.1, managers’ actions at the start of routine projects implicitly included a risk

identification stage, as a side-effect of their detailed scrutiny of the tasks required to complete

the project. However, since any such implicitly recognized risks were never explicitly

acknowledged, the second stage of risk assessment, which comprises an analysis of the risks

for their impact and probability and a prioritization of risks, did not appear to be carried out,

although the immediate actions taken on schedule and budget, technology, and requirements

issues suggests that these three risks were implicitly highly ranked. For the third stage of risk

response planning, managers relied, for the most part, on the contingent action planning

already carried out at the pre-sales stage, and only attempted risk mitigation in the

circumstances noted above – new technology or unclear requirements, and inadequate

budget or schedule. There was no evidence that managers attempted to plan responses to any

other risks on a risk-by-risk basis. Rather, they appeared to expect their broad general

application of Control strategies to address any other problems that might arise. The fourth

stage of risk monitoring does not apply at this start-up stage of the project. Thus, managers’

processes at the start-up stage of routine projects conform, in part and mainly implicitly, to

rational risk management prescriptions, but there are gaps particularly in the risk assessment

and risk response planning stages. Figure 6.1 below, which is a modification of Figure 2.1,

illustrates the partial rational risk management process evident at the start-up stage of routine

projects.

Fig. 6.1: Partial rational risk management processes during initial assessment of routine projects

Naturalistic perspective. Does the RPD model better describe the processes being

used by these managers? The RPD model’s primary emphasis on situation assessment

corresponds very closely to the managers’ focus on developing a detailed work breakdown

IMPLICIT RISK IDENTIFICATION

- Through Control strategy of WBS

RISK RESPONSE PLANNING − Risk Elimination or Mitigation of technology

and requirements risks through Research Strategy

− Risk Elimination or Mitigation of schedule/ budget risks through internal Negotiation strategy

− Contingent Action Planning by default through assumption that contingencies provided by pre-sales are adequate

IMPLICIT RISK PRIORITIZATION

- Through immediate action on schedule/budget, technology, and requirements risks

167

structure. As the RPD model suggests, managers who faced unfamiliar situations (new

technology or unclear requirements) initiated a Research strategy to seek more information,

but otherwise the manager focused on fully understanding the situation through the

development of the work breakdown structure. The primary goal at this start-up stage in

routine projects was straightforward, being simply to deliver the project requirements on time

and within budget. The cues being drawn on for the situation assessment were the contract

and pre-sales specifications of requirements, schedule and resources, together with the

manager’s own knowledge and experience and the knowledge and experience of the project

team.

Managers’ expectancies were also straightforward. Typically, they expected that the

targets would be tight, but achievable, although an expectation of working overtime was the

norm. No respondent described a project where they considered the time and budget

allowances to be generous. Respondents also commented that, even if no specific

requirements or technology issues had been identified at this stage, they expected that some

requirements would change, and that some technology issues would arise, but that both of

these would occur within manageable limits that could be accommodated by the

contingencies built into the budget and schedule. Managers’ expectations about their client

relationships were more implicit, but I deduced from the surprise they expressed when client-

related problems did arise later in their projects that, at the start of their projects, they

typically expected client relationships to run smoothly, and this deduction is supported by the

findings reported in Chapter 4 that anticipated problems were most likely to be

schedule/budget related, while client-related issues were most likely to be unanticipated.

Once the WBS was fully elaborated, the managers could compare their expectancies

about the time and resource requirements for the tasks with the allowances provided in the

original schedule and budget provided by the pre-sales team. If these expectancies matched

the original schedule and budget, then the managers took no specific actions, and simply

applied Control strategies to manage the project. If, however, the detailed analysis of WBS

revealed a serious shortfall in time or resources, then the manager moved into an internal

Negotiation strategy as described in Section 5.3.4, in addition to applying the normal Control

strategies. There was no evidence of the ‘imagine action’ step of the RPD model even when a

specific strategy such as internal negotiation for more resources was chosen. Rather, if the

assessment revealed a need to apply such a strategy then it was immediately implemented.

168

The modified RPD model in Figure 6.2 provides a descriptive model of the

processes described by managers at the start of their routine projects. It shows total focus on

understanding the situation through the development of the work breakdown structure,

together with implementation of a Research strategy for unfamiliar situations. The actions

resulting from the situation assessment are the routine application of Control strategies, and an

additional internal Negotiation strategy if resources or schedule are inadequate. This model

reflects the actual practice of respondents, in that risks are not explicitly identified or assessed

as suggested by the rational risk management approach. Instead, the managers simply

watched out for certain issues that they knew were likely to cause problems – unfamiliar

technology, unclear requirements, inadequate schedule or budget – and initiated actions if

their assessment revealed any of these issues.

Fig. 6.2: Descriptive RPD model of processes at start of routine projects

Experience Situation in Changing Context

Is the situation familiar?

Develop WBS

Goals:Meet schedule

& budget

Expectancies:Tight schedule & budget;

Good client relations

Cues:Contract; Reqmts specs;

Pre-sales schedule & budget; Manager & team knowledge &

experience

Are expectancies violated?

Implement Control strategies

Implement internal Negotiation strategies

Apply Research strategySeek more information

Reassess situation

No:Unfamiliar reqmtsOr new technology

Apply Monitoringstrategies

Yes:schedule or

budget too tight No

Yes

Experience Situation in Changing Context

Is the situation familiar?

Develop WBS

Goals:Meet schedule

& budget

Expectancies:Tight schedule & budget;

Good client relations

Cues:Contract; Reqmts specs;

Pre-sales schedule & budget; Manager & team knowledge &

experience

Are expectancies violated?

Implement Control strategies

Implement internal Negotiation strategies

Apply Research strategySeek more information

Reassess situation

No:Unfamiliar reqmtsOr new technology

Apply Monitoringstrategies

Yes:schedule or

budget too tight No

Yes

169

6.3.2. Decision-making processes in initial assessment situations of ‘troubled’

projects

The managers of the ten ‘troubled’ projects, with one exception, all described

carrying out a very careful and detailed analysis when they took over the projects they were

asked to ‘rescue’. The one exception was a ‘troubled’ project taken over by a manager at an

early stage in her career. She had been asked to rescue the project because of her technical

expertise in the package being implemented, and because she was aware of this expectation

about her technical knowledge, and because of her lack of more general project management

experience, she immediately made the assumption that the cause of the project’s difficulties

was technical, and began to work on rescuing the project on that basis without making any

further investigations. She later learned that her assumption was misguided, and in fact the

project was being sabotaged by a contract worker seeking to prolong his contract on the

project. As she said, somewhat ruefully, on reflection:

[S2] “If I were doing it now, I’d do things differently. I would interview all the stakeholders first and ask them why they thought it was going wrong - tell me everything, and ideas for solutions. Let’s talk together - you’ve been here 3 years, so you know what’s going on. … Back then, I simply assumed the problem was technical and that sent me on the wrong track.”

More typically when faced with a ‘troubled’ project, just as with the routine projects,

managers focused intensively on understanding the situation, using the work breakdown

structure, the contract, and the details of the tasks already completed as a starting point. In

addition, however, they described a very careful process of evaluating and prioritizing the

remaining tasks on the project in order to find the key problem areas and identify the most

important issues to focus on. This quotation from a ‘troubled’ project manager exemplifies

this approach:

[P1] “When I took up the project, I first understand the business requirements from the functional specifications because the project team is mid way already and they’ve produced the functional specification. I look at the contract, I look at my contractual commitments, I look at the existing project plan and I look at their requirements. And then I started to look into the project plan, the WBS, and my existing team members’ profile to identify which are the tasks where my existing skill profile is not likely to live up to. Any task that has non zero % of chance that they will not be able to finish on scheduled end date, to me I have to understand their impact. Meaning any task that can potentially slip, I have to understand what’s the impact. If they are not on my critical path to me maybe that’s not a risk that I have to put on the top priority. And also look at the chances that they will happen and the timing, if they happen now I have more time to manage and to contain the risk, but if that type of task that I find is risky for me, risky in the sense of not being able to deliver on time and deliver the kind of quality that I want, maybe because we don’t have the right people assigned for the task, is likely to

170

happen at the end of the project, during my roll-out, to me of course the severity is higher so I would put them higher on my list of priority. And after going through the WBS I was able to identify which are the likely tasks that I could potentially fail, then I will after assessing their likelihood of occurrence and the impact, I probably just feature those that are likely to happen in the near future, and become my top 5 or top 3 or top 10 that I need to deal with.”

Although this initial assessment differed from the routine projects in terms of the

identification and prioritization of the critical potential problem tasks, managers still applied

the same general Control strategies described in Section 5.3.3 to get the project back on track.

I had expected to find more use of Negotiation strategies, both with the client to reduce the

scope, and internally within the vendor firm in order to gain more resources. However, when

these managers were given the mandate by the vendor firm’s executive to rescue a ‘troubled’

project, there was an acceptance that the vendor’s original budget limitations would not be

met i.e. the vendor firm accepted that this project would run with a smaller profit margin, or

even at a loss. The over-riding issue for the vendor firms in these circumstances was to find a

way to salvage the project in order to minimize the damage to the firm’s reputation. Thus the

issue of meeting budget constraints did not figure into the considerations of these managers,

and there was little need for them to use internal negotiation to get the support they needed.

As for external negotiation with the client, these managers typically felt they were in a very

weak negotiating position because of the project history, and hence they made little attempt

to negotiate for a more limited scope:

[N1] “From a negotiating point of view, I was in a very weak position, because we had twice delivered the system for acceptance and it was rejected.”

The key decision made by these managers when they took over their projects was a

prioritization of the problematic outstanding tasks. As with managers of routine projects,

these managers did not appear to consider a variety of alternatives for addressing the high

priority problems. Rather, once the key issues had been established, the Control strategies of

progress control and rescheduling and problem isolation described in Section 5.3.3 were applied,

together with the addition of extra staff, where a staffing or skills shortfall had been identified

as a problem. In addition, managers took immediate action to address confidence and

morale, applying team-building Negotiation strategies to boost their internal project team

morale, and identifying a few ‘quick wins’ in order to restore client confidence:

[U3] “What we need to do is have a quick scan, what are the top three issues of the customer, which one you can resolve, which one you cannot? Those you can resolve make it happen immediately, don’t look at the contract…”

171

Rational perspective. In contrast to the initial assessment of routine projects, the

initial assessment situations for ‘troubled’ projects, with their greater degree of uncertainty

and higher stakes, showed much clearer demonstrations of the application of rational risk

management processes. While managers taking over ‘troubled’ projects also focused their

attention on situation assessment, their approach matched the rational prescription

recommended in the literature much more closely than the approach described by managers

of routine projects, in that they carefully evaluated and prioritized the remaining tasks on the

project in order to identify the most important issues to focus on. Thus their initial situation

assessment using the work breakdown structure included explicit risk identification and risk

assessment steps. Once this prioritization was done, however, managers did not describe any

explicit process, corresponding to the third stage of risk response planning, of considering

and evaluating different alternative approaches for addressing the problems. Rather, they

focused on probing the details of the key problem tasks until they felt they had identified the

underlying issues, and then simply applied Control strategies in a sequential manner to bring

the project back on track. Their actions to restore client confidence and boost team morale

are examples of implicit risk response planning by mitigation, and their assumption that

original budget constraints no longer applied may be considered to be default contingent

action planning, of allowing for increased costs to be incurred by the vendor. Figure 6.3

illustrates the rational aspects of the risk management process during initial assessment of

‘troubled’ projects.

Fig. 6.3: Rational risk management processes during initial assessment of ‘troubled’ projects

Naturalist perspective. Viewing these situations from the NDM perspective,

respondents still placed a heavy emphasis on situation assessment, examining the course of

the project in great detail to try to determine what the key problems were. Managers’ primary

EXPLICIT RISK IDENTIFICATION

- Through Control strategy of WBS

RISK RESPONSE PLANNING − Risk Mitigation to restore confidence

through ‘quick wins’ Negotiation Strategy − Risk Elimination or Mitigation of schedule/

budget risks through Control strategies − Contingent Action Planning by default

through assumption that vendor budget constraints could be relaxed

EXPLICIT RISK ASSESSMENT

− Risk Analysis − Risk Prioritization

172

goal at this stage was to protect the firm’s reputation by completing the project. In order to

achieve this primary goal, the immediate aim was to establish an in-depth understanding of

what was going on in the project. The cues were drawn from the contract and details of the

project documentation to date, particularly the existing project plans and WBS, and details on

the deliverables such as functional specifications that had already been achieved. The

expectancies were that problems would be inter-related and most likely be schedule/budget

or vendor staffing related, that team morale would be low, and that client trust and

confidence would also be low. There was no evidence of evaluation of different options to

address the problems. Instead, immediate steps were taken to boost morale and restore

confidence, and once the detailed situation assessment had been completed, Control strategies

were applied sequentially. Once again, there was no evidence of the ‘imagine action’ step.

Fig. 6.4: RPD model incorporating rational and naturalistic situation assessment during initial

assessment of ‘troubled’ projects

Experience Situation in Changing Context

Gain Familiarity with Situation by WBS Review

Goals:Protect firm’s

reputation;Understand

situation

Expectancies:Inter-related schedule, budget, &

staffing problems;Low team morale;

Low client trust & confidence

Cues:Contract; Project plans to

date; Functional rqmts specs; Pre-sales schedule & budget; Probability & impact of high

risk tasks

Are immediate morale and

confidence actions required?

Implement Control strategies

Implement Negotiationstrategies to boost morale & restore

confidence (‘quick wins’)

Apply Monitoringstrategies

Yes

Use rational assessment of impact & probability of high risk tasks to prioritize tasks to address

Apply Monitoringstrategies

Experience Situation in Changing Context

Gain Familiarity with Situation by WBS Review

Goals:Protect firm’s

reputation;Understand

situation

Expectancies:Inter-related schedule, budget, &

staffing problems;Low team morale;

Low client trust & confidence

Cues:Contract; Project plans to

date; Functional rqmts specs; Pre-sales schedule & budget; Probability & impact of high

risk tasks

Are immediate morale and

confidence actions required?

Implement Control strategies

Implement Negotiationstrategies to boost morale & restore

confidence (‘quick wins’)

Apply Monitoringstrategies

Yes

Use rational assessment of impact & probability of high risk tasks to prioritize tasks to address

Apply Monitoringstrategies

173

The modified RPD model shown above in Figure 6.4, captures this combination of

rational and naturalistic approaches by showing a two stage situation assessment of, firstly, a

thorough analysis of the work breakdown structure and progress to date, and secondly, a

rational prioritization of the key problems. The application of Negotiation strategies for morale

and confidence was implemented even before the situation assessment was fully completed,

with Control strategies being actioned once the problem situation had been fully assessed.

The differences in approach observed here between routine project initial assessment

and ‘troubled’ project initial assessment, with managers in the latter situation placing

additional emphasis on a more rational assessment of potential problems, provides support

for the finding reported by Sutcliffe and McNamara (2001) that decision makers were more

likely to use prescribed decision practices when faced with important decisions, particularly

those involving high levels of resources and uncertainty. Managers of ‘troubled’ projects

conveyed the impression that they felt entrusted by their senior executive with the task of

recovering from a potentially damaging situation for their firm, and hence saw their actions

as critically important for protecting their firms’ reputations in the marketplace. The

‘troubled’ projects were also typically already in a cost-overrun state, and their new managers

were working under expectations that while they would be able to draw on additional

resources to solve the problems they would also be able to limit as far as possible any further

budget blow-out. Finally, these managers also typically referred to their uncertainties at the

start of their involvement about what might have caused the project to get into difficulties,

and hence, with the exception of the inexperienced manager [S.] noted at the start of this

section, they were particularly cautious about making any assumptions about what had gone

wrong. Thus, in such conditions of uncertainty with high stakes, particularly for the firm’s

reputation, these managers fell back on prescribed practices in order to try and determine the

best way to proceed.

6.3.3. Decision-making processes in problem-arising situations of routine and

‘troubled’ projects

The approach described to address problems arising during the project was

consistent across both ‘troubled’ and routine projects. As described in Section 6.3.6,

managers maintained constant situation awareness with a Monitoring strategy, relying on

explicit cues derived from monitoring progress against the work breakdown structure and

174

implicit cues from their general awareness of the relationship with their clients and their own

team to alert them to problems. When a problem was identified they used the same

sequential application of strategies to attempt to resolve the issue, both when the problem

had previously been anticipated and when it was unexpected.

Typically, when problems arose, managers first attempted to apply Control strategies

to keep the project on track. If a simple rescheduling of tasks on the work breakdown

structure would work then this was done first. However, more often, the strategy of

increasing available man-hours by requiring the project team to work more hours was

applied, in conjunction with rescheduling of tasks on the work breakdown structure. If the

problem involved requirements changes, initially the Control strategy version of change

control was applied, either to try and absorb the requirements change if it was minor, or to

try and apply the change control terms of the contract.

If these Control strategies did not work, managers turned to Negotiation to try to

redefine the boundaries and targets of their project, either by negotiating internally for more

resources, or by negotiating with the client, typically using a balancing cost, schedule and

budget strategy. Even within a strategy group such as Negotiation, when faced with a problem,

respondents did not describe a process of identifying several possible strategies, evaluating

them, and then deciding which one to choose. Rather if the initial negotiation was

unsuccessful, managers would iterate through a number of cycles of various negotiation

tactics within the same overall strategy before moving on to escalate the problem to the

senior executive for resolution:

[E1] “Oh, no, we didn’t have a battery of choices, I think we were as surprised as anyone else. So our first response was, whoa, [the client’s] expectation was wrong. OK, that lesson learned was our expectation was different, [the client’s] not wrong. And, then try another tack, try another tack, what can we do next?”

Escalation was typically the last resort, and while an exit option was always built in to

the contracts the managers operated under, they displayed a very high reluctance to take the

exit path, even when it was the most obvious and logical answer to a very difficult situation.

[H1] “At that point we should have stepped away ... But at that point … we were committed to this project … the company came to us because they heard that we would just do whatever it took.”

[X] “[We] typically do not want to damage reputation by walking away from a contract, so we will aim to find a way to complete it. Again this relates to reputation - references from previous

175

customers, particularly from firms that are well established and respected in the marketplace, are very important for securing future business.”

Rational perspective. From the rational perspective, these situations occurred

during the risk monitoring stage of risk management, and clearly the close attention paid by

managers to their progress against the WBS shows a high level of conformance to this stage.

However two aspects of processes described in these problem-arising situations vary from

the rational prescriptions. Firstly, under the rational approach, when considering corrective

actions for problems which fall outside previously planned contingencies, a number of

alternative solutions should be considered and evaluated. However, as noted above,

managers described a process of sequential application of strategies, and did not appear to

use any option evaluation to decide which strategy to apply.

Secondly, when problems arose, managers made no reference at all to drawing on the

contingencies (particularly time and budget) that had been planned at the start of the project

to address any risks identified then. This may be because, as discussed in Section 4.5.2, most

of the problem-arising situations involved unanticipated client-related issues, which would

not have been provided for in the original contingencies. However, even in the problem-

arising situations involving problems that had been anticipated, managers still did not refer to

using the planned contingencies.

It was not clear from the managers’ comments why this was so, but there are two

possible explanations. First, managers may have been aware that a particular task was

slipping, but may not have explicitly identified the situation as a problem until all of the

planned contingency had been used. However, given that most of the problem-arising

situations related to routine projects, a second possibility arises from the lack of explicit

acknowledgement of risks and risk response plans that was noted above in the discussion on

initial assessment situations in routine projects. If there is no explicit separation between the

expected time and budget for a high-risk task, and any contingency allowance made for that

task, then there is a danger that the contingency allowance will simply ‘disappear’ into the

overall expected time and budget for the task, so that the warning of task slippage will only

come after the all planned contingency has already been used. If this is so, it would be a

serious consequence of the failure to explicitly adhere to the rational risk management

process, since part of the purpose of distinguishing between expected time/budget and

additional contingency, is to ensure that the manager is alerted to a potentially problematic

176

task early, when there is still time to take further corrective action if necessary. Further

research is needed to explore managers’ handling of allotted contingencies, since failure to

handle them according to rational prescriptions would provide support for the contention of

some researchers (for example, Charette, 1996b; Powell & Klein, 1996) that lack of adequate

risk management processes contributes to on-going problems with IT projects.

Naturalistic perspective. Once again, the close attention paid by managers to

progress against the WBS is evidence of strong emphasis on situation assessment. Managers’

goals in these problem-arising situations were generally to resolve the problem as quickly as

possible, and since most of the problem-arising situations involved client-related problems

they had an additional goal of maintaining or restoring good client relations.

Managers used two key cues for recognizing problem situations, the explicit cue of

task slippage, and the implicit cue of sensing customer unease. If the explicit cue of task

slippage was observed, managers first identified the cause of the slippage and then moved

sequentially through Control and/or Negotiation strategies depending on the nature of the

problem. If the implicit customer unease cue was sensed, managers searched for better

understanding of the situation by imagining from the customer perspective, but while they

spent some time trying to assess the problems from the customer perspective, they did not

appear to evaluate a potential action from that perspective. Rather, once they felt they had

fully understood the situation, they moved directly to a Negotiation strategy, and moved

through several iterations of negotiation tactics before finally escalating the problem if they

were unable to resolve it. Thus while there was again no evidence of the ‘imagine action’ step

of the RPD model in terms of evaluating a potential action, ‘imagining’ did take place earlier,

as managers tried to view the situation from the customer’s perspective.

Managers’ expectancies in problem-arising situations were interesting and they may

throw some light on why unanticipated problems were most often client-related, and also

why these client problems were so difficult to anticipate. Managers appeared to assume that

the clients would share the vendors’ overall expectations of how a typical project should run,

and would understand that problems would inevitably arise, which would need a willingness

to negotiate on both sides in order to arrive at a mutually acceptable solution. It seemed that

unanticipated problems often arose in projects because these expectations were not met:

clients’ expectations and understanding of the project process itself were different from those

177

of the vendor project managers. Typically the managers’ initial solutions for the problems

were based on their expectancies about the clients, and so these initial solutions were often

unsuccessful. This resulted in sequential application of strategies to fix the problem, with

managers trying ‘another tack’, as described in E.’s quotation above. A fruitful area of future

research would be the exploration of alignment of client and vendor expectations about

project processes rather than outcomes, and particularly whether taking steps to educate the

client about these processes would reduce the incidence of unanticipated client-related

problems.

Fig. 6.5: RPD model for problem arising situations in all projects

Experience Situation in Changing Context

Recognize Problem Situation

Goals:Resolve problem as soon as possible;

Restore client relations

Expectancies:Common understanding

of project processes;Client willingness to

negotiate

Implicit Cue:Customer unease; Imagine from customer perspective

Implement Controlstrategies

Apply Monitoringstrategies

Explicit Cue:Task slippage on

WBS

Will Controlstrategies work?

Implement Negotiationstrategies

No

Yes

Has Negotiationworked?

No

Yes

Experience Situation in Changing Context

Recognize Problem Situation

Goals:Resolve problem as soon as possible;

Restore client relations

Expectancies:Common understanding

of project processes;Client willingness to

negotiate

Implicit Cue:Customer unease; Imagine from customer perspective

Implement Controlstrategies

Apply Monitoringstrategies

Explicit Cue:Task slippage on

WBS

Will Controlstrategies work?

Implement Negotiationstrategies

No

Yes

Has Negotiationworked?

No

Yes

178

For problems arising during the progress of projects, the modified RPD model

shown in Figure 6.5 above captures with the decision-making processes described by

respondents, by showing the different actions triggered by the explicit and implicit cues, and

the serial application of negotiation strategies.

6.4. Summary of discussion: Research Questions 4 and 5

In this chapter, I have presented and discussed the results from the investigation into

Research Questions 4 and 5. Research Question 4 relates to the knowledge, both explicit and

tacit, that managers were drawing on in order to manage risk, and Research Question 5

explores the differences between theory and practice revealed in this study by analyzing

respondents’ decision-making processes in the light of rational prescriptions from the

literature and through the lens of the RPD model proposed by naturalistic decision making

researchers.

In Research Question 4, I examined the explicit and tacit knowledge utilized by

respondents when managing their IT projects. All of the respondents were familiar with the

explicit knowledge of the IT project risk management discipline area, and clearly applied the

recommended practices in their work. However, the variances between respondents’

practices and literature prescriptions enabled me to deduce tacit knowledge about IT risk

management practice held by this group of Hong Kong project managers. In addition to the

use of a few broad general strategies rather than many risk-specific strategies already

mentioned, managers applied rules of thumb to guide their development and control of the

work breakdown structure, and generally approached client interactions from a negotiation

standpoint rather than from a contract enforcement point of view. These aspects of tacit

knowledge were also reflected in the cues the managers observed during their Monitoring

processes, with the primary explicit cue being task slippage on the work breakdown structure,

while implicit cues were drawn from a close observation of their clients’ expectations and

interactions, which contributed to the managers’ effectiveness in their Negotiation strategies.

In Research Question 5 I considered the decision-making processes of the

respondents in specific situations at the start of, and during, their projects, and reviewed their

actual practice in the light of rational risk management prescriptions and the RPD model of

naturalistic decision-making. Aspects of both rational and naturalistic decision-making

processes were evident in all the situations, but rational methods were most apparent in the

179

initial assessment situations of ‘troubled’ projects. The modified RPD models shown in

Figures 6.2, 6.4 and 6.6 encapsulate the hybrid nature of the practices described by

respondents, showing their key focus on situation assessment, but also highlighting the more

rational aspects of their decision-making, particularly evident in the ‘troubled’ projects. While

an ‘imagine action’ step was not described by respondents in this study, there was evidence of

an earlier ‘imagining’ process in the problem-arising situations, where part of the assessment

involved viewing the situation from the customer perspective. This analysis of respondents’

decision-making processes revealed two areas worthy of further study, namely the

management of contingencies added to the schedule or budget by the pre-sales team, and

differing client and vendor expectations about the actual process of carrying out an IT

implementation project.

180

C h a p t e r 7

7CONCLUSIONS AND IMPLICATIONS

7.1. Introduction

In this final chapter, I discuss the overall findings of this research in the light of the

literature reviewed in Chapter 2, and the methodology described in Chapter 3 to address the

research questions. I present conclusions and implications from these findings, and discuss

some limitations and opportunities for future research.

In Chapter 2, I discussed the literature relating to IT project risk management, to tacit

knowledge, and to naturalistic decision-making, and drew together these diverse strands of

research to show the need for further exploratory investigation into the actual practice of IT

project risk management. I discussed the rational assumptions behind literature prescriptions

for risk management in IT projects, and suggested that these assumptions may not reflect

what actually happens in practice. I noted that while previous research has identified many

risk factors that may impact on IT projects, much less is known about how IT project

managers carry out risk management practices and what strategies they apply both to protect

their projects against potential problems and to address problems that actually arise. In

particular, little attention has been paid to the tacit knowledge that IT project managers may

bring to their management practices, and how this tacit knowledge may influence their

approaches to the application of rational prescriptions for risk management and decision-

making in their projects. Finally, I noted that most previous IT risk management research has

focused on in-house custom software development, rather than software package

implementation, and suggested that, with the increasing trend away from custom system

development towards software packages (Russo, 2000), more attention was warranted for

these types of IT projects.

This review of literature culminated in the presentation of five inter-related research

questions, focusing on vendor driven IT projects, with a software package implementation

emphasis. Research Question 1 was designed to identify the key risks actually attended to by

managers of these projects, rather than simply extracting a list of all possible risks the

managers might be able to nominate. These key risks were compared with factors identified

181

in the literature both for custom development and for software packages. Research Question

2 explored the risk management practices used by respondents, again with an emphasis on

what they actually did and how this compared with literature prescriptions. Similarly,

Research Question 3 explored the strategies actually used by managers to address both risks

(potential problems) and actual problems that arose in the course of their projects.

Underpinning these first three questions was the goal of shedding light on any differences

observed between actual practice and theory, and this goal was addressed by Research

Questions 4 and 5. Research Question 4 drew on previous research into tacit knowledge and

aimed to tease out instances of both explicit and tacit knowledge being applied by managers

that might help to explain congruencies and variances between their practice and literature

prescriptions. Finally, Research Question 5 examined managers’ decision-making processes in

key situations from rationalistic and naturalistic decision-making perspectives in order to

investigate how well each of these theories described the actual practices observed and

reported on in the first four research questions.

Chapter 3 presented the methodology chosen to address these research questions,

and I showed how researchers from two different domains, the tacit knowledge domain (see,

for example, Sternberg & Horvath, 1999) and the naturalistic decision-making domain (Klein

et al., 1989), had converged on a very similar methodology, based on Flanagan’s (1954)

critical incident technique, for eliciting tacit knowledge in situations requiring expert

judgment. This methodology was particularly applicable to the requirements of the current

research because it suited the exploratory nature of the research, supporting the capture of

respondents’ interpretations of their experiences, rather than the researcher’s preconceived

ideas. It was also well designed to elicit details of what actually happened in specific

situations, rather than what respondents thought ought to happen, and has been shown by

previous researchers to be powerful in eliciting tacit knowledge. It is also worth noting that,

as the results presented in Chapters 4 to 7 demonstrate, use of this method was able to

surface a number of gaps and variances between actual practice and prescribed theory that

have not been identified in previous studies based on more typical survey techniques.

In Section 3.3, I explained the detailed research procedures, applying this

methodology to a sample of 25 Hong Kong IT project managers, and discussed how the data

collection process moved from an initial very exploratory approach to a more confirmatory

approach in the later stages. The respondents were primarily drawn from a variety of IT

182

vendor organizations within Hong Kong, ranging from small software houses serving local

clients, to Hong Kong branches of large multinational vendor and consulting firms.

However, I also interviewed three respondents, one from a commercial firm and two from

government organizations, to gain a contrasting in-house perspective. The respondents

discussed projects in which they had played a number of different roles, including vendor

project manager, in-house project manager, project executive and pre-sales consultant, and

described 60 different projects in total, with 39 of these projects being covered in sufficient

depth to contribute to a project-by-project analysis.

While all respondents were located in Hong Kong, they represented a very

cosmopolitan group, since all of them had trained and worked abroad for part of their

careers, and all had experience of working in different cultures. The majority of respondents

were aged between 30 and 49 and all had experience with at least five major projects. Even

the apparently more junior respondents had been identified by their senior executives as

‘experts’ in the project management field. Eight of the managers were recommended for their

skills as ‘rescuers’ of ‘troubled’ projects, and this group in particular provided some

interesting results in their discussion of ‘troubled’ projects that contrasted with the results

from ‘routine’ projects.

Chapter 3 also presented the analysis and coding processes used to investigate the

research questions described above. A qualitative content analysis procedure was used,

supported by the NVivo version 2.0 software package. The transcripts were coded several

times to address different aspects of the research questions, and, particularly for Research

Question 1, a cross referencing between coding under two different frameworks was used to

check for consistency of coding. Since the respondents typically described two or three

different projects during their interviews I was able to analyze the transcripts both across

project managers and across projects.

In Chapters 4, 5, and 6, I discussed the results and analysis of the interview

transcripts for each of the research questions outlined above. In the next section, 7.2, I

discuss the conclusions and implications derived from the investigation of these research

questions. In the subsequent sections I review the implications that can be drawn for theory

and for practice, and discuss the limitations for this research. Finally I conclude with an

outline of further research that can now build on the foundation created here.

183

7.2. Conclusions about each research question

In this section I discuss the implications of the results and discussion presented in

Chapters 4, 5, and 6 about the research questions that form the basis of this research.

7.2.1. Research Question 1

The analysis for Research Question 1 focused on identifying the risk factors that

respondents actually attended to in their projects, and comparing these factors with lists of

factors identified in the literature. As a result of this analysis, I found that project managers

from vendor firms considered risks arising from three major perspectives, namely risks

situated with the vendor firm itself, risks associated with the client, and risks associated with

any third parties to the project. Most of the previous research into risk factors appears to

have focused on in-house development (for example, Barki et al., 1993; Schmidt et al., 2001),

and hence this distinction into three perspectives has not been made previously. However,

for project managers working on vendor-driven projects it is very important to be clear about

the source of potential risks, since this may influence the strategy used to address the risk.

For example, later in Chapter 5 where I discussed strategies used to address different risks,

issues relating to project management within the vendor firm were generally addressed with a

Control strategy, while a Negotiation strategy was applied to issues relating to client project

management.

The teasing out of the risk factors into these different perspectives also contributed

to the identification of several new risks, summarized in Table 4.4, that have not been

previously recognized in the literature. Several of these new risks relate particularly to vendor-

driven projects, and can be considered from the three perspectives noted above. Firstly, from

the vendor perspective, project managers have another layer of senior management to deal

with and must be aware of risks from internal competition for resources, and from their

personal ability to ensure adequate support from their own management for their projects. In

addition, respondents highlighted the need to be constantly vigilant for threats from their

competition in the marketplace, and threats to their own firm’s reputation. Secondly, from

the client perspective, project managers must be aware of potential risks related to contract

terms and conditions and the impact on future business if performance on the current

project was poor. Issues related to potential problems with the client IT department and their

willingness to support an outsourced project, and general client readiness for the system,

184

were also of concern. Finally, also from the client perspective, project managers were aware

of risks arising from their lack of familiarity with the detail of the client’s technical

environment. This last risk spilled over into the third party perspective, in that if integration

with third party products was required, the vendor project manager must address

compatibility issues both between the third party product and the vendor product, and

between the third party product and the client’s existing technology.

An additional area of risks cut across all three perspectives and related particularly to

the location of respondents in Hong Kong. A number of the projects described by

respondents involved multiple country client locations or overseas third parties, or required

significant interaction between the local project team and an overseas vendor head office.

Each of these aspects of projects, if present, was identified by respondents as adding an extra

layer of potential logistical and communication risks to the project. It might be expected that

these aspects have not been identified in previous research, since these studies typically

appear to have been focused on in-house development within large organizations. However,

it is surprising that the issue of working at multiple sites or branches within one organization

has not been identified as a risk factor previously, since multiple site implementations within

a single country are likely to be at risk from the same logistical and local variation issues that

occur on a larger scale in multiple country implementations.

Some specific new technology risks identified in this study related to the focus on

software package implementation projects, and while issues relating to new technology have

been identified by previous researchers (for example, Schmidt et al., 2001), package-specific

technology risks have received less attention, although there is some overlap with factors

identified by Sumner (2000). For package implementation projects, two key new risks arise,

namely the risk associated with the amount of customization to the product required by the

client, and the risk associated with the complexity of the solution i.e. the number of different

modules or functionalities to be implemented. Both of these risks cut across the vendor and

client perspectives noted above, since project managers reported the need to negotiate with

their clients to minimize the customization requirements, and with their own internal

management for support to modify the package for any non-negotiable client requirements,

and for support to quickly address problems that arose when unusual and potentially

untested combinations of modules were required by a particular client.

185

In addition to identifying new perspectives from which to view risks, and new risk

factors within these perspectives, I also noted in Chapter 4 that previous literature appeared

divided over the relative importance of two key risk factors, lack of top management support, and

schedule and budget management. While some researchers rank the former risk as one of the most

important (Schmidt et al., 2001), others do not mention it, and rate the latter factor much

higher (Boehm, 1991). In the present study, I found that lack of management support did not

appear in the seventeen most often mentioned risks, while schedule and budget management was

mentioned most often. A closer examination of the type of respondents and target reader

audience in these previous studies suggested that Boehm’s study might be aimed more at the

operational level of management, while Schmidt at al. may have captured more of a strategic

management focus. This conjecture was supported by an examination of the contrasting

comments of the three project executive respondents in the present study, who made clear

distinctions between responsibilities appropriate at project manager level, and responsibilities

at the executive level, with the issue of ensuring adequate top management support being a

project executive responsibility, while the project manager’s primary responsibility was seen

as managing the project schedule and budget. Similar findings have been reported in a

comparison of IT executive and IT project manager roles in Norway (Karlsen, Gottschalk, &

Andersen, 2002). Karlsen et al. note that the IT executives in their survey were typically more

externally focused, while the IT project managers were more internally focused. Clearly more

research is needed in this area to distinguish between those risk factors that can appropriately

be considered to be part of a project manager’s focus, as opposed to those that must be

addressed at higher levels of the organization.

The results discussed above were drawn from an analysis of risk factors identified at

the project manager level. I also investigated risks identified at the project level, using the 39

different projects that were discussed in depth by the respondents as a basis for this part of

the analysis. In particular, I compared potential problems anticipated at the start of projects

with problems that arose during those projects, in order to identify, firstly, those problems

that most typically still occur in projects even if they have been anticipated at the start, and,

secondly, which of the problems that arise are most often unanticipated at the start. I

examined the ten ‘troubled’ projects to identify the problems that their managers considered

were responsible for the projects becoming ‘troubled’.

186

The most likely problem to arise even after being anticipated and addressed with

mitigating actions was schedule and budget management, often compounded by the second

and third most common problems, vendor staffing issues, and difficulties arising from the

newness of technology. All three of these problems were seen by respondents as having their

underlying cause in inadequacies in the pre-sales analysis, resulting in either an overall

shortfall in budget or time allowance, or in an underestimation of the skill-set required or of

the extra work involved in coming to grips with new technology. On the surface, it might not

seem so surprising that managers would try to shift the responsibility for problems in their

projects away from themselves. However, there was a strong consensus among respondents

that the issue of over-selling or over-promising was almost endemic in the industry, with the

pressures to win business, and the desire to satisfy customers often outweighing more

cautious responses to client requests for proposals. Indeed, respondents were honest about

their own tendencies to present the ‘best picture’ when they themselves had been involved in

pre-sales work.

Given this tendency to ‘paint a rosy picture’ to clients at the start of projects, it is not

surprising that the most common unanticipated problems tended to be client relationship

problems. Client relationship problems appeared unlikely to be expected at the start, partly

because signs that such problems might occur are typically not evident early in the vendor-

client relationship, and also because their intangible nature makes them difficult to quantify

and assess within the rational risk assessment frameworks that these firms used at the pre-

sales stage. However, the frequency with which client relationship problems did occur

supports the need for strong measures to build good and lasting relationships at the start

(Russell & Chatterjee, 2003), and as I discussed later in Chapter 6, relationship building

strategies were among the key Negotiation strategies used by managers.

The six problems highlighted in the ‘troubled’ projects were a combination of the

most often anticipated problems and the most common unanticipated problems, with client

expectations featuring highly in all three areas. The prominence of client expectations in both the

anticipated, unanticipated and ‘troubled’ problem sets is particularly significant. Customer

satisfaction is related not only to what has been delivered and how well the process went, but

also to what was expected in the first place. Indeed, disconfirmation of expectations, i.e. the

difference between outcomes and expectations, is strongly positively correlated with

customer satisfaction, as noted by Szymanski and Henard (2001) in their recent meta-analysis

187

of customer satisfaction studies. What this means is that if expectations are high to start with,

and the outcomes are not quite as high as those expectations, then customers will ultimately

be dissatisfied, even if final functionality and performance is a good match with

requirements. As Szymanski and Henard comment, “negative ramifications … can result

when firms overpromise and under-deliver”. Customer satisfaction is obviously extremely

important for vendor firms since it has a major impact on their reputation and can

significantly influence future business, particularly since new clients are likely to ask for

reference sites when assessing vendor bids (Russell & Chatterjee, 2003). Thus strategies to set

realistic client expectations are a very important part of the project manager’s repertoire, as

was highlighted in the investigation of Research Question 3, discussed later in Section 7.2.3.

7.2.2. Research Question 2

The results and discussion for Research Question 2 on risk management practices

revealed a very close match between practice at the pre-sales stage of IT vendor-driven

projects and the most typical risk management prescriptions in the literature (see, for

example, PMI Standards Committee, 1996). However, the three in-house respondents

reported a more mixed approach to risk management, with only one referring to conducting

a rigorous risk assessment at the proposal stage for in-house projects, and one referring to a

careful evaluation of potential vendors for outsourced projects, but not an evaluation of risks

for the project itself. All three in-house respondents gave the impression that risk

management generally was a low priority for them and got little attention, in contrast to the

vendor project managers who, with the exception of two respondents from the one very

small firm, described rigorous risk assessment processes at the pre-sales stage.

While it is not possible to draw conclusions from only three in-house respondents,

the contrast between these respondents’ view of risk assessment and management and that of

the vendor project managers was marked. This suggests that a fruitful line of future research

could be to replicate the present study with a larger in-house sample in order to determine

whether the variety of problems with IT projects, attributed in the literature to shortfalls in

IT project risk management (for example, Charette, 1996b; Keil et al., 1998; Powell & Klein,

1996), is largely an issue for in-house projects, and might be alleviated by a move to

outsourced projects under vendor project management control.

188

Another strand of the IT project risk management literature (for example, Barki et al.,

2001) recommends a contingency approach of matching the risk management and

development methodologies to the level of risk exposure in a project. I found little evidence

that such an approach was applied, except for the addition of extra contingencies on time

and budget for projects that were perceived to be high risk. Indeed, as I note later in

discussion of the results from Research Question 3 on strategies and Research Question 4 on

tacit knowledge, in practice, managers applied a few broad strategies that were effective in

addressing a wide range of potential problems. This use of a few strategies to cover many

different possible situations, rather than developing specific strategies for each potential

problem, may suggest that managers would consider a contingency approach, with its

emphasis on situation-specific actions, to be an impractical alternative in practice.

There was a weak link in most firms’ risk management practices in the hand-over

from pre-sales to implementation teams, with an apparent lack of follow-up on execution of

the risk management plans prepared at the pre-sales stage. This gap was potentially

compounded by the practice in most firms of separating the risk assessment duties from the

pre-sales and implementation teams. Although this separation of duties is commendable in

terms of reducing potential conflict of interest for the risk assessors, it removes responsibility

for, and a sense of ownership of, the risk management plan from the project implementation

manager thus aggravating the problem of failure to execute risk mitigation actions. It also

appeared that firms did not capitalize on the store of tacit knowledge present within their

experienced project managers by using post-project reviews to encourage knowledge transfer

among staff. These weaknesses of risk management practice have implications for practice,

which are discussed later in Section 7.5.

7.2.3. Research Question 3

Turning to Research Question 3, the investigation reported in Chapter 5 revealed

sharp distinctions between the risk management strategies applied at the pre-sales stage and

those applied during the implementation of the project. While vendor project managers

described, as discussed above, a rigorous process of risk assessment at the pre-sales stage, the

key risks that they identified from the pre-sales stage in terms of impact on their projects

were related not so much to potential problems for the project itself, but rather to the vendor

firm’s own performance in two areas, namely the risk of over-selling or over-promising in

189

order to win the sale, and the risk of failing to fully understand the requirements because of

the desire to minimize pre-sales costs.

Both of these risks can have a significant impact on the ability of the project manager

to meet the project time and budget targets. While neither risk has received much attention in

the literature, clearly the vendor firms were well aware of potential problems that could

result, and most respondents described the strategy of the separation of risk assessment

duties already alluded to as fundamental in protecting the firm from the impact of these two

risks. In addition, two of the smaller firms adopted a pre-project partnering approach to

separate the requirements specification work into a stand-alone chargeable activity. This

tactic enabled them to ensure they fully understood client requirements before preparing a

bid for the implementation stage, and had the added bonuses of allowing them to exercise

some influence over the framing of the requirements and also to establish a working

relationship with the client, both of which could provide some advantage for their firm in the

subsequent bidding process for the implementation project. The contributions of pre-project

partnering to subsequent project team performance have been investigated in the slightly

different context of in-house projects by Jiang, Klein et al. (2002), and the success of the pre-

project partnering strategy described by respondents from two firms in this study extend the

argument for the positive effects of this approach to the vendor-driven software project

arena.

The investigation into strategies applied during the project implementation

uncovered a substantial variation from the typical prescriptions in the literature (for example,

PMI Standards Committee, 1996), which recommend a risk-by-risk approach to plan actions

to address each of the risks identified for a project. Respondents in the present study

described four broad categories of strategy, of Control, Negotiation, Research, and Monitoring,

which they applied generally to all their projects, regardless of specific risks that had been

identified. Building on the view of risk strategies derived from respondents in this research,

the large number of potential risks (43 identified in this research as discussed in Chapter 4)

were consolidated into four major categories according to which strategy is best applied to

them. Thus most of the risks placed in the project management theme in Chapter 4 fall into

the category addressed by the Control strategy. Most of the risks in the relationships theme are

addressed by Negotiation strategies. Technology and solution related risks are amenable to a

Research strategy, and all the remaining risks fall under the Monitoring strategy.

190

As noted in Section 7.2.1 above, respondents were keenly aware of the importance of

establishing good, lasting relationships with their clients, both for the success of the current

project and to maximize the opportunities for future contracts for their firms (Russell &

Chatterjee, 2003). Even though client relationship risks were rarely specifically identified in

the pre-project risk assessments, Negotiation strategies, particularly relationship-building and

expectation-management, were routinely applied at the beginning of projects as a general

mitigation for all ‘soft’ relationship risks, whether anticipated or not, but in spite of this

approach, problems with client relationships and unrealistic client expectations were the two

most often unanticipated problems reported by these respondents. While interpersonal skills

in relationship building are widely recognized as one of the core capabilities for IT

organizations (see, for example, Feeny & Willcocks, 1998; Lee, Trauth, & Farwell, 1995;

McFarlan & Nolan, 1995; Rockart, Earl, & Ross, 1996), typical project management literature

and training courses do little to help novice managers develop skills in these areas, and the

prevalence of such problems in projects suggests that firms could do more in assisting their

staff development here.

The model developed in Chapter 5, of consolidated risks and broad strategies to

address them, is well grounded in the data analyzed in this study, and breaks new ground in

presenting a practical solution to the difficulties inherent in the typically recommended

practice of planning actions against potential problems on a risk-by-risk basis. The use of a

few broad strategies enabled the managers to control and respond to a wide variety of

potential problems in their projects, without having to attempt to quantify each possible risk

in terms of probability and impact. As Pfleeger (2000) has noted, such quantification is

particularly difficult in situations of high uncertainty, and can often amount to little more

than guesswork. Through their reliance on these broad general strategies, the managers in the

present study were able to reduce a lengthy and unwieldy checklist of potential risks to a

manageable form and to avoid spending limited time and resources on specifying individual

responses to each particular risk. While this model has much to recommend it to

practitioners as a pragmatic and well-tested approach to managing risks in IT projects, the

difficulties in managing the ‘soft’ relationship risks suggest a need for further investigation

into strategies to address this particular set of risks.

191

7.2.4. Research Question 4

In Research Question 4, I examined the explicit and tacit knowledge utilized by

respondents when managing their IT projects. All of the respondents were familiar with the

explicit knowledge of the IT project risk management discipline area, and clearly applied the

recommended practices in their work. Managers referred to their university studies as one

source of explicit knowledge, but also noted that they had gained further knowledge from in-

house training courses or from external specialist training providers such as the Project

Management Institute (PMI). In addition, most of the firms required the use of standard

project management methodologies and templates, and the application of these templates

assisted managers in the preparation of the bid during the pre-sales stage. The methodologies

prescribed by the firms included a risk assessment template that gave guidelines on the

various risks to consider and prompted managers to assess each risk on a yes/no or

low/medium/high scale. The templates typically required mitigating actions to be specified

for the high ranked risks, and the results of these assessments were used to determine the

level of contingency time and budget to be added to the project bid. However, as noted in

the section on risk management practices there was a noticeable gap in the hand-over from

pre-sales team to implementation team in terms of the follow-up and execution of planned

mitigating actions, and the respondents’ management of risk during the implementation of

their projects reflected a high level of application of tacit knowledge, rather than of explicit

knowledge prescriptions.

The variances between respondents’ practices and literature prescriptions enabled me

to deduce tacit knowledge about IT risk management practice held by this group of Hong

Kong project managers. One key aspect of the managers’ tacit knowledge has already been

discussed above -- the use of a few broad general strategies to manage risk in their projects,

rather than the development and application of many risk-specific strategies. This approach

reflected managers’ tacit knowledge about how best to manage their own time and resources,

and about how to be best prepared for whatever problems might arise, whether or not they

were anticipated at the start of the project. Managers also applied rules of thumb to guide

their application of Control strategies, and generally approached client interactions from a

negotiation standpoint rather than from the contract enforcement point of view that often

predominates in the literature, particularly with reference to issues of change control.

192

While some of the items of tacit knowledge identified (for example, the Negotiation

strategy of balancing cost, schedule, and scope) appear to be widely known amongst

practitioners, they have not received attention in the academic literature, and hence fall into

the category of collective implicit knowledge described in Chapter 2. Other items of tacit

knowledge may seem very commonplace, and on the surface appear to add little to the body

of explicit knowledge. However these items reflect very practical rules of thumb, developed

through experience, which address the question of how to enact the basic steps found in any

project management guide. For example, development and control of the WBS are

fundamental steps in all prescriptive literature about project management and yet

respondents felt it important to highlight what they had not learned from the basic instruction

courses, but had found out through their own experience, namely, the rules of thumb on

developing the WBS (break down tasks to a maximum of one week duration for practical

control) and on controlling progress on the WBS (probe for specific evidence to support

claims about the percentage completion of tasks). Similarly, while standard literature

prescriptions recommend strict adherence to specified requirements, the application of

change control, and the use of escalation to quickly resolve any problems, the respondents in

this study described a much more flexible approach, with a rule of thumb of trying to view

the situation from the customer’s perspective in order to build and maintain trust with their

clients. As noted by Sabherwal (1999), while trust between customers and the IT project

team is important in any IT project, it is particularly important to establish trust in

outsourced projects where the customer and vendor parties have no prior relationship with

one other to build on. The experience of the managers in the current study had led them to

place high importance on the need to build and maintain a trusting relationship from the

start. In order to achieve this, they were very watchful of their client interactions, and they

showed a reluctance to rely on the contract specifications to enforce progress, preferring to

negotiate agreement, and being prepared to waive contract conditions at times in order to

maintain the overall relationship with their clients.

Both explicit and tacit knowledge were also evident in the cues the managers

observed during their Monitoring processes. The primary explicit cue was task slippage on the

WBS, as might be expected from the managers’ close adherence to the standard practice of

using a detailed WBS to control project progress. However, managers also drew implicit cues

193

from a close observation of their clients’ expectations and interactions, which contributed to

the managers’ effectiveness in their Negotiation strategies.

Indeed the managers’ emphasis on their negotiation skills suggests that IT project

risk management might be better conceptualized as a form of negotiation, rather than as a

process of risk assessment and decision making. Much of the risk management literature has

focused on controlling tangible outcomes such as performance against schedule and budget

through the application of strong Control strategies, while neglecting the complex relationship

issues that are difficult to anticipate and that require strong interpersonal and negotiation

skills to manage successfully. The tacit knowledge revealed in the interviews in this research

suggested a much more complex interplay of give and take, with the vendor managers

mentally ‘taking the pulse’ of their relationships throughout the project.

7.2.5. Research Question 5

In Research Question 5 I examined decision-making in specific situations described

by respondents at the start of, and during, their projects, and reviewed the congruence and

gaps identified between their actual practice and risk management prescriptions in the light of

rational and naturalistic decision-making theory. I categorized the problem situations

described by respondents into three groups: initial assessment situations in both routine and

‘troubled’ projects, and problem-arising situations in all projects.

While there was evidence that, when making initial assessments of projects, managers

implicitly carried out the first stage in the rational prescription of risk identification, it

appeared that they applied the second stage of formally evaluating probability and impact of

potential problems only in the initial assessment of ‘troubled’ projects. I conjectured that, as

Sutcliffe and McNamara (2001) have suggested in their study of decision-making practices in

organizations, in these ‘troubled’ projects, the high risk of damage to the firm’s reputation

together with high levels of uncertainty about what was going wrong, caused project

managers to act more cautiously and fall back on prescribed practices in order to get the

projects back on track.

In the problem-arising situations, managers’ close adherence to rational prescriptions

for monitoring risks was revealed by their use of the Monitoring strategy discussed earlier and

in particular their attention to the explicit cue of task slippage against the WBS. However,

194

when problems actually became apparent, managers varied from the rational prescriptions

for risk response in two areas. Firstly, managers did not refer to drawing on planned

contingencies to address anticipated problems, which they would be expected to do, if they

had followed rational prescriptions and explicitly distinguish between expected time/budget

and additional contingency for a given task. I suggested two possible explanations for this

observation. It could be that anticipated problems were not considered to be actual problems

by these managers until after all planned contingency had been applied, and thus when asked

to describe situations where problems arose in their projects, they focused on those problems

that had gone beyond the bounds of all their original planning. However, it is also possible

that contingencies allotted at the pre-sales stage may have simply been absorbed into the

general schedule and budget, without retaining any distinction between the expected time and

budget for a high-risk task, and the additional contingency applied to that task. This lack of

distinction between expected targets for tasks and their contingency could arise either from

the failure to explicitly adhere to the rational risk management process during initial

assessment, or from the gap, noted in the discussion on Research Question 2, in the

handover from pre-sales to implementation teams. If the distinction between expected

targets and contingencies is not maintained, this would be a serious consequence of the lack

of explicit application of risk management prescriptions, since part of the purpose of allotting

contingency is to allow early warning of a problem, before the contingency is applied, so that

managers can take any extra remedial action that may be required. Further research is needed

into how managers actually handle planned contingencies, since failure to handle them

according to rational prescriptions could well contribute to some of the problems that have

been identified in IT project management.

The second area of variance from rational prescriptions occurred with unanticipated

problem situations, where I found no evidence of option generation in order to identify the

best solution. Instead managers described a reactive and sequential process of trying one

strategy after another in an attempt to resolve the problem. These unanticipated problems

were most commonly client-related, and respondents typically described the application of

Negotiation strategies to address them. As noted above, managers’ negotiation skills were

paramount in dealing with these unanticipated problems.

Of particular interest in the problem-arising situations was the apparent mismatch

between managers’ and clients’ expectancies of the project process as distinct from the project

195

outcomes. While managers generally had an expectancy that the typical project would run into

some difficulties that would require a certain amount of negotiation and compromise to

resolve, they often found that their clients did not share this expectation, and this lack of

common understanding about typical project process could cause a deterioration in the client

relationship when unexpected problems did arise. Even if the final outcomes of the project

were delivered to the client’s satisfaction the client could remain somewhat dissatisfied

overall because the process did not run as smoothly as they expected. There has been some

research indicating the importance of setting client expectations about information systems

outcomes at the appropriate level (Staples, Wong, & Seddon, 2002), and as noted above in

Section 8.2.1, the difference between outcomes and expectations is strongly positively

correlated with customer satisfaction (Szymanski & Henard, 2001). However, as noted by

(Russell & Chatterjee, 2003), with the increasing focus on outsourcing IT services, further

research is needed in the area of ensuring client satisfaction with the performance of the

project process as well as the outcomes.

In all the problem situations, managers placed a high degree of emphasis on situation

assessment, through their detailed examination of the WBS. This aspect of their approach

closely corresponds with the RPD model of decision-making, and thus I proposed modified

RPD models to capture managers’ actual processes. These modified models show the

rational explicit aspects of the respondents’ decision-making in initial assessment situations,

embedded in a naturalistic decision-making framework that captures the implicit features of

their approach. These models reflect the hybrid nature of manager’s risk management and

decision-making processes in practice.

7.3. Implications for theory

The gaps between theory and practice can be analyzed from two perspectives. Firstly,

we can consider the extent to which managers do not adhere to the literature prescriptions;

and secondly, we can examine the extent to which the theoretical prescriptions allow for the

realities of risk management in actual project situations. From the first perspective, this study

has shown that while respondents were certainly aware of, and used rational risk management

methods, there were gaps in their application in certain areas. Some of these gaps (e.g.

handover from pre-sale to implementation teams, and management of contingencies) may

indeed have contributed to the difficulties these managers experienced in controlling certain

196

areas of their projects. From the second perspective, however, the analysis of decision-

making approaches presented in Research Question 5 suggests other gaps between the

managers’ practice and theoretical prescriptions (for example, the lack of development of a

specific response to each specific high ranked risk) reveal the impracticalities of the

prescriptions and their failure to reflect practical realities.

The RPD models developed in Chapter 6 provide better descriptions of actual practices

than the rational model, and better accommodate the tacit knowledge being applied by

respondents in this study. The incorporation of some aspects of the theoretical prescriptions

derived from the rational model into these RPD models also provides better and more

pragmatic prescriptive guidance for practitioners.

The study has demonstrated the power of the critical decision method of

interviewing, in terms of its ability to uncover new dimensions of topics like the risk factors

applying to IT projects, which appear on the surface to be well-researched already. Just

asking respondents to identify key risk factors and the strategies they used to manage risk

would have been unlikely to reveal anything new or anything more than the respondents’

espoused theories. However, the process of focusing respondents on what they had actually

done in specific projects enabled the surfacing of tacit knowledge, and the identification of

contextual and environmental details that provided important pointers to how project

managers approach real situations in practice rather than what they say they would do in

hypothetical situations.

7.4. Implications for practice

The findings from this research indicate a number of areas where IT vendor firms,

and their project managers, could derive benefits in their approaches to risk management.

Some of the variances between respondents’ practice and theoretical prescriptions have

important implications for practice, both positive and negative. On the negative side, while

vendor firms have clearly embraced the risk management theory at the pre-sales stage, there

was evidence that respondents regarded these processes as little more than checklists to be

ticked off as complete at the pre-sales stage, rather than the foundation for on-going actions

to mitigate potential risks applying to a particular project. Vendor firms would be well

advised to consider the hand-over from pre-sales to implementation teams, and particularly

how contingencies are viewed and managed by the implementation manager, in order to

197

ensure that the purpose and benefit of the contingency planning is not lost. On the positive

side, the study provides a better understanding of some practical aspects of risk management,

in particular: the need to view risks from three different perspectives (vendor, client, third

party); the relationship between the four general strategies (Control, Research, Negotiation, and

Monitoring) and the groups of risks they address; and the key explicit and implicit cues that can

provide warning of impending problems. All of these aspects can help project managers to

gain greater insight into their own understanding and handling of risk in their projects.

Finally, since these suggestions all derive from the experience-sharing of the project

managers themselves, these ideas were already available to vendor firms through the store of

tacit knowledge held by their project managers. As noted in Chapter 5, few firms in this study

appeared to recognize the potential of this store of tacit knowledge, and it seems likely that

there would be significant benefit to be obtained from the establishment of formal

mechanisms such as mentoring to improve knowledge transfer to novice project managers.

7.5. Limitations

The study reports an exploratory, qualitative investigation, and as such reflects my

own perspective as researcher. However, as described in detail in the sections on coding

processes in Chapter 3, I worked with two different risk factor coding structures and cross-

referenced these to ensure that I applied coding consistently throughout the analysis. The on-

going comparisons between my findings and literature prescriptions also helped to provide a

clear benchmark for my conclusions, and in many cases I found a closer congruence between

respondents’ practices and the prescriptions than my initial biases may have led me to expect.

Viewing the findings from the two contrasting perspectives of rational and naturalistic

decision-making theory provided a theoretical triangulation (Patton, 2002) that ensured I

maintained a balanced approach. Finally, by seeking contrasting cases, for example those of

in-house managers, pre-sales consultants, and project executives, I was able to clarify certain

themes that emerged from the data.

The interview method chosen for the study, the critical decision method, has the

limitation that it relies on managers’ self-reports and recollections of their actions, and carries

with it the assumption that these self-reports will provide an accurate picture of respondents’

actual risk management practices, including the tacit knowledge they applied in their projects.

While these limitations are acknowledged, previous use of the critical incident technique

198

((Sternberg et al., 2000) and the critical decision method (Klein et al., 1989) for eliciting tacit

knowledge has demonstrated that these techniques can effectively tease out the tacit

knowledge applied by respondents in performance of key tasks.

The respondents in this study were all located in Hong Kong and thus there is a

possibility that the findings are country/culture specific. However, the sample included

respondents from a broad range of backgrounds and nationalities, and all of the respondents

had trained or worked abroad at some stage in their careers, and had experience working in a

wide variety of different cultures. Several had worked on large multinational projects, and

most had experience with software packages that are marketed globally. Hence the

respondents may be considered to be a cosmopolitan cross-section of project managers, and

the results are unlikely to show any mono-cultural bias. While many of the risks identified by

these managers correspond to those reported in the literature previously, some risks, such as

multi-location projects and overseas third parties, were clearly related to the Hong Kong

location of the projects, and these location-specific risks also have implications for managers

in other countries, particularly in view of the increasing trend towards multinational projects

that cross traditional country/culture boundaries.

Finally, the study focused solely on packaged software implementation, and the

exploratory nature of the research placed practical limitations on the sample size. In

particular, very small cohorts of in-house project managers, pre-sales managers, and project

executives were used to test and clarify the themes emerging from the main cohort of vendor

project managers, and the size of these contrasting cohorts may be considered a limitation.

However, the in-depth analysis of information across a range of informants and

organizations in the packaged software field has enhanced our knowledge of risk

management practices within this range of empirical circumstances, and hence can increase

our confidence in applying this knowledge to new settings sharing similar circumstances to

those investigated in this research (Baskerville & Lee, 1999).

7.6. Further research

While this study provides a strong foundation for a better understanding of how risk

management processes are applied in IT projects, it also raises new questions, some of which

arise from the limitations noted in the previous section. For example, while the location of

this study in Hong Kong may be considered to be a limitation it also represents an important

199

broadening of our perspective on IT project management into a more global arena. With the

increasing trend towards globalization, and more multinational companies operating across

traditional boundaries, the insights offered here on multi-country and multi-cultural software

project implementations are particularly significant, and more investigation is needed into the

additional risks that arise when working in different locations, with different languages, and

different cultures. Similarly, while this research was limited mainly to software package

implementation projects, these projects share many features in common with custom

development projects and a future research opportunity is to explore the extent to which the

findings from the present study are applicable in other IT project settings.

Other questions arise from the findings of this study. The investigation into Research

Question 1 on risk factors suggests a possible explanation – the strategic versus operational

perspective of respondents -- for the contrasting views found in previous research on the

relative importance of two key risk factors, lack of top management support and schedule and budget

management. More research is needed in this area to distinguish between those risk factors

under the control of the project manager, and those that must be addressed at higher levels

of the organization.

While some of the gaps identified between theory and practice appear to be examples

of tacit knowledge being used to make a complex and uncertain environment more

manageable, some gaps, such as the hand-over from pre-sales to implementation teams, and

the possible failure to maintain distinctions between expected targets and contingencies,

point to potential areas of weakness that need further investigation. I also noted in the

discussion on risk management practices that in-house respondents appeared to place a lower

priority on risk management than the vendor project managers, and a worthwhile line of

future research would be to replicate the present study with a larger in-house sample to

explore further the priorities placed on risk management by in-house managers.

The tacit knowledge revealed by this study, particularly in the pragmatic approach of

utilizing broad general strategies to address a wide range of different risks, suggests further

research opportunities, both to confirm the strategy framework developed in Chapter 5, and

to explore ways of improving transfer of this tacit knowledge within organizations. The

emphasis revealed in Chapter 6 on Negotiation and Monitoring strategies suggests that future

investigations could conceptualize the project management process as a form of negotiation

200

and explore in more depth the key factors that can result in successful client interactions.

Particularly deserving of attention is the question of managing customer expectations not

only about the project outcomes but also about the project process in order to maximize the

potential for achieving customer satisfaction at the close of the project.

7.7. Self Assessment of Research Quality

This thesis describes an exploratory, descriptive field study of current risk

management practice in IT projects. The primary aim was to carry out an in-depth

exploration of risk management processes and decisions made during the course of IT

projects, with a particular focus on software package implementation projects and on the

tacit knowledge of experienced project managers. The secondary aim was to test whether

decision making theories adequately describe IT risk management decision making practice.

As discussed in Chapter 3, the research approach chosen to meet these aims was an

interpretive one, and thus measures of evaluating the rigor of this research and the credibility

of its findings are most appropriately those that have been developed in recent years for

measuring and assessing interpretive research. Following the example of recent qualitative

researchers (Davidson, 2002; Nelson et al., 2000; Trauth & Jessup, 2000; Walsham & Sahay,

1999), I present a self-reflection and assessment of my research, considering, firstly, the

soundness and rigor of the qualitative research methods used, and secondly, the degree to

which the research meets recognized principles for conducting interpretive research.

7.7.1. Application of sound and rigorous qualitative research methods

The methodology section of the thesis describes explicitly and in depth the research

methods employed in this study. As discussed there, the method chosen to investigate the

research questions was specifically selected to meet three criteria related to the overall aims of

the study, namely:

1) The method should allow participants themselves to identify what they considered to be

‘risky situations’ from their own interpretations of their experience;

2) The method had to facilitate the elicitation of what actually happens in practice, rather

than on simply reporting what respondents thought they ought to do in practice; and

201

3) The method should have the potential to prompt and assist recall of the underlying tacit

knowledge, and thus get beyond the theories or rationalizations that a person may use to

explain his/her actions.

The data collection method chosen to meet these criteria was well founded in

previous research, with its effectiveness having been demonstrated by teams of researchers

(Klein et al., 1989; Sternberg et al., 2000) from the two quite different strands of research

providing the theoretical underpinnings of this study, tacit knowledge and decision making,.

The fact that these previous researchers have used the method in more positivist research

approaches does not preclude the application of the method in a qualitative study, since

interviews are typically the primary data source for most ‘outside observer’ interpretive

studies (Patton, 2002; Walsham, 1995a). Indeed, the interviewing technique used in the

present research is particularly well suited for an interpretive exploratory study since the

semi-structured nature of the protocol provides enough guidance to ensure overall

consistency across interviews, while allowing enough flexibility to allow a degree of openness

to the field data and to meet the need to adapt and refine the investigation as suggested by

findings from the concurrent data analysis (Walsham, 1995a). Thus the choice of method is

appropriate for the type of research and is well tested in similar areas of research.

I have provided clear and explicit details of the data collection process, including a

section describing how the interview protocol was refined as the interviews progressed, and

my own reflections on the interview process. The data analysis methods are drawn from

widely recognized and respected sources such as Miles and Huberman (1994) and Patton

(Patton, 2002), and a detailed description of the coding, analysis and interpretation processes

has been provided. At all stages I have taken care to remind the reader of my personal

involvement as researcher, and the fact that the analysis represents my personal

interpretations of the data. The use of the first person in the descriptions, together with

phrases such as “develop my own interpretations of the data” reinforce this recognition that

the analysis presented is my own construction of the data, reflecting my own theoretical

interests, personal background and experience, and the development of my thinking through

the analysis process. In the penultimate section of the methodology chapter (Section 3.5) I

discussed my perspective as researcher and drew attention to my role in the research. In this

section I also explicitly qualified my personal biases and detailed the steps I have taken to

202

minimize the effect of these biases on my interpretations of the data (Golden-Biddle &

Locke, 1993).

7.7.2. Recognized principles for conducting interpretive research

The research described in this thesis falls into Klein and Myers’ (2001) category of

“applying an interpretive frame of reference in empirical investigations” and as such may

appropriately be evaluated against the seven inter-related principles proposed by Klein and

Myers (1999) for the evaluation of interpretive field research in information systems. I briefly

review each of these principles below, and assess this research against these seven criteria.

The fundamental principle of the hermeneutic circle: According to this

principle, understanding is only achieved by iterating between a consideration of the inter-

related parts that make up a whole, and a consideration of the complex whole. Klein and

Myers (1999) argue that this principle should underpin all interpretive research, and that it is

the foundation for the following six principles. They note that the use of this principle is

often implied rather than explicitly stated in interpretive research, suggesting that it can be

seen in the way the researcher iterates between different interpretations of the data, moving

back and forth between an understanding of the parts and an understanding of how these

parts all fit together into a whole. While I have not explicitly used a hermeneutic

interpretation in the current study, the application of this principle is evident both in the

development of the two risk factor coding frameworks discussed in Section 3.4.1 and in the

iteration between data and two contrasting decision-making theories used to develop the

analysis discussed in Chapter 6.

The principle of contextualization: The principle of contextualization involves

setting the research topic and subjects in their “social and historical context so that the

intended audience can see how the current situation under investigation emerged” (Klein &

Myers, 1999). In Chapter 3, I have provided detailed information about the respondents,

their backgrounds, the firms they worked for and the types of projects they worked on. At

various stages throughout the discussion of the five research questions, I explicitly draw on

aspects of the background or environment of a particular project manager or a particular

project to ensure that the comments I am making at that stage are set in the appropriate

context. Following are two typical examples of this contextualization. In Chapter 4, I note

that the new risk factors identified in this study relate specifically to three contextual aspects,

203

namely the vendor-driven nature of the projects, the primary focus on software package

implementation, and the location of most of the projects in Hong Kong. In Chapter 6, when

discussing decision-making in ‘troubled’ projects, I discuss how the background of one of the

project managers at an early stage in her career, led her to make misguided assumptions

about the project she was ‘rescuing’, and contrast the process she applied from a basis of

inexperience and false assumptions with the more typical approach applied by more

experienced project managers when faced with ‘troubled’ projects.

The principle of interaction between the researcher and the subjects: This

principle involves the recognition of not only my own influence on the interpretations that

can be drawn from the data, but also the participants’ role in interpreting their experiences

and choosing what to discuss. As noted in Section 3.3.4, I allowed the respondents

themselves to be the judge of which projects they wished to discuss, within the broad

guidelines I had set, and thus the results represent both the participants’ interpretations of

what might be relevant and my interpretations of the ideas they presented. Further, as I also

note in Section 3.3.4, after 14 interviews I decided to take a more confirmatory approach

with respect to certain aspects that were emerging from the concurrent data analysis. Thus

my interpretations of the data I had collected at that stage influenced the discussion and data

collection in subsequent interviews. Both of these points are clearly acknowledged in Section

3.3.4. In Section 3.5, I describe my own background as a client project manager and my

perspective as researcher and explain the potential my background and perspective has to

shape the interpretations that I have made. I also clearly set out the steps I took to minimize

that influence, and discussed how the chosen method assisted in this aim.

The principle of abstraction and generalization: While generalization in the

positivist sense is not an aim of interpretive studies, there is an expectation that the insights

reported in interpretive research can be used to inform other settings (Myers & Avison, 2002;

Orlikowski & Baroudi, 1991; Walsham, 1995b). The principle of abstraction and

generalization concerns the way in which explanations regarding the particular details derived

from the study are related to more general theoretical concepts in order to provide theoretical

insight (Klein & Myers, 1999). In the present study, I have provided rich insight by drawing

on the empirical statements of my respondents to illustrate both the extent of use and the

limitations of current risk management prescriptions in the particular contexts of those

respondents. In addition, by relating data interpretation about respondents’ decision-making

204

processes to the more general theoretical concepts of two contrasting decision-making

theories, I have demonstrated a more confused picture of actual decision-making processes

in practice, thus testing the extent and limitations of these two theories to fully describe

actual practice.

The principle of dialogical reasoning: This principle requires that the researcher

demonstrates sensitivity to possible contradictions between the original theoretical

underpinnings of the research and the actual findings. The fundamental requirement for

meeting this principle is clarity in the intellectual basis of the research and its underlying

philosophical assumptions (Klein & Myers, 1999). Additionally, the researcher should explain

how the analysis developed and changed over the course of the research as various findings

emerged. I have detailed the intellectual basis of the research in Section 3.2 where I discussed

the interpretive nature of the research approach. In addition, in Section 3.4, I also gave

insight into the way in which my thinking developed as the research progressed, particularly

through the coding and analysis processes.

The principle of multiple interpretations: This principle requires the researcher to

seek out and document possible multiple view-points in order to examine conflicting

interpretations of the participants. This principle is demonstrated by my seeking out different

perspectives among respondents in the later stages of data collection in order to provide

confirmatory support for certain themes that appeared to be emerging. In particular, the

evidence from respondents with specific in-house, pre-sales, and project executive experience

helped to shed light on emerging themes and to consolidate my thinking in certain areas. This

principle is closely related to the previous principle of dialogical reasoning, and the interplay

of these two principles can be seen in the present study in the examination in Section 4.4.2 of

conflicting interpretations of the relative importance of the two risk factors lack of top

management support and schedule and budget management. Here I was faced with contradictions

between the most recent and arguably most comprehensive theory about relative importance

of risk factors and the emerging data, and also contradictions within the literature itself. By

carefully seeking out contrasting perspectives from participants in the later stages of data

collection I was able to provide some insight into why the literature contradictions may have

arisen, as well as resolve the apparent differences of opinions among participants.

205

The principle of suspicion: The principle of suspicion requires that the researcher

takes a critical perspective about the data being provided by respondents, rather than just

accepting the respondents’ comments at face value. With regard to this principle, one of the

strengths of the critical decision method was that it encouraged participants to report on a

specific project, rather than simply to discuss their general opinions, and I found that by

maintaining respondents’ focus on what actually happened I was able to draw out the gaps

between the typical risk management prescriptions, which all respondents were aware of, and

what they actually did with respect to risk management. In one area in particular, namely

respondents’ claims that schedule and budget problems were typically the result of ‘over-

enthusiastic’ selling practices during the pre-sales stage, I retained considerable suspicion,

since as I comment in the present chapter, it might well be expected that respondents would

try to shift the blame for the most common problem in their projects away from themselves.

This suspicion led me to probe managers’ comments in these areas in some detail, and in the

later stages of data collection I sought out respondents who specialized in the pre-sales stage.

As I have noted in this chapter, I found a very strong consensus across all respondents about

the problem of ‘over-selling’ and respondents were refreshingly candid about the need to

‘paint a rosy picture’ at the pre-sales stage in order to remain competitive.

7.8. Summary

The research described in this thesis has identified new risk areas not previously

identified in the literature, and has shown that the risk management strategies applied by

Hong Kong project managers, particularly those involved in implementing software package

related projects for vendor firms, are a pragmatic response to the difficulties and

uncertainties involved in addressing the many and varied possible risks affecting their IT

projects. The variances identified between practice and theory contribute to our

understanding about the tacit knowledge being applied by these respondents in the process

of managing their projects, but also highlight some potential areas of weakness that may

contribute to the occurrence of both expected and unexpected problems during projects.

This study was motivated by suggestions in the literature that continuing problems

with IT projects can be attributed, at least in part, to failure to apply rational risk management

techniques that have for some time now been considered to be ‘best practice’ in project

management. The exploratory research reported here has shown a high degree of

206

conformance to prescribed actual risk management practices at a number of key stages of

projects, and has also revealed variances between theory and practice, some of which may

indeed contribute to problems arising in IT projects. The picture revealed here of risk

management practice provides a better understanding of the complexities of managing many

different risks in a highly uncertain and changing environment under significant time and

budget constraints, and sets a foundation for further research into risk management practice

in IT projects.

207

8REFERENCES

Alter, S. (1996). Information systems: A management perspective (2nd ed.). Menlo Park, CA: Benjamin/Cummings.

Alter, S., & Ginzberg, M. (1978). Managing uncertainty in MIS implementation. Sloan Management Review, 20(1), 23-31.

Ambrosini, V., & Bowman, C. (2001). Tacit knowledge: Some suggestions for operationalization. Journal of Management Studies, 38(6), 811-829.

Anderson, J. R. (1982). Acquisition of cognitive skill. Psychological Review, 89(4), 369-406.

Anderson, J. R. (1983). The architecture of cognition. Cambridge, Mass: Harvard University Press.

Argyris, C., & Schön, D. A. (1978). Organizational Learning II. Reading, MA: Addison-Wesley.

Barki, H., Rivard, S., & Talbot, J. (1993). Toward an assessment of software development risk. Journal of Management Information Systems, 10(2), 203-225.

Barki, H., Rivard, S., & Talbot, J. (2001). An integrative contingency model of software project risk management. Journal of Management Information Systems, 17(4), 37-69.

Baskerville, R. L., & Lee, A. S. (1999). Distinctions among different types of generalizing in information systems research. In O. Ngwenyama & L. D. Introna & M. D. Myers & J. I. DeGross (Eds.), New Information Technologies in Organizational Processes: Field Studies and Theoretical Reflections on the Future of Work (pp. 49-65). Boston: Kluwer Academic Publishers.

Baskerville, R. L., & Stage, J. (1996). Controlling prototype development through risk analysis. MIS Quarterly, 20(4), 481-504.

Berry, D. C. (1987). The problem of implicit knowledge. Expert Systems, 4(3), 144-151.

Berry, D. C., & Dienes, Z. (1993). Implicit learning: Theoretical and empirical issues. Hove, U.K.: Lawrence Erlbaum.

Blackler, F. (1995). Knowledge, knowledge work and organizations: An overview and interpretation. Organization Studies, 16(6), 1021-1046.

Boehm, B. W. (1991). Software risk management: Principles and practices. IEEE Software, January 1991, 32-41.

Castillo, J. (2002). A note on the concept of tacit knowledge. Journal of Management Inquiry, 11(1), 46-59.

208

Charette, R. N. (1996). Large-scale project management is risk management. IEEE Software, July 1996, 110-116.

Charette, R. N. (1996b). The mechanics of managing IT risk. Journal of Information Technology, 11, 373-378.

Cohen, M. S., Freeman, J. T., & Thompson, B. B. (1997). Training the naturalistic decision maker. In C. E. Zsambok & G. A. Klein (Eds.), Naturalistic decision making (pp. 257-268). Mahwah, NJ: Lawrence Erlbaum.

Collins, H. M. (1993). The structure of knowledge. Social Research, 60(1), 95-116.

Davidson, E. J. (2002). Technology frames and framing: A socio-cognitive investigation of requirements determination. MIS Quarterly, 26(4), 329-358.

Davis, G. B. (1982). Strategies for information requirements determination. IBM Systems Journal, 21(1), 4-30.

Davis, G. B. (1998). Commentary: Information Systems - To Buy, Build or Customize. Accounting Horizons, March 1988, 101-103.

Dennis, & Haley Wixom, B. (2003). Systems Analysis Design (2nd ed.). New York: John Wiley & Sons.

Drummond, H. (1996). The politics of risk: trials and tribulations of the Taurus project. Journal of Information Technology, 11, 347-357.

DuBois, D. A. (2002). Leveraging hidden expertise: Why, when, and how to use cognitive task analysis. In K. Kraiger (Ed.), Creating, implementing, and managing effective training and development: State-of-the-art lessons for practice (pp. 80-114). San Francisco, CA: Jossey-Bass.

Dun & Bradstreet (HK). (2002). D&B Small and medium enterprises in Hong Kong: Dun & Bradstreet (HK).

El Louadi, M., Galletta, D. F., & Sampler, J. L. (1998). An empirical validation of a contingency method for information requirements determination. Data Base, 29(3), 31-51.

Fairley, R. (1994). Risk management for software projects. IEEE Software, 11(3), 57-67.

Feeny, D. F., & Willcocks, L. P. (1998). Core IS capabilities for exploiting information technology. Sloan Management Review, 39(3), 9-21. Retrieved Spring 1998, from

Fitts, P. M. (1964). Perceptual-motor skill learning. In A. W. Melton (Ed.), Categories of Human Learning. New York: Academic Press.

Flanagan, J. C. (1954). The critical incident technique. Psychological Bulletin, 51, 327-358.

Gable, G. G. (1996). A multidimensional model of client success when engaging external consultants. Management Science, 42(8), 1175-1198.

209

Gable, G. G. (1998). Large Package Software: a Neglected Technology? (Editorial). Journal of Global Information Management, 6(3), 3-4.

Golden-Biddle, K., & Locke, K. (1993). Appealing work: An investigation of how ethnographic texts convince. Organization Science, 4(4), 595-616.

Gordon, S. E., & Gill, R. T. (1997). Cognitive task analysis. In C. E. Zsambok & G. A. Klein (Eds.), Naturalistic Decision Making (pp. 131-140). Mahwah, NJ: Lawrence Erlbaum.

Heemstra, F. J., & Kusters, R. J. (1996). Dealing with risk: A practical approach. Journal of Information Technology, 11, 333-346.

Hsia, P. (1993). Learning to put lessons into practice. IEEE Software, 1993(September), 14-17.

Jiang, J. J., Klein, G., & Discenza, R. (2002). Pre-project partnering impact on an information system project, project team and project manager. European Journal of Information Systems, 11(2), 86-97.

Johnson, P. E. (1983). What kind of expert should a system be? The Journal of Medicine and Philosophy, 8, 77-97.

Karlsen, J. T., Gottschalk, P., & Andersen, E. S. (2002). External or internal focus? A comparison of IT executive and IT project manager roles. Engineering Management Journal, 14(2), 5-11.

Keil, M. (1995). Pulling the plug: Software project management and the problem of project escalation. MIS Quarterly, December 1995, 421-447.

Keil, M., & Carmel, E. (1995). Customer-developer links in software development. Communications of the ACM, 38(5), 33-44.

Keil, M., Cule, P., Lyytinen, K., & Schmidt, R. (1998). A framework for identifying software project risks. Communications of the ACM, 41(11), 76-83.

Klein, G. A. (1993). A recognition-primed decision (RPD) model of rapid decision making. In G. A. Klein & J. Orasanu & R. Calderwood & C. E. Zsambok (Eds.), Decision making in action: Models and methods (pp. 138-147). Norwood, NJ: Ablex.

Klein, G. A. (1997a). An overview of naturalistic decision making applications. In C. E. Zsambok & G. A. Klein (Eds.), Naturalistic decision making. Mahwah, NJ: Lawrence Erlbaum.

Klein, G. A. (1997b). The recognition-primed decision (RPD) model: Looking back, looking forward. In C. E. Zsambok & G. A. Klein (Eds.), Naturalistic decision making. Mahwah, NJ: Lawrence Erlbaum.

Klein, G. A. (1999). Applied decision making. In P. A. Hancock (Ed.), Human performance and ergonomics (pp. 87-107). San Diego, CA: Academic Press.

Klein, G. A., Calderwood, R., & MacGregor, D. (1989). Critical decision method for eliciting knowledge. IEEE Transactions on Systems, Man, and Cybernetics, 19(3), 462-472.

210

Klein, H. K., & Myers, M. D. (1999). A set of principles for conducting and evaluating interpretive field studies in information systems. MIS Quarterly, 23(1), 67-94.

Klein, H. K., & Myers, M. D. (2001). A classification scheme for interpretive research in information systems. In E. M. Trauth (Ed.), Qualitative research in IS: Issues and trends. Hershey, PA: Idea Group.

Klemp, G. O., & McClelland, D. C. (1986). What characterizes intelligent functioning among senior managers? In R. J. Sternberg & R. K. Wagner (Eds.), Practical intelligence: Nature and origins of competence in the everyday world. Cambridge, U. K.: Cambridge University Press.

Lam, A. (2000). Tacit knowledge, organizational learning and societal institutions: An integrated framework. Organization Studies, 21(3), 487-513.

Lassila, K. S., & Brancheau, J. C. (1999). Adoption and utilization of commercial software packages: Exploring utilization equilibria, transitions, triggers, and tracks. Journal of Management Information Systems, 16(2), 63-90.

Lee, D. M. S., Trauth, E. M., & Farwell, D. (1995). Critical skills and knowledge requirements of IS professionals: A joint academic/industry investigation. MIS Quarterly, 19(3), 313. Retrieved Sep 1995, from

Leonard, D., & Sensiper, S. (1998). The role of tacit knowledge in group innovation. California Management Review, 40(3), 112-132.

Lewicki, P., Hill, T., & Bizot, E. (1988). Acquisition of procedural knowledge about a pattern of stimuli that cannot be articulated. Cognitive Psychology, 20, 24-37.

Lucas Jr., H. C., Walton, E. J., & Ginzberg, M. J. (1988). Implementing packaged software. MIS Quarterly, 12(4), 537-549.

Lyytinen, K., Mathiassen, L., & Ropponen, J. (1996). A framework for software risk management. Journal of Information Technology, 11, 275-285.

Lyytinen, K., Mathiassen, L., & Ropponen, J. (1998). Attention shaping and software risk - A categorical analysis of four classical risk management approaches. Information Systems Research, 9(3), 233-255.

March, J. G., & Shapira, Z. (1987). Managerial perspectives on risk and risk taking. Management Science, 33(11), 1404-1418.

Martin, J., & McClure, C. (1983). Buying software off the rack: Packages can be the solution to the software shortage. Harvard Business Review, Nov-Dec 1983, 32-60.

McCloy, R. A., Campbell, J. P., & Cudeck, R. (1994). A confirmatory test of a model of performance determinants. Journal of Applied Psychology, 79(4), 493-505.

McFarlan, F. W. (1981). Portfolio approach to information systems. Harvard Business Review, 59(5), 142-150.

211

McFarlan, F. W., & Nolan, R. L. (1995). How to manage an IT outsourcing alliance. Sloan Management Review, 36(2), 9. Retrieved Winter 1995, from

McLeod, G., & Smith, D. (1996). Managing information technology projects. Cambridge, MA: Course Technology.

Means, B., Salas, E., Crandall, B., & Jacobs, O. (1993). Training decision makers for the real world. In G. A. Klein & J. Orasanu & R. Calderwood & C. E. Zsambok (Eds.), Decision making in action: Models and methods (pp. 306-326). Norwood NJ: Ablex.

Miles, B. M., & Huberman, A. M. (1994). Qualitative Data Analysis: An Expanded Sourcebook (2nd ed.). London: Sage.

Mintzberg, H., Raisinghani, D., & Théorêt, A. (1976). The structure of "unstructured" decision processes. Administrative Science Quarterly, 21, 246-275.

Moynihan, T. (1996). An inventory of personal constructs for information systems project risk researchers. Journal of Information Technology, 11, 359-371.

Moynihan, T. (1997). How experienced project managers assess risk. IEEE Software, 14(3), 35-41.

Moynihan, T. (2000a). Coping with 'requirements-uncertainty': the theories-of-action of experienced IS/software project managers. Journal of Systems and Software, 53, 99-109.

Moynihan, T. (2002). Coping with client-based 'people-problems': The theories-of-action of experienced IS/software project managers. Information and Management, 39, 377-390.

Myers, M. D., & Avison, D. E. (2002). An introduction to qualitative research in information systems. In M. D. Myers & D. E. Avison (Eds.), Qualitative research in information systems: A reader. London: Sage.

Nelson, K. M., Nadkarni, S., Narayan, V. K., & Ghods, M. (2000). Understanding software operations support expertise: A revealed causal mapping approach. MIS Quarterly, 24(3), 475-507.

Nonaka, I. (1994). A dynamic theory of organizational knowledge creation. Organization Science, 5(1), 14-37.

Nonaka, I., Toyama, R., & Konno, N. (2001). SECI, Ba, and leadership: A unified model of dynamic knowledge creation. In I. Nonaka & D. J. Treece (Eds.), Managing industrial knowledge: Creation, transfer and utilization. London: Sage.

Orlikowski, W. J., & Baroudi, J. J. (1991). Studying information technology in organizations: Research approaches and assumptions. Information Systems Research, 2(1), 1-28.

Pablo, A. L. (1999). Managerial risk interpretations: does industry make a difference? Journal of Managerial Psychology, 14(2), 92-107.

Parnas, D. L., & Clements, P. C. (1986). A rational design process: How and why to fake it. IEEE Transactions on Software Engineering, SE-12(2), 251-257.

212

Parr, A. N., & Shanks, G. (2000). A model of ERP project implementation. Journal of Information Technology, 15, 289-303.

Parr, A. N., Shanks, G., & Darke, P. (1999). Identification of necessary factors for successful implementation of ERP systems. In O. Ngwenyama & L. D. Introna & M. D. Myers & J. I. DeGross (Eds.), New information technologies in organizational processes: Field studies and theoretical reflections (pp. 99-119). Boston: Kluwer.

Patton, M. Q. (2002). Qualitative Research & Evaluation Methods (3rd ed.). Thousand Oaks, CA: Sage.

Pfleeger, S. L. (2000). Risky business: what we have yet to learn about risk management. Journal of Systems and Software, 53, 265-273.

PMI Standards Committee. (1996). A guide to the project management body of knowledge. Upper Darby, PA: Project Management Institute.

Polanyi, M. (1966). The tacit dimension. Garden City, NY: Doubleday.

Potter, S. S., Roth, E. M., Woods, D. D., & Elm. (2000). Bootstrapping multiple converging cognitive task analysis techniques for system design. In J. M. Schraagen & S. F. Chipman & V. L. Shalin (Eds.), Cognitive Task Analysis. Mahwah, NJ: Lawrence Erlbaum Associates.

Powell, P. L., & Klein, J. H. (1996). Risk management for information systems development. Journal of Information Technology, 11, 309-319.

Reber, A. S. (1989). Implicit learning and tacit knowledge. Journal of Experimental Psychology: General, 118, 219-235.

Reber, A. S. (1993). Implicit learning and tacit knowledge: An essay on the cognitive unconscious. New York: Oxford University Press.

Rockart, J. F., Earl, M. J., & Ross, J. W. (1996). Eight imperatives for the new IT organization. Sloan Management Review, 38(1), 43. Retrieved Fall 1996, from

Russell, B., & Chatterjee, S. (2003). Relationship quality: The undervalued dimension of software quality. Communications of the ACM, 46(8), 85-89. Retrieved Aug 2003, from

Russo, N. L. (2000). Expanding the horizons of information systems development. In R. L. Baskerville & J. Stage & J. I. DeGross (Eds.), Organizational and social perspectives on information technology (pp. 103-112). Boston: Kluwer.

Ryan, G. W., & Bernard, H. R. (2000). Data management and analysis methods. In N. K. Denzin & Y. A. Lincoln (Eds.), Handbook of Qualitative Research (2nd ed., pp. 769-802). Thousand Oaks, CA: Sage.

Ryle, G. (1949). The concept of mind. London: Hutchinson.

Sabherwal, R. (1999). The role of trust in outsourced IS development projects. Communications of the ACM, 42(2), 80-86. Retrieved February 1999, from

213

Schein, E. H. (1992). Organizational culture and leadership (2nd ed.). San Francisco: Jossey-Bass.

Schmidt, R., Lyytinen, K., Keil, M., & Cule, P. (2001). Identifying software project risks: An international Delphi study. Journal of Management Information Systems, 17(4), 5-36.

Schön, D. A. (1983). The reflective practitioner: How professionals think in action. New York: Basic Books.

Scribner, S. (1986). Thinking in action: Some characteristics of practical thought. In R. J. Sternberg & R. K. Wagner (Eds.), Practical intelligence: Nature and origins of competence in the everyday world. Cambridge, U.K: Cambridge University Press.

Simon, H. A. (1979). Rational decision making in business organizations. American Economic Review, 69, 493-513.

Spender, J.-C. (1996). Organizational knowledge, learning and memory: Three concepts in search of a theory. Journal of Organizational Change Management, 9(1), 63-78.

Stanley, W. B., Mathews, R. C., Buss, R. R., & Kotler-Cope, S. (1989). Insight without awareness: On the interaction of verbalisation, instruction, and practice in a simulated process control task. Quarterly Journal of Experimental Psychology, 41, 553-577.

Staples, D. S., Wong, I., & Seddon, P. B. (2002). Having expectations of information systems benefits that match received benefits: Does it really matter? Information & Management, 40(2), 115-131. Retrieved Dec 2002, from

Sternberg, R. J., Forsythe, G. B., Hedlund, J., Horvath, J. A., Wagner, R. K., Williams, W. M., Snook, S. A., & Grigorenko, E. L. (2000). Practical Intelligence in Everyday Life. Cambridge, UK: Cambridge University Press.

Sternberg, R. J., & Horvath, J. A. (Eds.). (1999). Tacit knowledge in professional practice: Researcher and practitioner perspectives. Mahwah, NJ: Lawrence Erlbaum.

Sternberg, R. J., & Wagner, R. K. (Eds.). (1986). Practical intelligence: Nature and origins of competence in the everyday world. NY: Cambridge University Press.

Sumner, M. (2000). Risk factors in enterprise-wide/ERP projects. Journal of Information Technology, 15, 317-327.

Sutcliffe, K. M., & McNamara, G. (2001). Controlling decision-making practice in organizations. Organization Science, 12(4), 484-501.

Swap, W., Leonard, D., Shields, M., & Abrams, L. (2001). Using mentoring and storytelling to transfer knowledge in the workplace. Journal of Management Information Systems, 18(1), 95-114.

Szymanski, D. M., & Henard, D. H. (2001). Customer satisfaction: A meta-analysis of the empirical evidence. Journal of the Academy of Marketing Science, 29(1), 16-35.

214

Takeuchi, H. (2001). Towards a universal management of the concept of knowledge. In I. Nonaka & D. J. Treece (Eds.), Managing industrial knowledge: Creation, transfer and utilization. London: Sage.

Trauth, E. M., & Jessup, L. M. (2000). Understanding computer-mediated discussions: Positivist and interpretive analyses of group support system use. MIS Quarterly, 24(1), 43-79.

Vicente, K. J. (1999). Cognitive work analysis: Toward safe, productive, and healthy computer-based work. Mahwah, NJ: Lawrence Erlbaum Associates.

Wagner, R. K. (1987). Tacit knowledge in everyday intelligent behavior. Journal of Personality and Social Psychology, 52(6), 1236-1247.

Wagner, R. K., Sujan, H., Sujan, M., Rashotte, C. A., & Sternberg, R. J. (1999). Tacit Knowledge in Sales. In R. J. Sternberg & J. A. Horvath (Eds.), Tacit Knowledge in Professional Practice: Researcher and Practitioner Perspectives. Mahwah, NJ: Lawrence Erlbaum Associates.

Walsham, G. (1995). Interpretive case studies in IS research: nature and method. European Journal of Information Systems, 4, 74-81.

Walsham, G. (1995a). Interpretive case studies in IS research: nature and method. European Journal of Information Systems, 4, 74-81.

Walsham, G. (1995b). The emergence of interpretivism in IS research. Information Systems Research, 6(4), 376-394.

Walsham, G., & Sahay, S. (1999). GIS for district-level administration in India: Problems and opportunities. MIS Quarterly, 23(1), 39-66.

Willcocks, L., & Margetts, H. (1994). Risk assessment and information systems. European Journal of Information Systems, 3(2), 127-138.

Wolcott, H. F. (1994). Transforming qualitative data: Description, analysis, and interpretation. Thousand Oaks: CA: Sage.

Yeates, D., & Cadle, J. (1996). Project management for information systems (2nd ed.). London: Financial Times Pitman Publishing.

Zsambok, C. E. (1997). Naturalistic decision making: Where are we now? In C. E. Zsambok & G. A. Klein (Eds.), Naturalistic decision making. Mahwah, NJ: Lawrence Erlbaum.

215

9APPENDIX A: INITIAL INTERVIEW PROTOCOL

Risk Management Interview guide

Firm:

Person: Position: Date:

Email:

Introduction:

The aim of the research is to understand more about actual risk management practices for

software package implementation projects, and in particular to learn about the practical

problems faced by managers of these projects and the judgments they exercise during the

course of the project. I’m particularly interested in the practical skills and judgment that

managers acquire and apply with experience, rather than the ‘official’ view we tend to teach in

formal classes.

Confidentiality:

Give an assurance of confidentiality – firm and person will not be identified in any way in any

publications resulting from this research. Seek permission to tape-record the interview.

Questions:

Please focus on a recently completed project that you’ve been involved with, preferably one

that involved some challenging situations.

Could you give me a brief overview of the project and how it progressed? (General

description to get overall context/big picture.)

Project overview - key points:

− Brief description of project

− Package type, modules etc

− Type of client – size, type of business, number of employees, public/private, IT

capability (staff numbers, experience, level of existing IT applications)

− Project details

− Budget $

− Time schedule

216

− Project team – number, who involved, skills

− Team experience

− Project leadership structure

− Project sponsor within client?

− Did you view the whole project process as a success – why or why not?

Overview of risk management processes:

Please describe any formal risk management processes that you did at the start of the project.

(e.g. identification of potential risks/problems; assessment of possible impact; evaluation and

prioritization; risk reduction actions).

(If formal processes were carried out:) Did these processes help in terms of problems that

actually came up during the course of the project and how these were handled – were there

any difficult situations that arose, had you anticipated these, were you able to handle them

according to the processes you had already been through? (Examples of possible problems:

mismatches between package and client; relationship with client firm; your firm’s lack of

domain knowledge of the client business).

(If no formal processes were used:) Had you anticipated in any way any difficult situations

that arose? How did you handle them?

Elicitation of specific incidents:

Please focus on any specific incidents that occurred during the course of the project that

were particularly challenging – especially situations that you think might have been handled

differently by a novice manager. (Use the following table to guide questioning at this stage.

217

Table A.1: Critical decision probe questions (adapted from Klein et al., 1989)

Area Questions

Problem description

Describe the situation?

What happened leading up to the situation? (Context, environment)

What did you do?

What was the outcome?

Planning Had you anticipated the possibility of this problem at the planning stage?

Did your plans include contingency measures for a problem like this?

Did the contingency measures work?

Did you deviate from the plan – how, what factors caused the deviation?

Cues What key points alerted you to …? How did you know that …?

Options What alternatives did you consider …?

What limitations did you face regarding possible actions?

Interactions Did you have direct control?

Who were the key players?

Analogues Were you reminded of any previous experience …?

Goals What were your specific goals at this point?

Basis How did you decide on your choice of action/reject other options?

Knowledge What information did you use for deciding …? What training or experience was useful in making this decision? How did you learn about …?

Hypotheticals With hind-sight, what would you have done differently …? What training or experience would have helped? What do you think a novice might have done in this situation?

Exceptions Can you think of another situation where you would have done things differently?

Results of actions

Did your action work as expected? If not, why do you think that was? If so, what might have caused it not to work? What would have happened if your action hadn’t worked? What would you have done?

Repeat until no new situations are forthcoming.

Repeat for another project.

218

Exploration of typicality:

How typical are these incidents of projects in your experience – were there particular aspects

of these projects that made them more risky/problematic?

Demographics:

− Your age group (20-29,30-39, 40-49, 50-59)

− Gender

− Ethnicity

− Education

− Experience (years, number of projects)

− Scale of projects managed (duration, effort (man-months), team size, budget, degree of

customization)

219

10APPENDIX B: ETHICS CONSENT PACKAGE

Consent form

Risk Management and Tacit Knowledge in IT Projects

Researcher Hazel Taylor

Visiting Scholar Dept of Information & Systems Management

University of Science and Technology Clear Water Bay

Hong Kong ph: (852) 2358 7635

mobile: (852) 9043 0951 email: htaylor@ust.hk

Statement of consent

By signing below, you are indicating that you:

• have read and understood the information sheet attached about this project;

• have had any questions answered to your satisfaction;

• understand that if you have any additional questions you can contact the researcher;

• understand that you are free to withdraw at any time, without comment or penalty;

• understand that you can contact the Secretary of the Hong Kong University of Science &

Technology Committee of Research Practices if you have concerns about the ethical conduct of the project; and

• agree to participate in the project.

Name

Signature

Date / /

220

Information sheet

Risk Management and Tacit Knowledge in IT Projects

Researcher Hazel Taylor

Visiting Scholar Dept of Information & Systems Management

University of Science and Technology Clear Water Bay

Hong Kong ph: (852) 2358 7635

mobile: (852) 9043 0951 email: htaylor@ust.hk

I am a Visiting Scholar at the Hong Kong University of Science and Technology, teaching in the Information and Systems Management Department. This research project is being conducted as

part of my PhD studies at Queensland University of Technology, Brisbane, Australia.

Description

The aim of the project is to try and identify the subtle differences between more effective/experienced and less effective/experienced IT consultants in terms of the project and risk

management aspects of their work. The research involves interviews with experienced project managers or consultants. Each interview typically takes between 45 minutes and an hour, and is

tape-recorded, and later transcribed. Once the transcript has been completed, a copy is given to the interviewee for verification, and the tape recording is then erased.

In the interview we focus on a recent project that you have worked on that was challenging from a

project and risk management perspective. I ask for a brief description of the project so that I have an idea of the scope and type of project (I don't need any details that would identify clients). Then I ask for an overview of the risk management processes that the manager used in this project, again to get an idea of the general progress of the project. Next I ask you to focus on specific incidents or

problems that arose in the project and we discuss various aspects of those problems and how they were solved.

Expected benefits

Results of this type of study have the potential to augment existing training programs for new

consultants - similar studies in the US have been applied to examine business management skills and military leadership skills, for example, and results have been used to extend existing training programs. As such, the research has potential to help identify areas that may assist in training of

new consultants.

Confidentiality

All interviews are confidential, and I do not identify the firms, projects, clients, or project managers in any reports resulting from this project. Only the research team will have access to the information

you provide. Your anonymity and confidentiality will be safeguarded in any publication of the results of this research, through the use of pseudonyms.

Feedback

A summary of the findings within your firm will be prepared, but individual project managers will not be identified in this report. A copy will be provided to you and to your manager within the firm. Once

221

the project is completed, I will also provide you and your manager with a summary across firms, although the firms will not be identified in that summary.

Voluntary participation

Your participation in this research project is voluntary and you may withdraw at any time, or withdraw

your data at a later time.

Questions / further information

Please contact the researcher if you have any questions or require additional information about the project.

Concerns / complaints

If you have any concerns or complaints about the ethical conduct of the project, please contact the Secretary of the Committee of Research Practices, Office of the Vice-President for Research & Development, Hong Kong University of Science & Technology, Clearwater Bay, Kowloon, Hong

Kong, phone 2358-6166. Since this research project is being conducted as part of my PhD studies at Queensland University of Technology, any concerns or complaints reported to the HKUST

Committee of Research Practices will also be forwarded to the Secretary of the University Human Research Ethics Committee at Queensland University of Technology.

222

11APPENDIX C: PRACTITIONER REPORT

RISK MANAGEMENT & TACIT KNOWLEDGE IN IT

PROJECTS PRACTITIONER REPORT

223

EXECUTIVE SUMMARY

Over the past year, I have been interviewing highly experienced IT project managers in Hong Kong as part of a research study on risk management and tacit knowledge (i.e. practical knowledge gained through experience) in IT projects. The aim of the research was to review risk management practices in IT vendor firms and to try and identify key aspects of tacit knowledge used by experienced project managers in the management of risk in their projects. This report summarizes the findings of that research. Key points that may be of particular interest to practitioners include:

1. Risk factors: Risk checklists used by IT vendor firms cover all the key risks identified in standard project management literature (see, for example, the Project Management Institute’s Project Management Book of Knowledge, http://www.pmi.org or the PRINCE2 methodology, http://www.ogc.gov.uk/prince/). However, the experienced managers in this study categorized the standard risks into three sources: risks arising from the vendor, risks arising from the client, and risks arising from third parties. Understanding the source of the risk helped managers to determine which strategy to use address the risk. Typically, managers used control strategies to manage vendor-based risks, and negotiation strategies to address risks client-based or third party-based risks.

2. Gaps in risk management practices: Risk management practices at pre-sale (or project proposal) stage were very thorough. However, the execution of risk management plans that had been developed at pre-sales stage was less rigorous, particularly in terms of the management of contingencies added to schedule and budget at the pre-sales stage to address specific risks.

3. Most common anticipated problems: Schedule/budget shortfall was the most commonly anticipated risk, and was also the most common risk to emerge as an actual problem during a project even after mitigating actions had been taken to address it.

4. Most common unanticipated problems actually arising: Issues related to client trust and expectations were the most likely problems to arise unexpectedly during projects. In particular, management of client expectations was seen as a key factor in successful project implementation. At the start of projects, managers tended to downplay to the client the typical challenges that will arise in any IT project. However the result of giving this ‘rosy picture’ is that when such problems do inevitably occur, the client is surprised and disappointed with the whole project, even when the problems are successfully addressed and the promised functionality is delivered. Educating the client to have a realistic expectation of how the project will progress is a key factor in ensuring client satisfaction at the close of the project.

5. Strategies to contain risks: Managers generally used four broad groups of strategies to manage risk, focusing primarily on the application of a strong control strategy through close management of the Work Breakdown Structure, and using negotiation strategies to build and maintain client relationships. When there were obvious shortfalls in knowledge about, for example, specific technologies, a strategy of carrying out further research was applied. Underpinning these three strategies was the application of a continuous monitoring strategy to keep ‘a finger on the pulse’ of the project.

224

6. Tapping tacit knowledge: The collective experience of project managers within a firm is potentially a very valuable source of tacit knowledge. Setting up mentoring arrangements and encouraging experience-sharing between senior and junior project managers could be an effective approach to tapping into this store of tacit knowledge, in order to facilitate knowledge transfer about possible problems that managers may encounter in their projects, and in identifying strategies to address these problems.

THE PARTICIPANTS AND PROJECTS

Twenty five experienced project managers participated in the research from twelve different organizations within Hong Kong. The organizations ranged from very small ‘boutique’ software houses, mainly focused on local clients, to Hong Kong branches of large multinational vendor and consulting firms, and in-house IT departments in both commercial and government organizations. The project management experience of the respondents ranged from 3 to 30+ years, and 5 to 150+ projects. All of the respondents had trained or worked abroad for part of their careers, and all had experience working with team members from a wide variety of cultures. Seventeen of the respondents worked as project managers for vendor firms, while three worked as in-house project managers, another three held a project executive role and the final two were pre-sales consultants.

The respondents described experiences with a very wide variety of projects, ranging from small local projects with a team size of one to five people, budget of $US20,000, and duration of three months, to very large, complex, multi-national projects with a budget of millions of US dollars, team size of 70+ people and a duration of more than two years. In total, since some of these projects were drawn from their experiences earlier in their careers, the respondents discussed in depth 39 projects carried out by 25 different organizations.

Most (77%) of the clients in these projects were commercial organizations, with the remaining 23% being carried out for government and semi-government clients. Similarly, most (78%) of the projects were vendor-driven, with the remaining 22% being in-house projects. Half of the in-house projects were in-house custom development, but the other half were outsourced custom development. These in-house outsourced projects provided a contrasting perspective to the vendor-driven projects, since essentially, all vendor-driven projects are outsourced projects from the client’s perspective. While most (65%) of the projects fell into a broad category of package implementation project, only two were described as straightforward package implementations with little or no customization. More typically, the projects included extensive customization, front-end web development work, and/or major infrastructure upgrades. The remaining projects involved mainly custom development and some consulting work.

Finally, ten of the projects discussed by respondents were ‘troubled’ projects which the managers had been called in to ‘rescue’. These projects in particular revealed some key insights into what causes projects to go wrong, and the key steps needed to fix them.

225

FINDINGS

The analysis of the interviews focused on three areas. Firstly, I identified the key risks that respondents had found were most likely to cause problems. Secondly, I examined the risk management practices typically prescribed by the respondents’ firms. Thirdly, I explored the strategies actually used by managers to address both risks (potential problems) and actual problems that arose in the course of their projects. Underpinning these three areas was the goal of identifying tacit knowledge used by managers in the project situations they faced.

KEY RISK FACTORS

I found that project managers from vendor firms considered risks arising from three major perspectives, namely risks situated with the vendor firm itself, risks associated with the client, and risks associated with any third parties to the project. Most of the commonly used risk factor checklists appear to focus on in-house development, and hence this distinction into three perspectives has not been made previously. However, for project managers working on vendor-driven projects it is very important to be clear about where potential risks arise from, since this may influence the strategy used to address the risk. For example, if a risk is identified with understanding requirements for the project it is important to be clear about whether this risk is associated with the understanding of the vendor project team, or the understanding of the client team, since different strategies are required to address these different situations. The top 11 risks are shown in Table 1 below.

Table 1: Top 11 risks mentioned by respondents

Risk Factor Rank (no. respondents)

Schedule & budget management 1(22) Client expectations 2 (18) Change management 2 (18) Client understanding of requirements 4 (17) Vendor staffing 4 (17) Client trust 6 (14) Vendor understanding of requirements 6 (14) Vendor competition 6 (14) Vendor team morale 9 (13) Newness of technology 9 (13) Client multiple sites/countries 9 (13)

226

As well as identifying the key risks noted by managers, I also compared potential problems anticipated at the start of projects with problems that arose during those projects, in order to identify, firstly, those problems that most typically still occur in projects even if they have been anticipated at the start, and, secondly, which of the problems that arise are most often unanticipated at the start. I examined the ten ‘troubled’ projects to identify the problems that their managers considered were responsible for the projects becoming ‘troubled’.

Problems most often occurring even after mitigating action

As shown in Table 2, the most likely problem to arise, even after being anticipated and addressed with mitigating actions, was schedule and budget management, often compounded by the second and third most common problems of vendor staffing issues and difficulties arising from the newness of technology. All three of these problems were seen by respondents as having their underlying cause in inadequacies in the pre-sales analysis, resulting in either an overall shortfall in budget or time allowance, or in an underestimation of the skill-set required or of the extra work involved in coming to grips with new technology. On the surface, it might not seem so surprising that managers would try to shift the responsibility for problems in their projects away from themselves. However, there was a strong consensus among respondents that the issue of over-selling or over-promising was almost endemic in the industry, with the pressures to win business, and the desire to satisfy customers often outweighing more cautious responses to client requests for proposals. Indeed, respondents were honest about their own tendencies to present the ‘best picture’ when they themselves had been involved in pre-sales work.

Table 2: Problems most likely to happen even after being anticipated

Risk factor No. of projects in which problem was anticipated

(out of total 39)

No. of projects in which problem still arose despite

mitigating action Schedule & budget management 24 8 Vendor staffing 16 5 Newness of technology 12 5 Multiple sites; multiple countries 11 1 Vendor understanding of requirements 9 0 Client expectations 8 3 Client organization culture 7 4

227

Most common unexpected problems

Given this tendency to ‘paint a rosy picture’ to clients at the start of projects, it is not surprising that the most common unanticipated problems tended to be client relationship problems. Client relationship problems appeared unlikely to be expected at the start, partly because signs that such problems might occur are typically not evident early in the vendor-client relationship, and also because their intangible nature makes them difficult to quantify and assess. However, the frequency with which client relationship problems did occur supports the need for strong measures to build good relationships at the start. Key unanticipated problems are shown in Table 3.

Table 3: Problems most likely to arise unexpectedly

Risk factor No. of projects in which problem occurred

unexpectedly (out of total 39)

Client trust 12 Client expectations 9 Vendor team morale 8 Change management 7 Client understanding of requirements 6 Client project management 5 Client ‘bad news’ 5

Troubled projects

The six problems most often highlighted in the ‘troubled’ projects were a combination of the most often anticipated problems and the most common unanticipated problems, with client expectations featuring highly in all three areas. The prominence of client expectations in all three problem sets is particularly significant. Customer satisfaction is related not only to what has been delivered and how well the process went, but also to what was expected in the first place. What this means is that if expectations are high to start with, and the outcomes are not quite as high as those expectations, then customers will ultimately be dissatisfied, even if final functionality and performance is a good match with requirements. Customer satisfaction is obviously extremely important for vendor firms since it has a major impact on their reputation and can significantly influence future business, particularly since new clients are likely to ask for reference sites when assessing vendor bids. Thus strategies to set realistic client expectations are a very important part of the project manager’s repertoire.

228

RISK MANAGEMENT PRACTICES

I found a very close match between practice at the pre-sales stage of IT vendor-driven projects and the most typical risk management prescriptions in the literature (see, for example, PMI Standards Committee, 1996), with firms using a rigorous process of identifying and ranking the risks for their projects. Most typically, the only risk mitigation action carried out at this stage was to add extra time or budget contingency when high risk areas were identified. However, there was a weak link in most firms’ risk management practices in the hand-over from pre-sales to implementation teams, with an apparent lack of follow-up on execution of the risk management plans prepared at the pre-sales stage.

Gaps between risk plans and execution of plans

The gap between the pre-sales risk assessment and execution of risk management plans was most apparent when the managers described situations where problems arose in their projects. In these situations, managers made no reference at all to drawing on the contingencies (particularly time and budget) that had been planned at the start of the project to address any risks identified then. This may be partly because many of these situations involved unanticipated client-related issues, which would not have been provided for in the original contingencies. However, even in the problem-arising situations involving problems that had been anticipated, managers still did not refer to using the planned contingencies.

It was not clear from the managers’ comments why they apparently did not draw on planned contingences, but there are two possible explanations. Firstly, managers may have been aware that a particular task was slipping, but may not have explicitly identified the situation as a problem until all of the planned contingency had been used. A second possibility, however, arises from the lack of explicit follow-up on the pre-sales risk assessment plans by the implementation managers. Project managers appeared to take the final pre-sales determination of budget and schedule as the expected targets, rather than distinguishing between the expected time and budget for a high-risk task, and any contingency allowance made for that task. If any contingency allowances are simply ‘absorbed’ into the overall expected time and budget for the task, the warning of task slippage will only come after the all planned contingency has already been used. This defeats the purpose of adding contingencies, since distinguishing between expected time/budget and additional contingency ensures that the manager is alerted to a potentially problematic task early, when there is still time to take further corrective action if necessary.

STRATEGIES TO ADDRESS RISKS

As noted above, the implementation managers did not appear to actively review the pre-sales risk assessments and nor did they attempt to plan specific responses to each of the many possible risks that might be identified for a project. Instead, they relied on their own management skills to control the project’s progress and on constant situation evaluation and assessment to give them sufficient warning to react when problems actually occurred. Some managers attributed the lack of specific risk response planning to the need for firms to survive in a fiercely competitive market, with customers unwilling to pay extra for a project plan that included a risk management phase. Others argued that risk management was essentially a pre-sales activity, which if done correctly would provide the correct budget and schedule to enable the project to run smoothly.

229

Broad general strategies

However, there also appeared to be reluctance on the part of the managers themselves to attempt to make elaborate plans to address, particularly, the intangible risks that were difficult to articulate and specify, and also difficult to quantify. For example, while managers were clearly aware that a project involving an overseas third party was a higher risk, whether that risk evolved into an actual problem depended on many uncertainties and unknowns, such as the skills of the third party staff, their willingness and cooperativeness, other work on the third party’s agenda, unknown technical problems that would only surface at the stage of testing and integration of products, and logistical issues of import and delivery. All of these uncertainties could fall together and produce a major impact on the project’s progress, or none of them could happen at all. Where a specific technical problem amenable to further research could be isolated, then this would be immediately addressed. Otherwise, respondents described general Control, Negotiation, and Monitoring strategies that would enable them to respond quickly to any problems if they arose, but that did not involve any extra work or planning at the outset that would be wasted effort if everything ran smoothly.

Control strategies were typically applied in those situations that the project managers saw to be directly within their locus of control, such as vendor staffing resources, change management, schedule and budget, documentation and client sign-off control. Control of these risks was seen as being at the heart of a project manager’s skill-set and strong management of the schedule and budget was seen as the foundation for project success. Risks associated with multiple sites/countries were also partly addressed by control strategies, since close management of logistical issues was seen to be paramount in addressing this risk.

Negotiation strategies were used in situations seen by the respondents as only partly within their locus of control. These situations were typically the ‘soft’ situations involving inter-personal interactions and relationship issues, and success is dependent not only on the project manager’s skills but also on the willingness of other parties to work together towards a successful outcome. Change management was one risk area which was addressed by Control and Negotiation strategies because while managers clearly considered it important to exercise very close control over change requests, they also recognized the inevitability of requirements change and the need to negotiate with the client a satisfactory solution for requested changes.

Research strategies, like Control strategies, were applied in situations that the managers considered to be fully within their control, namely situations that required further information in order to determine what action to take. In particular, when technological risks such as newness or compatibility had been identified, and also when there was a lack of vendor understanding of the client requirements, managers quickly set a section of their team to work on researching further information about the problem. In some ways these situations were the ones that the managers felt most confident about, because if problems arose in these areas they could be resolved simply by getting more information. If the new information revealed that the problem was insoluble, then at least they had a definitive answer and could either look for a different solution, or escalate the problem upwards for resolution at executive level.

230

Finally, for a large number of risks, respondents appeared to use no mitigating strategies at all, and instead adopted a Monitoring strategy. Most of these risks could be termed ‘environmental risks’ since they were potential problems related to the particular context of a project, such as contract terms and conditions, client organizational culture, implementation at multiple sites or countries etc. These types of risks were not seen as high priority, but they could influence how managers approached a particular project situation, and they provided situational cues that could alert the managers to particular areas requiring close attention.

Examples of specific Control and Negotiation strategies are shown in Table 4 on the next page, together with the tacit knowledge that managers had developed through their experience in applying these strategies. Much of the tacit knowledge seems like common sense and is informally well known in the IT project management community. However, it has not been formally or explicitly detailed in the typical training literature, and thus novice project managers may not be aware of these details that guide experienced managers in their application of the strategies.

These key general strategies, Control, Negotiation, Research, and Monitoring, are a very practical and ad hoc response to risk management in a highly uncertain, complex, and changing environment under significant time and budget constraints. The Control strategies enable the manager to quickly get to grips with the complexities and potential problems inherent in the project, without necessarily specifically identifying individual risks. Where initial Control strategies reveal specific, easily identifiable technical or requirements problems, these can immediately be addressed with Research strategies. Control strategies also provide a first line of defence against any issues that do arise, providing early identification of problems and typically having general contingency built in to give time to resolve these problems. The Negotiation strategies provide managers with tools for pre-empting potential relationship problems with external and internal parties, by establishing positive and cordial working relationships. If unexpected relationship breakdowns occur, or if other problems such as functionality or technical difficulties arise, Negotiation strategies help managers to establish compromises with their clients or with their own management in order to maintain the forward progress of the project. Underpinning these three strategy groups is the Monitoring strategy, which involves constant situation awareness and assessment to ensure early warning of any problems on the horizon.

231

Table 4: Examples of tacit knowledge associated with risk management strategies

Strategy group

Strategy Tacit knowledge

Control Develop detailed WBS 1. Rule of thumb: break down WBS to tasks of one week duration for practical control.

2. Developing WBS to this level of detail will ensure manager has in-depth grasp of the requirements, proposed solution and technology, required skill-set, and potential problems without specifically evaluating these.

Control progress on WBS 1. Rule of thumb: probe for specific evidence on staff and sub-contractor claims about % completion of tasks.

2. Tight control of progress will provide best early warning of any problems actually arising.

Control the documentation 1. By retaining control of, for example, meeting minutes, managers can ensure wording is cast favourably towards the vendor as insurance in the event of later disputes.

Negotiation Apply change control but understand the need to maintain a balance in client relationships

1. Always view change control as a negotiation process rather than simple contract enforcement, even when applying change control clauses strictly.

2. Application of strict change control can be used to ease tight schedule and budget conditions.

3. Where a client is assessed as potentially difficult, early application of change control can be used to establish discipline and ‘train’ the client, but this approach should be used cautiously in order to ensure that goodwill is not undermined.

4. Indicating that a request falls within the change control scope, but then waiving the application of the clause can build goodwill, but such practices should always be carefully documented.

5. For in-house managers faced with departments that are accustomed to using internal politics to ensure that on-going change requests are met, using outsourcing and supporting change control enforced by the external provider can help to change the internal culture and improve understanding within internal departments of the need for careful requirements specification.

Trust & relationship building 1. Rule of thumb to mentally “stand in the customer’s shoes” to help determine how to approach any issues that arise.

2. Recognize when to give in rather than argue. This is often a useful tactic at the start of a project, but may also be necessary throughout a project with a particularly difficult client, especially if the only other option appears to be to exit the contract.

3. Find ways to enable client staff to look good in front of their own bosses.

Restoring confidence in ‘troubled’ projects

1. Identify two or three issues that can be solved quickly and solve them.

2. Be honest about unsolvable problems. Balance cost, schedule & scope 1. Use these three points to frame negotiation and understand

the trade-offs involved.

232

CONCLUSIONS

The findings from this research indicate a number of areas where IT vendor firms, and their project managers, could derive benefits in their approaches to risk management. Some of the variances between respondents’ practice and theoretical prescriptions have important implications for practice, both positive and negative.

On the negative side, while vendor firms have clearly embraced risk management prescriptions at the pre-sales stage, there was evidence that many respondents regarded these processes as little more than checklists to be ticked off as complete at the pre-sales stage, rather than the foundation for on-going actions to mitigate potential risks applying to a particular project. Vendor firms would be well advised to consider the hand-over from pre-sales to implementation teams, and particularly how contingencies are viewed and managed by the implementation manager, in order to ensure that the purpose and benefit of the contingency planning is not lost.

On the positive side, the study provides a better understanding of some practical aspects of risk management, in particular: the need to view risks from three different perspectives (vendor, client, third party); the relationship between the four general strategies (Control, Research, Negotiation, and Monitoring) and the groups of risks they address; and the key explicit and implicit cues that can provide warning of impending problems. All of these aspects can help project managers to gain greater insight into their own understanding and handling of risk in their projects.

Finally, since these suggestions all derive from the experience-sharing of the project managers themselves, these ideas were already available to organizations through the store of tacit knowledge held by their project managers. However, few firms in this study appeared to recognize the potential of this store of tacit knowledge, and it seems likely that there would be significant benefit to be obtained from the establishment of formal mechanisms such as mentoring to improve knowledge transfer to novice project managers.