P INFORMATION SYSTEMS PORTFOLIO ... - Dr. Prasanna Karhade · Prasanna Karhade Department of...

35
RESEARCH ARTICLE PATTERNS IN INFORMATION SYSTEMS PORTFOLIO PRIORITIZATION: EVIDENCE FROM DECISION TREE INDUCTION 1 Prasanna Karhade Department of Information Systems, Business Statistics and Operations Management, The Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, HONG KONG {[email protected]} Michael J. Shaw and Ramanath Subramanyam Department of Business Administration, University of Illinois at Urbana–Champaign, Champaign, IL 61820 U.S.A. {[email protected]} {[email protected]} Questions pertaining to the locus of information systems (IS) governance have been extensively examined in existing research. However, questions pertaining to the decision rationale applied for IS portfolio prioriti- zation (why are certain initiatives approved, and why are certain others rejected), noted to be a critical component of IS governance, need further investigation. We submit that the IS strategy of a firm is likely to explain the decision rationale it applies to IS portfolio prioritization and maintain that it is critical to ensure this decision rationale is in congruence with the firm’s IS strategy. By extending prior theoretical work on IS strategy types, we develop theoretical profiles of the decision rationale applied to IS portfolio prioritization using three attributes: communicability of decision rationale, consistency in applying decision rationale, and risk appropriateness of decision rationale. Since the decision rationale applied for IS portfolio prioritization is often tacit, unknown even to the decision makers themselves, we employ the decision tree induction methodology to discover this tacit decision rationale. We analyze over 150 IS portfolio prioritization decisions on a multimillion dollar IS portfolio of a multibusiness, Fortune 50 firm and our findings, which support our propositions, indicate that firms that adopt different IS strategies rely on systematically different profiles of decision rationale for IS portfolio prioritization. Implications for IS governance practices are developed. Keywords: IS strategy, IS portfolio prioritization, IT portfolio management, IS governance, IT governance, decision making, decision tree induction Introduction 1 For most Fortune 500 firms today, their investment in initia- tives that depend on information systems (IS) is growing in importance (Kohli 2007; Piccoli and Ives 2005). As most firms have hundreds of such initiatives running simulta- neously (Jeffery and Leliveld 2004), chief information officers (CIOs) need to ensure that the decision rationale used for IS portfolio prioritization is in congruence with their IS strategies. The impact of IS strategies on the decision rationale (DR) used for IS portfolio prioritization (ISPP) has been rather understudied. 2 Although prior research contrib 1 Rajiv Kohli was the accepting senior editors for this paper. Paul Pavlou served as the associate editor. The appendices for this paper are located in the “Online Supplements” section of the MIS Quarterly’s website (http://www.misq.org). 2 Business initiatives that depend on IS are proposed by managers within a firm and CIOs/decision makers responsible for IS governance decide upon these initiatives. Certain initiatives are approved whereas certain others are MIS Quarterly Vol. 39 No. 2, pp. 413-433/June 2015 413

Transcript of P INFORMATION SYSTEMS PORTFOLIO ... - Dr. Prasanna Karhade · Prasanna Karhade Department of...

Page 1: P INFORMATION SYSTEMS PORTFOLIO ... - Dr. Prasanna Karhade · Prasanna Karhade Department of Information Systems, Business Statistics and Operations Management, The Hong Kong University

RESEARCH ARTICLE

PATTERNS IN INFORMATION SYSTEMS PORTFOLIOPRIORITIZATION: EVIDENCE FROM DECISION

TREE INDUCTION1

Prasanna KarhadeDepartment of Information Systems, Business Statistics and Operations Management, The Hong Kong University of

Science and Technology, Clear Water Bay, Kowloon, HONG KONG {[email protected]}

Michael J. Shaw and Ramanath SubramanyamDepartment of Business Administration, University of Illinois at Urbana–Champaign,

Champaign, IL 61820 U.S.A. {[email protected]} {[email protected]}

Questions pertaining to the locus of information systems (IS) governance have been extensively examined inexisting research. However, questions pertaining to the decision rationale applied for IS portfolio prioriti-zation (why are certain initiatives approved, and why are certain others rejected), noted to be a criticalcomponent of IS governance, need further investigation. We submit that the IS strategy of a firm is likely toexplain the decision rationale it applies to IS portfolio prioritization and maintain that it is critical to ensurethis decision rationale is in congruence with the firm’s IS strategy. By extending prior theoretical work on ISstrategy types, we develop theoretical profiles of the decision rationale applied to IS portfolio prioritizationusing three attributes: communicability of decision rationale, consistency in applying decision rationale, andrisk appropriateness of decision rationale. Since the decision rationale applied for IS portfolio prioritizationis often tacit, unknown even to the decision makers themselves, we employ the decision tree inductionmethodology to discover this tacit decision rationale. We analyze over 150 IS portfolio prioritization decisionson a multimillion dollar IS portfolio of a multibusiness, Fortune 50 firm and our findings, which support ourpropositions, indicate that firms that adopt different IS strategies rely on systematically different profiles ofdecision rationale for IS portfolio prioritization. Implications for IS governance practices are developed.

Keywords: IS strategy, IS portfolio prioritization, IT portfolio management, IS governance, IT governance,decision making, decision tree induction

Introduction1

For most Fortune 500 firms today, their investment in initia-tives that depend on information systems (IS) is growing inimportance (Kohli 2007; Piccoli and Ives 2005). As most

firms have hundreds of such initiatives running simulta-neously (Jeffery and Leliveld 2004), chief informationofficers (CIOs) need to ensure that the decision rationale usedfor IS portfolio prioritization is in congruence with their ISstrategies. The impact of IS strategies on the decisionrationale (DR) used for IS portfolio prioritization (ISPP) hasbeen rather understudied.2 Although prior research contrib

1Rajiv Kohli was the accepting senior editors for this paper. Paul Pavlouserved as the associate editor.

The appendices for this paper are located in the “Online Supplements”section of the MIS Quarterly’s website (http://www.misq.org).

2 Business initiatives that depend on IS are proposed by managers within afirm and CIOs/decision makers responsible for IS governance decide uponthese initiatives. Certain initiatives are approved whereas certain others are

MIS Quarterly Vol. 39 No. 2, pp. 413-433/June 2015 413

Page 2: P INFORMATION SYSTEMS PORTFOLIO ... - Dr. Prasanna Karhade · Prasanna Karhade Department of Information Systems, Business Statistics and Operations Management, The Hong Kong University

Karhade et al./Patterns in IS Portfolio Prioritization

utes insights by examining approved IS investments, “it isalso important to understand which investments do not makeit through the decision process and why” (Xue et al. 2008, p.88, emphasis added).

One of the key goals of IS governance is to encourage desir-able behavior in the prioritization and use of IS (Weill andRoss 2004, p. 11). As decision rights are integral to IS gover-nance, it is important to ensuring effective ISPP decisionmaking. Understanding the DR used for ISPP is critical asincongruence between DR and a firm’s IS strategy is not onlylikely to be associated with allocation of investment in unsuit-able IS initiatives, but also associated with firms missing outon key IS-enabled strategic business opportunities. In spite ofextensive research delving into who, and at what stages of theprocess, makes IS governance decisions (Brown 1997;Sambamurthy and Zmud 1999; Weill and Ross 2004; Xue etal. 2008), there exists a need for research that focuses on whycertain initiatives are approved and why certain others arerejected.

To address this gap, we extend prior theoretical work on ISconservative–innovator Strategy types (Chen et al. 2010) todevelop theoretical profiles of the DR used for ISPP. Wedevelop theoretical profiles of the DR by relying on threeattributes that succinctly characterize decision making,namely (1) communicability of DR (Segars and Grover 1999),(2) consistency in applying DR (Chen et al. 2010), and finally(3) risk appropriateness3 of DR (Boynton and Zmud 1987). We propose that characteristics of IS conservative strategy—a stable external environment, formal decision-making struc-ture, and risk-averse tendencies for continual efficiencyimprovements—are likely to collectively exert a similar influ-ence on DR it uses for ISPP. Characteristics of IS innovatorstrategy include a dynamic external environment, organicdecision-making structure, and risk-taking tendencies for pur-suing new opportunities. As ignoring emerging opportunitiesin the external environment imposes prohibitively high oppor-tunity costs on IS innovators (Chen et al. 2010, p. 252), thedynamic external environment is likely to exert a strong influ-ence on the DR it uses for ISPP.

We analyze ISPP decisions pertaining to a multimillion dollarIS portfolio, gathered from two business units, that haveadopted two distinct IS strategies within a naturally controlledFortune 50 multibusiness empirical setting. We collecteddetailed data on all initiatives in the IS portfolio of both thesebusiness units, with each initiative characterized by benefit,risk-assessment, and risk-mitigation information attributes. At the end of the ISPP, we gathered data on the final (approveor reject) prioritization decision awarded to each initiative.

Our contributions to the literature are threefold. First, webroaden prior theoretical work on IS strategy types by devel-oping theoretical profiles for the DR used for ISPP incongruence with these IS strategies. This contingency viewof IS strategy considered in the context of ISPP is consistentwith the view that firms adopting different IS strategies fol-low correspondingly different DR for ISPP (Chen et al. 2010;Weill and Ross 2004, p. 159). We also articulate the impli-cations of deviations from these theoretical profiles for eachIS strategy.

Second, while much of the prior research has centered on thelocus of decision making for IS governance (e.g., Samba-murthy and Zmud 1999), we address a key question linking ISgovernance and ISPP: Why are certain initiatives approvedand simultaneously why are certain others rejected? Webroaden research on IS governance by maintaining that the ISstrategy adopted by firms is likely to explain the DR it usesfor ISPP.

Third, our research makes key methodological contributions. DR used for ISPP is often tacit, making it difficult to captureor share (Markus et al. 2002). This difficulty inevitablymeans that, even when this rationale can be made explicit, itcannot easily be represented numerically, but must instead berepresented in terms of if–then decision rules (Baligh et al.1996; Huber 1981). All information attributes4 are availableto the decision makers, but interconnections among theseinformation attributes that occur during decision making—leading to the formation of decision rules—often remain taciteven to the decision makers themselves. Decision tree induc-tion methodology (DTIM) enables us to codify these tacitinterconnections and link these emergent decision rules tofinal decision outcomes. Thus, we employ a DTIM (Quinlan1993), which enables us to open up the black box of decisionmaking (Langley et al. 1995) by uncovering tacit interconnec-tions among multiple decision attributes without imposing anyex ante biases on the manner in which these attributes areexpected to be associated with decisions. Decisions trees,

rejected. We refer to this activity as prioritization of IS portfolios andinvestigate the DR used for ISPP.

3With regard to risk taking during ISPP, two mechanisms pertain to the roleof managers. First, taking some risks is an essential part of a manager’s role. It is also likely that managers cannot just reject all high-risk initiatives. Thetrue business value of IS would be difficult to realize if managers behaved inthis manner. Secondly, to take risks intelligently, managers are required toexert effort to mitigate risks. If managers approve high-risk initiatives with-out exerting effort or gathering additional information, they are gamblingwith scarce firm resources. These two mechanisms collectively define riskappropriate behavior (Boynton and Zmud 1987; March 1994; March andShapira 1987).

4Information attributes are inputs to DTIM and decision attributes (a subsetof information attributes) are its outputs. Decision attributes are the mostpertinent information attributes for explaining decisions.

414 MIS Quarterly Vol. 39 No. 2/June 2015

Page 3: P INFORMATION SYSTEMS PORTFOLIO ... - Dr. Prasanna Karhade · Prasanna Karhade Department of Information Systems, Business Statistics and Operations Management, The Hong Kong University

Karhade et al./Patterns in IS Portfolio Prioritization

outcomes of DTIM, are collections of decision rules and arecredible approximations of the tacit DR5 (Huber 1981;Quinlan 1993) used for ISPP. While prior research on ISgovernance has exclusively focused on approved decisions,we submit that a juxtaposition of the DR used for approvinginitiatives with the DR used for rejecting other initiatives islikely to deepen our understanding of IS governance.

This paper is organized as follows. The following sectionintegrates existing literature. The subsequent section presentsthe theoretical logic guiding our propositions. The DTIM isthen described, followed by a discussion on the implicationsof our findings. Finally, we present our concluding remarks.

Theoretical Background

While assessing the business application needs of IS and ISPPhas been suggested to be a critical component of IS gover-nance (Weill and Ross 2004, p. 11), research that examinesthe DR firms use for ISPP is much needed (Xue et al. 2008). Our framework, which examines DR used for ISPP, is pre-sented in Figure 1 and our theory development effort issummarized in Table 1.

IS Strategy Types

Recent research that has developed an IS strategy typology by developing profiles of IS conservatives and IS innovators(Chen et al. 2010) serves as our theoretical foundation. ISconservatives adopt a strategy that enables them to reap thebenefits of IS by reducing operational inefficiencies. IS inno-vators adopt a strategy to apply IS in innovative ways toexplore new business opportunities. Looking ahead, althoughprofiles of IS strategy types have been developed, theoreticallinkages between IS strategy and corresponding profiles ofDR used for ISPP need further investigation.

Decision Rationale

Although prior research has identified information attributescrucial for ISPP, a school of thought maintains that decisionsoften emerge based on tacit interconnections among informa-tion attributes during decision making (Markus et al. 2002).

Interconnections among multiple information attributes con-verge into patterns as decisions emerge (Mintzberg 1994). Since decision making often involves tacit interconnectionsamong multiple information attributes, a suitable method-ology is needed for opening up the black box of decisionmaking (Langley et al. 1995). DTIM is suitable for dis-covering these tacit interconnections as it does not impose exante biases (Quinlan 1990, 1993) on the manner in whichinformation attributes influence decisions. DTIM identifiesthe most informative attributes for explaining decisions(namely, decision attributes) and excludes all noninformativeattributes from the decision tree. Decision trees, outcomes ofDTIM, are collections of decision rules generated based onthe informative attributes. We employ three heuristics (highprediction accuracy, parsimony, and reliability) to help usselect the most credible approximation (decision tree) of thetacit DR (Huber 1981).

IS Portfolio Prioritization and IS Governance

Although prior research has examined the locus of IS gover-nance decisions and the valuation of IS portfolios, theoreticalprofiles of DR used for ISPP have not yet been examined. Understanding the DR that firms apply for ISPP is critical asineffective prioritization is not only likely to be associatedwith investment in unsuitable IS initiatives, but also asso-ciated with firms missing out on IS-enabled strategic businessopportunities (Weill and Ross 2004).

We expect a strong link between ISPP and IS governance. One of the key goals of IS governance is to encourage desir-able behavior in the prioritization of IS portfolios. Decisionrights are central to IS governance and, thus, ensuring thatISPP decisions (approval and rejection decisions on IS initia-tives to pursue in the future) are made effectively is critical.

Prior research on IS governance has focused on the locus ofdecision making by examining the extent to which decisionmakers share responsibility (Sambamurthy and Zmud 1999). Recently, Xue et al. (2008) incorporated an additional dimen-sion of the timing of IS governance decisions by examiningwho the key decision makers are and when, at which stages inthe IS investment process, are they involved in making deci-sions? Although existing research (see Table 1) has studiedthe valuation of portfolios from a real-options perspective(Bardhan et al. 2004; Bardhan et al. 2010; Kauffman andSougstad 2008; Reyck et al. 2005), the influence of a firm’sIS strategy on the DR it uses for ISPP remains unaddressed.

We examine the link between IS strategy and IS governanceby submitting that a firm’s IS strategy is likely to influencethe DR it uses for ISPP. We develop theoretical profiles for

5True DR used for ISPP can often be unknown even to the decision makersthemselves. DTIM gives us DTs, which do not represent the actual scriptfollowed for prioritization but are approximations of the underlying DR. Weuse three heuristics (high predictive accuracy, parsimoniousness, androbustness) to guide us in selecting the best representative DT, whichrepresents a credible approximation of the underlying DR.

MIS Quarterly Vol. 39 No. 2/June 2015 415

Page 4: P INFORMATION SYSTEMS PORTFOLIO ... - Dr. Prasanna Karhade · Prasanna Karhade Department of Information Systems, Business Statistics and Operations Management, The Hong Kong University

Karhade et al./Patterns in IS Portfolio Prioritization

Figure 1. Research Framework

Table 1. Theoretical Development

TheoreticalBuilding Blocks

Role in Theory Development

Our Theoretical RationaleExisting Research Our Research

Decision Rationale

Boynton and Zmud 1987Chen et at. 2010 Earl 1993Quinlan 1990Segars and Grover 1999

Develops theoretical profiles ofDR based on communicability,consistency and riskappropriateness

DR is often tacit; DTIM givesus a credible approximationof this tacit DR

IS PortfolioPrioritization

Bardhan et al. 2004Bardhan et al. 2010Kauffman and Sougstad 2008Kumar et al. 2008Reyck et al. 2005

Examines the critical linkbetween ISPP and ISgovernance

Provides unique insights forIS Governance bydeveloping DR profiles

IS Strategy –Decision RationaleLink

No prior work in IS research. Extends work on IS strategiesby examining its impact onISPP DR

IS Strategy types enable usto develop DR profiles

the DR firms used for ISPP by relying on three key attributesthat have strong implications for IS governance. First, seniormanagement’s commitment, in terms of their time and atten-tion, is critical for effective prioritization (Rajegopal 2012,pp. 88-90). Thus, communicability of the DR has strongimplications for IS governance and effective resource utili-zation, especially the scarce resource of senior management’stime and attention (Segars and Grover 1999). Next, consis-tency in applying the DR is important to IS governance asproposals with similar characteristics warrant similar deci-

sions (e.g., Chen et al. 2010). Finally, for effective IS gover-nance, the DR used for ISPP must ensure that an appropriaterisk posture is embodied in the firm’s future IS portfolios(Boynton and Zmud 1987).

In summary, we broaden the literature on IS governance byaddressing a key question on ISPP: Why are certain initia-tives approved and why are certain others rejected? Next, wedevelop theoretical profiles of DR used for ISPP in con-gruence with a firm’s IS strategy.

416 MIS Quarterly Vol. 39 No. 2/June 2015

Page 5: P INFORMATION SYSTEMS PORTFOLIO ... - Dr. Prasanna Karhade · Prasanna Karhade Department of Information Systems, Business Statistics and Operations Management, The Hong Kong University

Karhade et al./Patterns in IS Portfolio Prioritization

Research Propositions

Decision Rationale Profiles

The IS strategy adopted by a firm is likely to exert a stronginfluence on the DR it uses for ISPP. We identify charac-teristics that determine the IS strategy of a firm (Chen et al.2010) and collectively influence its DR. We maintain thatDR applied for ISPP is likely to be effective, from an ISgovernance perspective, when it is easy to communicate, isapplied consistently, and is risk-appropriate with respect tothe IS strategy of the firm. We rely on three attributes todefine theoretical profiles of the DR: (1) communicability ofthe DR (Segars and Grover 1999), (2) consistency in applyingthe DR (Chen et al. 2010), and (3) risk appropriateness of theDR (Boynton and Zmud 1987).6 The three attributesdiscussed below have been noted to effectively characterizedecision processes (Boonstra 2003; Markus et al. 2002) andhave strong implications for IS governance.

Communicability: Complexity in decision making is likelyto be influenced by the IS strategy of a firm. As the com-plexity7 of DR increases, its communicability (the ease withwhich it can be articulated and shared with others) is likely tosuffer. Simple, communicable DR uses scarce resources(senior management’s time and attention) judiciously (Segarsand Grover 1999).

Consistency: Decision rules often emerge when informa-tional exchanges between decision makers converge intopatterns (Mintzberg 1994), facilitating tacit interconnectionsamong multiple information attributes. Since the tacit DRused for ISPP is typically difficult to capture (Markus et al.2002), this DR is best represented in the form of if–thendecision rules (Huber 1981). Each decision rule explicitlycodifies the tacit DR. Some decision rules are used with ahigher frequency and are consistently reused to make a largernumber of decisions. The frequency with which decisionrules are applied (the number of decisions made using thesame decision rule) represents the consistency with which thetacit DR is applied for ISPP.

Risk Appropriateness: Different IS strategies represent dif-ferent risk profiles. Firms that adopt different IS strategies arelikely to differ in their risk appetite (Boynton and Zmud 1987;March and Shapira 1987). Given differences in the risk appe-tites of firms, we propose that a correspondingly different DRis likely to be considered risk appropriate (March 1994), asfirms differing in risk appetites are likely to raise/answer dif-ferent kinds of questions before making ISPP decisions. Fora given IS strategy, the DR used for ISPP is risk appropriateif the right kinds of questions, in congruence with the chosenIS strategy, have been raised/answered before approving/rejecting initiatives. Theoretical profiles for the DR used forISPP are summarized8 in Table 2 and theoretical explanationsare presented next.

Decision Rationale for IS Conservatives

IS conservatives focus on improving the efficiency of theirinternal operations (Chen et al. 2010). IS conservative stra-tegy is associated with three characteristics: (1) a relativelystable external environment, (2) formal decision-makingstructure, and (3) risk-averse tendencies in the quest for effi-ciency improvements. We propose that these characteristicsare likely to collectively exert a similar influence on DR usedby IS conservatives for ISPP. Stability in the external envi-ronment coupled with a formal decision-making structuremakes it easy for risk-averse IS conservatives to apply DRconsistently in a top-down manner.

A stable external environment fosters simplicity in the DR;thus, complexity of the DR used by IS conservatives is likelyto be low. IS conservatives adopt a formalized decision-making structure (Jansen et al. 2006) to maintain stability(Chen et al. 2010) by applying a simple, highly consistentDR. Acting risk appropriately, decision makers with a low-risk appetite are likely to approve high-risk initiatives onlyafter ensuring that risk-mitigation mechanisms have beendesigned to control risks (Boynton and Zmud 1987; Strauband Welke 1998). We investigate the DR tacitly applied forISPP by relying on DTIM which yields DTs (Column 1 inTable 39). In summary, the easily communicable, highly con-

6The logic of appropriateness we adopt here is a rational model of choice(March 1994). There could be other logics of appropriateness, namely onesthat serve as mechanisms for appeasement or politicking. Studying theselogics of appropriateness is beyond the scope of this study. Although we donot specifically examine politically driven decision making, our methodologyenables us to identify instances of rationally unjustifiable decisions. Identi-fication of rationally unjustifiable decisions is an important step for investi-gating politically driven decision making. Thus, our methodology can guideother scholars who wish to investigate politically driven decision making.

7We acknowledge that complexity of decision making is a multifacetedconstruct, but in this study, we examine complexity of the DR exclusivelyfrom an information-theoretic perspective (Quinlan 1990).

8Some firms adopt dual IS strategies encompassing elements of both ISconservative and IS innovator strategy. We present a theoretical DR profilefor firms adopting such dual strategies in Appendix A.

9The number of attributes in a DT represent its complexity for two reasons. First, each decision attribute represents a question and information along thebranches represent answers to these questions. The number of questionsanswered during decision making increases as the number of nodes in the DTincreases. Second, as number of nodes included in the DT increases, com-plexity in determining interconnections among these nodes increases. Forrobustness, we use length of longest path in DT (maximum number of ques-tions answered before making a decision) and total number of leaves in DT(total number of decision rules in the DT) as alternative measures of thecomplexity of DR.

MIS Quarterly Vol. 39 No. 2/June 2015 417

Page 6: P INFORMATION SYSTEMS PORTFOLIO ... - Dr. Prasanna Karhade · Prasanna Karhade Department of Information Systems, Business Statistics and Operations Management, The Hong Kong University

Karhade et al./Patterns in IS Portfolio Prioritization

= Benefit-related information attribute

= Risk Assessment/Mitigation-related information attribute

= Thicker lines represent decision rules applied toconsistently make more decisions

Table 2. Theoretical Profiles for IS Portfolio Prioritization Decision Rationale

IS Conservative Strategy IS Innovator Strategy

Communicability of Decision Rationale High Communicability Low–Moderate Communicability

Consistency in applying Decision Rationale High Consistency Low–Moderate Consistency

Risk Appropriateness of Decision Rationale Focus on Risk Assessment/Mitigation Focus on Exploring Opportunities

Table 3. Illustrative Decision Trees

Column 1: IS Conservatives Column 2: IS Innovators

Illustrative Decision Tree (DT)

DTConservative DTInnovator

Legend

Decision Rationale Profile Attributes

Communicability: Inversely related to complexity of the DT: One key measure of complexity of a DT is the number ofdecision attributes included in the DT

Complexity of DTConservative = 3 Complexity of DTInnovator = 6

Consistency: Number of decisions made consistently using the same decision rule: Thicker lines represent decision rulesconsistently applied to make more decisions using the same rule

Risk Appropriateness: Kinds of questions raised/answered: Mix of decision attributes

2 of 3 attributes are risk assessment/mitigationattributes

4 of 6 attributes are benefit-related attributes

sistent DR used by IS conservatives for ISPP is likely to focuson the assessment/mitigation of risks.

Proposition 1: The decision rationale used by ISconservatives for IS portfolio prioritization is likelyto be easy to communicate, applied with high consis-tency, and focused on risk assessment/mitigation.

Decision Rationale for IS Innovators

IS innovators monitor an eclectic array of new opportunitiesin their dynamic external environment (Chen et al. 2010). ISinnovator strategy is associated with three characteristics:

(1) a dynamic external environment, (2) an organic decision-making structure, and (3) risk-taking tendencies. IS innova-tors are expected to enter new markets; therefore, they mustquickly adapt to changes in their dynamic external environ-ment since ignoring these changes imposes prohibitively highopportunity costs (Chen et al. 2010, p. 252). Thus, dynamismin the external environment is likely to exert a strong influ-ence on the DR used by IS innovators for ISPP.

IS innovators are expected to continually analyze dynamicinformation flows in their external environment, which islikely to necessitate a complex DR. To exploit new oppor-tunities, IS innovators are compelled to experiment, and areencouraged to act before engaging in extensive debate and

418 MIS Quarterly Vol. 39 No. 2/June 2015

Page 7: P INFORMATION SYSTEMS PORTFOLIO ... - Dr. Prasanna Karhade · Prasanna Karhade Department of Information Systems, Business Statistics and Operations Management, The Hong Kong University

Karhade et al./Patterns in IS Portfolio Prioritization

dialogue. Formalized/rigid rules stifle experimentation (Jan-sen et al. 2006), which is important to IS innovators. To takerisks intelligently, IS innovators often develop tacit rules,representing well-designed experiments (March and Shapira1987), to increase the likelihood of success in exploring newopportunities. However, these tacit rules are likely to beapplied less consistently owing to the high dynamism in theexternal environment.

IS innovators thrive on change in their environment and oftencreate this change by intelligently taking risks. They explorenew opportunities by focusing on the potential business valueassociated with their proposed initiatives, notwithstanding theriskiness of these initiatives. Thus, IS innovators are likely tode-emphasize risk mitigation during ISPP. In summary, DRused by IS innovators is not likely to be easy to communicate,is not likely to be applied with a high consistency, and islikely to be focused on exploring opportunities (Column 2 inTable 3).

Proposition 2: The decision rationale used by ISinnovators for IS portfolio prioritization is likely tobe focused on exploring opportunities, not easy tocommunicate, and not applied with high consistency.

An Empirical Analysis of ISPortfolio Prioritization

We choose a multibusiness firm as our research setting (seeour overall data collectoin strategy in Table 4). Businessunits within our empirical setting pursued different IS stra-tegies along the IS conservative–innovator continuum10 incongruence with their differing business strategies. IS port-folios of business units at the ends of the IS conservative–innovator continuum are selected for further investigation.11

Front-line12 managers characterized their proposals, businessinitiatives they submitted for funding, with a rich set of infor-mation attributes (see in Table 5). DTIM was applied across

data gathered from both business units to discover the DRtacitly applied for ISPP by CIOs and other key decisionmakers. Since DTIM discovers approximations of the tacitlyapplied DR, we rely on the best representative DT, a credibleapproximation of the tacit DR, for each portfolio to examinesupport for our propositions. The steps in DTIM are outlinednext.

Ascertaining the IS Strategy

To characterize differences in IS strategies adopted by busi-ness units at our research site, we adopt recent theoreticalresearch on IS strategy types (Chen et al. 2010). Our researchsite affords us a naturally controlled empirical setting thatenables us to closely examine differences in DR tacitly ap-plied for ISPP across business units, within one Fortune 50firm, that are pursuing different IS strategies. Data aregathered via various mechanisms to ascertain the IS strategiesadopted by these business units.13

For effective triangulation (Miles and Huberman 1984), dataare collected by the following methods: face-to-face, semi-structured, open-ended interviews with key informants (mem-bers of the top management team, vice president (VP) andCIO, and multiple senior business executives) spanning morethan 20 hours; confidential documents obtained from keyinformants; observation of ISPP sessions; content analysis ofannual reports. The IS strategy of one business unit wasclassified as an IS conservative whereas the IS strategy ofanother business unit was classified as an IS Innovator.14

Input Portfolio Data

Front-line managers can be a source of new ideas and firmscan miss out on numerous IS-enabled business opportunitiesif they do not listen to their front-line managers (Kohli 2007). On an annual basis, front-line managers at our research siteare encouraged to identify new IS-enabled business applica-tion areas to pursue in the future. These funding proposals arecollected across the two business units and each proposal isdescribed using multiple information attributes. We refer tothese data as the input portfolio data. We distinguish betweeninformation attributes and decision attributes as follows:Information attributes are inputs to DTIM, and decision attri-butes, a subset of information attributes, are outputs identified

10IT infrastructure portfolios are excluded from our analysis as they are oftengoverned as an enterprise-wide, shared capability. This sampling criterionis further explained in the first section of Appendix C.

11Given our unique empirical setting, we also find support for our claimsregarding the DR profile for organizations adopting dual IS strategies. Givenspace limitations, we present those claims in Appendix A.

12Front-line managers are responsible for managing customers, manufac-turing operations, supplier relationships, factory inventory, etc. They areoften in positions to design IS initiatives to pursue in the future. We refer tosuch managers as front-line managers to distinguish them from othermanagers (CIO and other business decision makers).

13No business units at our research site were classified as adopting anundefined IS strategy (Chen et al. 2010).

14Appendix B describes the process used to identify IS strategies adopted bybusiness units.

MIS Quarterly Vol. 39 No. 2/June 2015 419

Page 8: P INFORMATION SYSTEMS PORTFOLIO ... - Dr. Prasanna Karhade · Prasanna Karhade Department of Information Systems, Business Statistics and Operations Management, The Hong Kong University

Karhade et al./Patterns in IS Portfolio Prioritization

Table 4. Data Collection Strategy

Constructs Source of Data Data Collection Methods Validation of Data

IS Strategy: ISconservative and ISinnovator

VP, CIO, and one seniorbusiness executive each fromthe IS conservative andInnovator business unit

Open-ended, semi-structured interviews,research collaboration meetings, andimmersion sessions with VP and CIO

VP and CIO validated ourclassification; data from annualreports provided additionalvalidation

Input Portfolio Data: Benefits, risk,mitigation attributes

Front-line managers Proposals collected in collaboration withthe members of metrics group within thefirm

VP, CIO, and senior executivesvalidated the portfolio data,comprehensiveness of information attributes

ISPP Decisions: Initiatives rejected orapproved

VP, CIO, and one seniorbusiness executive each fromthe IS conservative andinnovator business unit

Collected from the VP, CIO, and seniorbusiness executives at the end ofunobtrusive observation sessions ofISPP meetings

Members of metrics group withinthe firm validated that the ISPPdecisions were recorded correctly

Table 5. Information Attributes for Characterizing Initiatives

Information Attribute Definition Key References

Benefit information attributes

1 Efficiency improvements IS that automated manual tasks and business activities Aral and Weill 2007;Broadbent et al.1999;Sabherwal and Chan 2001;Philip 2007

2 Inter-organizational processimprovements

IS aimed at improving interorganizational business processes

3 Cycle time reductions IS aimed at reducing product/service delivery cycle times

4 Marketing benefits IS that create and promote new products/services

5 Strategic benefits IS aligned to a firm’s strategic goals

Risk-assessment information attributes

6 Initiative size Firm Thresholds: Low: Size < $100,000 USDMedium: $100,000 < Size < $1,000,000 USDHigh: $1,000,000 USD < Size

Iversen et al. 2004; Lyytinenet al. 1995; McFarlan 1981;Nolan and McFarlan 2005

7 Initiative structure Low: Lack of clearly defined objectivesHigh: Well-defined objectives

8 Prior experience Low: Technologies not familiar to firmMedium: Technologies moderately familiar High: Standard technologies familiar to firm

Risk-mitigation information attributes

Internal Risk Mitigation Mechanisms

9 Employ in-house software Software application developed internally by the firm Earl 1993; Mitchell and Zmud2006

10 Internal maturity Low: Idea in early stages of developmentMedium: Goals/requirements are clearly definedHigh: Controls envisioned/put in place across life cycle stages ofthis mature initiative

Boonstra 2003; Ramasubbuet al. 2008

Process Risk Mitigation Mechanisms

11 Business process redesigncompleted

Process redesign completed and controls indicated Broadbent et al. 1999

12 Resources for processredesign committed

Process redesign resources identified and assigned Lambert 1986

External Risk Mitigation Mechanisms

13 Employ consultantknowledge

Initiative involved leveraging capabilities of integration partnersand/or external consultants

Ko et al. 2005; Susarla et al.2010

14 Utilize specialized softwareapplications

Initiative involved procurement of specialized software applications McFarlan 1981; Nolan andMcFarlan 2005

15 Leverage third-partysolutions

Initiative supported by third party software building blocks

420 MIS Quarterly Vol. 39 No. 2/June 2015

Page 9: P INFORMATION SYSTEMS PORTFOLIO ... - Dr. Prasanna Karhade · Prasanna Karhade Department of Information Systems, Business Statistics and Operations Management, The Hong Kong University

Karhade et al./Patterns in IS Portfolio Prioritization

by DTIM. Decision attributes are the most pertinent informa-tion attributes for explaining decisions, as identified byDTIM. Data on all information attributes are provided byfront-line managers (see Table 515). Input portfolio data werecollected by collaborating with the metrics group within thefirm. Since these initiatives deal with extensive financialcommitments, these data are strictly audited. By focusing ona controlled sample16 of business applications of IS, we attri-bute differences in DR used for ISPP to differences in ISstrategies of these two business units.

Input Portfolio Data

Three types of information attributes (Boynton and Zmud1987; McFarlan 1981; Sabherwal and Chan 2001) used torichly describe business initiatives are analyzed in this study. First, the CIO and other key business leaders seek a richdescription of initiatives in terms of potential benefits they canoffer. Such a description helps these decision makers togauge the business value associated with these initiatives andenables them to identify promising initiatives to pursue (Araland Weill 2007). Second, since ISPP involves the selectionof initiatives to pursue in the future, ISPP requires managingrisks. Thus, assessment of risks associated with proposedinitiatives in the portfolio is an integral component of ISPP. Third, risk mitigation is a critical component of IS governance(Nolan and McFarlan 2005) and thus decision makers at ourresearch site demanded data on risk-mitigation mechanismsdesigned by front-line managers to control risks associatedwith their proposals (March and Shapira 1987). Managerscan either exert additional effort or gather additional infor-mation to control risks associated with their proposals(Lambert 1986). Thus, these three types of attributes arepertinent for ISPP and essential for effective IS governance(Maizlish and Handler 2005). Descriptions of the 15 informa-tion attributes used by front-line managers to characterizetheir proposed initiatives are presented next.

Benefit information attributes: Based on the information onproposed initiatives prepared by front-line managers, wecreate a total of five variables to capture the different types ofbenefits, these business applications of IS can offer. Insights

from prior research (Aral and Weill 2007; Broadbent et al.1999; Sabherwal and Chan 2001) guided these transforma-tions. For all benefit-related variables, the inter-raterreliability was over 95 percent.17 Our characterizations ofbenefits are validated for us not only by the front-linemanagers proposing the initiatives, but also decision makersresponsible for ISPP.

Risk-assessment information attributes: Risk assessment iscritical for ISPP (Nolan and McFarlan 2005). Based onsuggestions from prior work (Lyytinen et al. 1995), measuresfrom McFarlan (1981) are used to assess risks associated withinitiatives in our data.18

Risk-mitigation information attributes: Risk-mitigationmechanisms are critical for the successful implementation ofIS-enabled business initiatives (Broadbent et al. 1999; Piccoliand Ives 2005; Ramasubbu et al. 2008; Straub and Welke1998). We rely on three categories of risk-mitigation mech-anisms (Iversen et al. 2004; Nolan and McFarlan 2005; Shererand Alter 2004), namely internal, external, and process risk-mitigation mechanisms. The comprehensiveness and effec-tiveness of these risk-assessment and risk-mitigation mech-anisms is validated for us by the front-line managers and thekey decision makers at our research site.

IS Portfolio Prioritization Decisions

For each proposed initiative, ISPP decisions are made by asteering committee comprising of the VP, the CIO, and seniorbusiness executives responsible for IS governance. Decisionsfor an initiative could either be a rejection or an approval. Atthe end of ISPP, we gather these decision data from thesedecision makers. The decision makers at our research sitewere fairly certain that they had effectively addressed therisks associated with the funding proposals and had selecteda promising set of initiatives to pursue in the future. Decision

15The second section of Appendix C describes information attributes charac-terizing the portfolio data.

16A stringent selection process, eliminating SOX, IS infrastructure (bothgoverned as enterprise-wide, shared capabilities), and initiatives of lowpriority, helped us isolate a portfolio of 104 initiatives. All initiatives in thisportfolio deal with business applications of IS. Prior work has encouragedstudying only one kind of portfolio at a time (Earl 1989; Sabherwal and Chan2001; Xue et al. 2008).

17Three Ph.D. students in the IS program at a top research university weregiven the proposed initiatives and were asked to independently code thebenefits offered by these initiatives based on definitions from prior literature(Aral and Weill 2007; Broadbent et al. 1999; Sabherwal and Chan 2001). Foreach benefit variable, a value of 1 was assigned to this variable if the raterthought the proposed initiative offered that kind of benefit. All threereviewers agreed in their coding for all benefit variables for over 95% of theinitiatives. Inconsistencies in the coding were resolved by discussionsamongst raters and key informants at the research site.

18We adopt McFarlan’s approach for assessing the risk of proposed initiativesfor two reasons: (1) This approach is geared toward the analysis of portfoliosand easily lends itself to our research objectives. (2) Decisions on initiativeswithin a portfolio require decision makers to compare risks associated withinitiatives.

MIS Quarterly Vol. 39 No. 2/June 2015 421

Page 10: P INFORMATION SYSTEMS PORTFOLIO ... - Dr. Prasanna Karhade · Prasanna Karhade Department of Information Systems, Business Statistics and Operations Management, The Hong Kong University

Karhade et al./Patterns in IS Portfolio Prioritization

makers at the research site validated for us that the ISPP waseffective. In summary, data on all the information attributespertaining to the input portfolio, prepared by front-linemanagers, are collected in collaboration with the metricsgroup within the firm (see Table 519) and final decisions onthese proposed initiatives are independently gathered from theVP, CIO, and senior business executives responsible for ISGovernance. This independence limits the extent to whichour study suffers from the common methods bias.

Decision Tree Induction Methodology

DTIM (Quinlan 1993) enables us to open up the black box ofdecision making ( Langley et al 1995) and empowers us todiscover the underlying, tacit DR applied during ISPP, whichcan often be unknown to the decision makers themselves. This is particularly true when decisions emerge based oninteractions between groups of decision makers. DTIM itera-tively groups together observations (i.e., initiatives) such thatthey are similar not only in certain information attributes butalso in their final decision outcomes. There are two keyinputs to DTIM: (1) a set of initiatives described by all the 15information attributes, and (2) the final decisions made onthese initiatives.

The objective of DTIM is to discover tacit combinations ofinformation attributes associated with similar final decisions(Quinlan 1990, 1993). The output of DTIM is a DT whichonly retains the most pertinent information attributes (i.e.,decision attributes) for explaining decisions. DTIM organizesattributes in a context-dependent manner; certain questionsare only raised depending on answers obtained to otherquestions answered previously (Quinlan 1990).

We would like to clarify that DTs discovered by DTIM arenot reflective of the exact rules or the “script” used by thedecision makers during ISPP, but rather, are approximations20

of the tacit underlying DR. DTIM utilizes the most informa-tive attributes to construct the DT. Instead of the correlationsbetween information attributes, DTIM relies on the amount ofinformation a particular attribute conveys about the finaldecision. Thus, we do not report correlations between infor-mation attributes, but rather present the relationship betweeneach information attribute and the final decision (see thefourth section of Appendix C).

Credible Approximations of Decision Rationale

To ensure that DR is comprehensively discovered from all theinitiatives in the input portfolio, a process of drawing boot-strapped, mutually exclusive, training and testing subsamplesis repeated multiple times. An iteration of DTIM is describednext. In each iteration (shown as artifact DTI in the Figure 2),we draw two random, mutually exclusive subsamples ofinitiatives from the original portfolio; one set, known as thetraining set (Step A1 in Figure 2), from which the tacit DR isdiscovered (Step B in Figure 2 and elaborated on in the nextsection) by the DTI algorithm (Quinlan 1986), and anotherdisjoint set of initiatives, known as the testing set (Step A2 inFigure 2), which is used to test the predictive accuracy of thisdiscovered DR.21

Specifically, a randomly drawn sample of 80 percent of theportfolio was used for training and the prediction accuracy ofthe discovered DR was tested on a disjoint randomly drawnsample of 20 percent of the total portfolio. Predictionaccuracy of the DT discovered from the training set isassessed by applying the DT to predict (Step C in Figure 2)decisions for initiatives in the mutually disjoint testing set.

Multiple such iterations are performed on 80/20 training/testing sample splits to yield multiple plausible DR approxi-mations (trees). Parsimonious DTs are preferable as theycompactly articulate the discovered tacit DR. Further, forrobustness, a similar analysis is conducted using othertraining/testing subsample splits. These multiple iterationshelp us judge the robustness of the discovered tacit DR. Inthe next section, we describe the DTI algorithm in detail andfollow it with a discussion of the heuristics used to determinethe best representative DT from the multiple approximationsgenerated by the DTI algorithm.

Discovering the Most Informative Attributes

The theoretical basis for inducing DTs on a portfolio of deci-sions, each of them described by a set of information attri-butes and final decisions, is summarized as follows (Quinlan1986,1990; Tessmer et al. 1993). This procedure recursivelypartitions the sample into smaller subsets, in step with thegrowth of the DT. DTIM (see Figures 2 and 3) chooses themost pertinent information attribute on which to split thetraining sample and thus which attributes will be included inthe DT (i.e., decision attributes) and is driven by theinformation-theoretic justification presented below.

19The second section of Appendix C describes information attributes forcharacterizing initiatives.

20Each node in the DT is analogous to a question answered prior to theapproval or rejection of an initiative.

21Data from a disjoint testing sample is used to test prediction accuracy ofDR discovered from a training sample.

422 MIS Quarterly Vol. 39 No. 2/June 2015

Page 11: P INFORMATION SYSTEMS PORTFOLIO ... - Dr. Prasanna Karhade · Prasanna Karhade Department of Information Systems, Business Statistics and Operations Management, The Hong Kong University

Karhade et al./Patterns in IS Portfolio Prioritization

Figure 2. Decision Tree Induction Methodology

An information attribute’s entropy H = - 3i pi log2 pi

where i = 1…n; n = distinct values for this attribute;and pi = the probability of each alternative value i,

If-- all decisions belong to a single class (e.g., are all accept/reject decisions,

the tree is a leaf labeled with that classOtherwise,

-- select a test, based on an information attribute, with mutually exclusiveoutcomes;

-- divide the sample into subsets, each corresponding to one value of the informationattribute;

Repeat the same procedure with each subset.

Figure 3. Decision Tree Induction Algorithm

The concept of information entropy (Shannon and Weaver1963) serves as the theoretical basis for determining whichattributes to include in the DT.

The amount of information (or the reduction in uncertainty)provided by the information attribute is the primary justifi-cation for classifying the examples and only the most infor-mative attributes are included in the DT. The entropy oruncertainty is equal to 0 if and only if all the pi’s but one areequal to 0. The entropy is maximum when all the pi’s are

equal; that is, when all alternatives are equally likely. Anychange toward an even distribution of pi’s increases theentropy but as soon as one of the alternatives become moreprobable than others, the entropy decreases. All the DTs areinduced by using the most widely used C4.5 algorithm22

22A precursor to the C4.5 algorithm was ID3, which suffered from somelimitations. The information gain ratio criteria, used by the C4.5 algorithm,which we employ here, corrected these limitations.

MIS Quarterly Vol. 39 No. 2/June 2015 423

Page 12: P INFORMATION SYSTEMS PORTFOLIO ... - Dr. Prasanna Karhade · Prasanna Karhade Department of Information Systems, Business Statistics and Operations Management, The Hong Kong University

Karhade et al./Patterns in IS Portfolio Prioritization

(Quinlan 1986, 1990). This algorithm builds DTs from adataset using information entropy and information gain ratiofor choosing which information attributes are included in theDT (Quinlan 1993). DTs are induced using the open source,software DTIM platform called Weka (Hall et al. 2009).

Three Heuristics for Selecting the BestRepresentative Decision Tree

Multiple approximations of the underlying, tacit DR arederived by repeating the process of drawing two mutuallydisjoint training/testing subsamples of differing sizes. Thisrepetition is integral to DTIM to ensure that multiple approxi-mations of the underlying DR are available to the researchers. Given these multiple approximations, we systematically relyon three heuristics to select the best representative approxi-mation, a credible approximation, of the underlying DR.

1. High predictive accuracy: Prediction accuracy of DTsinduced on a subset of training data is tested on amutually disjoint testing data set. This heuristic repre-sents a goodness-of-fit measure for the DT induced on thetraining dataset in terms of predicting decisions fromunseen data, namely from the mutually disjoint testingsubsample.

2. High parsimony: The induced DT is expected to be acompact, parsimonious approximation of the underlying,tacit DR so that it can serve as an effective decision-making aid.

3. High reliability: Since the process of drawing trainingsamples to induce DTs and testing the predictive accu-racy of induced DTs on mutually disjoint testing samplesis repeated several times, we are able to assess therobustness of the induced DT. For instance, DTs withthe same top-most attribute, showing up reliably acrossthese multiple iterations, represent a robust approxi-mation of the underlying DR. We are, thus, fairly certainthat the DTs presented in this research are credibleapproximations of the underlying, tacit DR applied forISPP. The best representative DTs for IS conservativeand IS innovator are presented in Figures 4 and 5.

Interpreting a Decision Tree

We would like to reiterate that the DTs presented here werediscovered by DTIM as approximations of the tacit DRapplied for ISPP. These DTs do not represent the exact rulesor the script that decision makers were following for ISPP. Since we employ three heuristics (high prediction accuracy,

parsimony, and reliability) to choose a best representative DT,we are fairly certain the DTs that we present here representcredible approximations23 of the tacit DR.

All 15 information attributes characterizing initiatives (seeTable 5) in conjunction with the final decision, are inputs toDTIM. All information attributes discovered by DTIM to bemost informative for explaining decisions are included in theDTs as decision attributes and DTIM excludes all the non-informative attributes from the DT. The most informativedecision attribute is the top-most attribute in the DT. Impor-tance of attributes decreases as we move away from the top ofthe DT to the leaves, namely the endpoints of the DT. DTIMorganizes attributes in a context-dependent manner; certainquestions are raised depending on answers obtained to ques-tions answered previously (Quinlan 1990).

Next, we summarize the operationalization details for threeattributes that define the DR profile: (1) communicability ofthe DR, (2) consistency in applying the DR, and (3) riskappropriateness of the DR. Communicability of the DR isinversely related to the complexity of the DT. Number ofdecision attributes included in the DT is an effective proxy ofthe complexity of the DR. Highest proportion of decisionsmade by tacitly applying the same decision rule represents theconsistency in applying the DR. As this proportion increases,it informs us that the same rule was consistently applied todecide upon a large number of distinct initiatives. Finally, weassess the risk appropriateness of the DR by counting thenumber and kinds of decision attributes included in the DTacross the three categories identified in Table 5: benefits, riskassessment, and risk mitigation attributes. A higher propor-tion of risk assessment and/or mitigation attributes in the DTrepresents a focus on risk assessment/mitigation, as opposedto a focus on exploring new business opportunities.

Discussion of Results andImplications

IS Strategy Types and Best RepresentativeDecision Trees

IS Conservative’s Decision Tree

The number of decision attributes included in the DT, as aproportion of the total number of 15 information attributesprovided as inputs to DTIM, serves as an effective proxy for

23We presented these DTs induced by DTIM to the decision makersthemselves. They validated for us that these artifacts indeed representedcredible approximations of the DR they applied during ISPP.

424 MIS Quarterly Vol. 39 No. 2/June 2015

Page 13: P INFORMATION SYSTEMS PORTFOLIO ... - Dr. Prasanna Karhade · Prasanna Karhade Department of Information Systems, Business Statistics and Operations Management, The Hong Kong University

Karhade et al./Patterns in IS Portfolio Prioritization

Figure 4. IS Conservative’s Decision Tree

Figure 5. IS Innovator’s Decision Tree

MIS Quarterly Vol. 39 No. 2/June 2015 425

Page 14: P INFORMATION SYSTEMS PORTFOLIO ... - Dr. Prasanna Karhade · Prasanna Karhade Department of Information Systems, Business Statistics and Operations Management, The Hong Kong University

Karhade et al./Patterns in IS Portfolio Prioritization

the complexity, which is inversely related to the communi-cability, of the tacitly applied DR. Of the 15 informationattributes provided as inputs, the DR tacitly applied by the ISconservative for ISPP is best24 represented using only five(5/15 = 33%) attributes (see Figure 4). This simple DT easilycommunicates the DR tacitly applied for ISPP. The IS con-servative’s DR contains a decision rule consistently appliedwith a very high frequency. In all, 63 percent of the decisionsfor the IS conservative’s portfolio are explained by the appli-cation of just this one main25 decision rule. In other words,we observe that the IS conservative’s DR comprises of asimple, highly consistent line of reasoning. IS conservative’sDT includes only 1 benefit attribute and 2 decision attributeseach for assessing and mitigating risks.

When deciding on the IS conservative’s portfolio, decisionmakers seem to rely on a higher proportion (4/5 = 80%) ofrisk assessment/mitigation decision attributes. Thus, the ISconservative’s DR is highly consistent, easily communicable,and focuses on assessment/mitigation of risks. Our findingssupport Proposition 1.

Additionally, the IS conservative’s DT reveals that certainmedium-sized initiatives that do not offer cycle time reduc-

tions are approved and similar small-sized initiatives arerejected. Collectively, these decisions seem rationally unjus-tifiable. Alhough we do not investigate the political aspectsof the DR in this study, DTIM empowers us to identifyinstances of rationally unjustifiable decisions. These deci-sions might be politically motivated, thus DTIM can serve asa starting point for scholars interested in investigatingpolitically driven decision making.

IS Innovator’s Decision Tree

Of the 15 information attributes provided as inputs, DTIMdiscovered that the DR tacitly applied by the IS innovator (seeFigure 5) for ISPP was best26 represented by six decisionattributes. A total of 40 percent (6/15 = 40%) of the informa-tion attributes are needed to communicate the IS innovator’sDR, implying that this rationale is not very simple. Only 40percent of the decisions in the IS innovator portfolio areexplained by the use of the most consistent decision rule27

(see Figure 5). The proportion of decisions made most con-

24The DT chosen to represent the IS conservative’s DR has good predictiveaccuracy (greater than 90%). Given that decisions for the IS conservative’sportfolio included approval and rejection decisions, in the absence of a DT,we have a 50% chance of predicting the right decision. A DT with predictionaccuracy greater than 90% represents a credible approximation. This port-folio contains a total of 72 decisions. A DT containing 36 decision rules for72 decisions would not be parsimonious at all. Alternatively, a DT with 1decision rule for the portfolio of 72 decisions would also not serve as aneffective decision-making aid. This DT is a collection of seven decisionrules, indicating that it is a parsimonious approximation. This DT had highreliability (the same attribute was the top-most attribute in over 50% of DTswith high predictive accuracy and parsimony), indicating that it is a robustapproximation.

25The main decision rule (from Figure 4) is used to make a large number ofdecisions on the IS conservative’s portfolio. This decision rule, presentednext, contains only two decision attributes.

IF (“Initiative Benefits = Cycle Time Reductions”) AND(“Internal Maturity = Medium”) THEN Approve Initiative

In all, 31 of the total 49 initiatives (31/49 = 63%) were approved using thisdecision rule. Thus, this rule was consistently used 31 times to decide on 31different initiatives (of the total 49 initiatives used to induce this DT) in theIS conservative’s portfolio. This finding is in line with IS conservativebehavior that relies on a formalized top–down approach to decision making. Finally, considered simultaneously, two attributes contained in this simpledecision rule holistically narrate a DR that is in congruence with risk averseIS conservative behavior. First, initiatives were approved after ascertainingthat they offered efficiency improvements. Second, this simple rule alsoreveals that risks associated with these initiatives were mitigated by ensuringthat these proposals were not nascent ideas but were relatively matureinitiatives (internal maturity of these initiatives was not low, but wasmedium).

26The DT chosen to represent the IS innovator’s DR has good predictiveaccuracy (greater than 90%), represents a parsimonious approximation, andhigh reliability (the same attribute was the top-most attribute for over 50% ofthe DTs with high predictive accuracy and parsimony).

27The dynamism in the external environment is likely to exert a stronginfluence on the IS innovator’s DR, which can be studied by closelyexamining the main decision rule from the IS innovator’s DT. Another ruleinvolving the top-most important decision attribute is also discussed here asthese two rules collectively highlight the IS Innovator-like behavior. Thesetwo key rules extracted from the IS innovator’s DT are presented below:

1. If (“Initiative Benefits = Interorganizational Process Improvements”)THEN Reject Initiative

2. If (“Initiative Benefits = Marketing Benefits”) THEN Approve Initiative

Eight of the total 20 initiatives (40%) were rejected using the first decisionrule whereas 2 of the total 20 initiatives were approved using the seconddecision rule. Initiatives that enable IS innovators to tap into new marketsusing IS by offering marketing benefits are approved. Initiatives designed forimproving the efficiency of existing interorganizational (IO) businessprocesses are rejected. Both of these rules suggest a strong preference forundertaking initiatives that enable IS innovators to explore new markets andan aversion to using IS only for incrementally improving business processesthat span their existing firm boundaries. It is not that IS innovators do notinvest in improving existing IO processes (5 of the total 16 such initiativeswere approved), just that they do not emphasize such improvements as ISconservatives do. It is easier to understand IS innovator behavior bycontrasting it with IS conservative behavior. IS conservatives maintainsimple business environments by developing stable business transactions, ISinnovators aspire to explore/enter new markets. Their customers and/orsuppliers in these new markets are likely to change and transactions withthese new partners are susceptible to constant adjustments. Given theemphasis on exploring new markets, initiatives designed for improving theefficiency of existing business processes are less appealing to IS innovators. In fact, IS innovators are likely to perceive such initiatives as harbingers ofrigidity that are likely to hinder their ability to successfully explore newmarkets in the future (Sabherwal and Chan 2001).

426 MIS Quarterly Vol. 39 No. 2/June 2015

Page 15: P INFORMATION SYSTEMS PORTFOLIO ... - Dr. Prasanna Karhade · Prasanna Karhade Department of Information Systems, Business Statistics and Operations Management, The Hong Kong University

Karhade et al./Patterns in IS Portfolio Prioritization

Table 6. Summary of Findings

IS Conservative Strategy IS Innovator Strategy

Decision Rationale Profile Attributes

Communicability: Inversely related to complexity of DT: Total Number of attributes provided as inputs to the decision ireeinduction methodology = 15

Measures of Complexity {Number of Decision Attributes, Number ofattributes in the Longest Decision Rule,Total Number of Leaves}

{5,4,7} {6, 6, 7}

Proportion of decision attributes included 5/15 = 33% 6/15 = 40%

Consistency: Proportion of distinct initiatives decided upon using the same rule

63% of distinct decisions madeusing one decision rule

40% of distinct decisions madeusing one decision rule

Risk Appropriateness: Focus during decision making

Proportion of risk assessment and mitigationattributes included

80% (4 of total 5 attributes are riskassessment/mitigation attributes)

50% (3 of total 6 attributes are riskassessment/mitigation attributes)

Decision Rationale Profile

DR Profile / {Proportion of Information Attributes Retained as Decision Attributes, Proportion of distinct decisions madeusing one decision rule, Proportion of Risk Assessment/Mitigation Attributes in the DT}

Observed Profile {33%, 63%, 80%} / {High Communicability , HighConsistency, Focus on RiskMitigation}

{40%,40%,50%}/ {Low-Moderate Communicability,Low-Moderate Consistency, Focuson Exploring Opportunities}

sistently using just one decision rule was lower than theproportion that was observed in the IS conservative’s port-folio. The IS innovator’s DT includes three benefits attri-butes, one attribute for risk assessment, and two decisionattributes for risk mitigation.

When prioritizing the IS innovator portfolio, decision makersseem to rely on a higher proportion (that is, 50%) of benefit-related decision attributes implying a focus on exploringopportunities. In summary, the IS innovator’s DR is notnecessarily easy to communicate, does not apply a decisionrule with high consistency, and focuses on exploring oppor-tunities. Our findings support Proposition 2. A summary ofour findings is presented in Table 6.

Theoretical Implications

One of the key goals of IS governance is to encouragedesirable behavior in the prioritization of IS portfolios. Thus,our research, which examines DR used for ISPP, has strongimplications for IS governance. Next, we present theseimplications from our research for IS governance.

First, although prior research has provided extensive insightson the locus (centralization and/or decentralization) of the

decision making for IS governance, questions pertaining towhy firms make ISPP decisions have been left unanswered. Our study is one of the first to empirically examine the rela-tionship between ISPP decisions (namely, approval andrejection decisions on IS-enabled business initiatives topursue for the future) and the IS strategy of the firm, which iscritical for IS governance.

Second, our research proposes that the IS strategy of a firm28

is likely to be a key antecedent in explaining the DR it appliesfor prioritizing its IS portfolios. We submit that congruencebetween a firm’s IS strategy and its ISPP DR is likely to beassociated with effectively planned IS portfolios as it not onlyincentivizes managers to develop the right kinds of initiativesbut also guides decision makers to approve the right kinds ofinitiatives for future implementation. We map DR profiles forstrategies along the IS conservative–innovator continuum inFigure 6.

28Theory for firms that adopt a dual IS strategies is presented in Appendix A. The results of the analysis of a portfolio of 57 initiatives are presented inAppendix D. Extending our research design, we develop insights for firmsthat adopt a dual IS strategy. Such firms are likely to have a complex DR incongruence with their conflicting objectives. Given these conflictingobjectives, the consistency with which such firms apply their DR is likely tobe low. Given their risk-averse tendencies, firms that adopt a dual IS strategyare likely to focus on risk mitigation when prioritizing their IS portfolios.

MIS Quarterly Vol. 39 No. 2/June 2015 427

Page 16: P INFORMATION SYSTEMS PORTFOLIO ... - Dr. Prasanna Karhade · Prasanna Karhade Department of Information Systems, Business Statistics and Operations Management, The Hong Kong University

Karhade et al./Patterns in IS Portfolio Prioritization

Figure 6. Mapping Decision Rationale Profiles

Third, while existing research has examined IS strategies byanalyzing variations in the composition of IS applicationportfolios (Sabherwal and Chan 2001), we broaden thisimportant body of work by investigating the theoreticalproperties of DR applied during ISPP. We contribute to theliterature on IS governance by addressing a key question onISPP: Why are certain initiatives approved and simul-taneously why are certain others rejected? We do so bydeveloping theoretical profiles of the ISPP DR by en-compassing three dimensions: communicability of DR, con-sistency in applying DR, and risk appropriateness of DR. Wesubmit that understanding theoretical DR profiles acrossdiffering IS strategies strongly complements extant ISgovernance research by explicitly examining if the right kindsof questions are raised during ISPP. Thus, we present theo-retical properties of the DR which can encourage rule-following in congruence with a firm’s IS strategy. We adopta portfolio-level unit of analysis to contribute insights on themeans firms adopt to effectively prioritize their IS portfolios.

Extensions

We extended research on IS conservative–innovator strategyby proposing DR profiles for ISPP, in congruence with these

IS strategies. Analysis of our data yielded support for ourpropositions. Building on this foundation, we identify twoextensions for future research.

Egregious Deviations from TheoreticalDecision Rationale Profiles

Deviations29 from the theoretical DR profile applied by firmswho have adopted strategies along the IS conservative–innovator continuum is likely to be associated with ineffectiveISPP (Segars and Grover 1999) for at least three reasons.

First, for IS conservatives, a complex DR is indicative of anunnecessarily complex decision process that is inefficiently

29Researchers can adopt a profile-deviation research design (Drazin and Vande Ven 1985) to study the impact of egregious deviations from theoretical DRprofiles on planning effectiveness. We submit that deviations from thetheoretical DR profile could be politically motivated. Favoritism on behalfof decision makers would result in deviations from the theoretical DR alongthe dimension of consistency. DTIM is ideal for detecting politically moti-vated, rationally unjustifiable decisions. Ethnographic research designs couldcomplement DTIM by shedding light on mechanisms (Eisenhardt andBourgeois 1988) underpinning these egregious decisions.

428 MIS Quarterly Vol. 39 No. 2/June 2015

Page 17: P INFORMATION SYSTEMS PORTFOLIO ... - Dr. Prasanna Karhade · Prasanna Karhade Department of Information Systems, Business Statistics and Operations Management, The Hong Kong University

Karhade et al./Patterns in IS Portfolio Prioritization

consuming resources such as executive time and attention. Incontrast, an overly simple DR for an IS innovator is indicativeof a decision process that is not sufficiently responsive to information flows in the IS innovator’s dynamic externalenvironment.

Second, a low consistency in the DR applied by an IS conser-vative is likely to indicate that similar initiatives are beingawarded different decisions, which would be unsuitable froman IS governance perspective. Favoritism could be oneunderlying explanation for why similar initiatives might in-consistently be awarded different ISPP decisions. This wouldbe ineffective from an IS governance perspective as it wouldmean that front-line managers whose proposals are beingrejected are being given no credible feedback for improvingtheir proposals. This outcome of the ISPP represents a majorloss of learning opportunities for front-line managers. Veryhigh consistency in the DR applied by an IS innovator isindicative of an rigid decision process, unsuitable to ISinnovators whose decision making is expected to be organic,in response to the dynamism of its external environment.

Third, a misguided focus during ISPP, revealed by an inap-propriate mix of decision attributes in the DTs, would beindicative of two crucial deficiencies. A misguided focusduring ISPP is likely to suggest that unsuitable initiatives arebeing approved and/or unsuitable initiatives are not beingrejected. Deviations along this dimension could result in anIS conservative’s portfolio that is incongruently focused onexploring opportunities, which is indicative of the IS conser-vative taking on unjustifiable risks. Along similar lines, an ISinnovator’s misguided focus on risk mitigation, would beindicative of missed opportunities.

Thus, deviations along any of the three dimensions definingthe theoretical DR profile, is likely to be detrimental to ISPPand IS governance.

Proposition 1 for Future Research: Deviationbetween the decision rationale applied by firmsadopting strategies along the IS conservative–innovator continuum and the theoretical decisionrationale profile, is likely to be associated withineffective IS portfolio prioritization.

Mediating Role of the Decision Rationale Profile

Prior research maintains that a chosen IS strategic posture ordigital business strategy is an antecedent of firm-level out-comes (Mithas et al. 2013). Our results highlight an inter-vening theoretical mechanism, namely DR applied by firmsfor ISPP, which is likely to have a strong impact on firm-levelperformance outcomes. This revised articulation sheds light

on the mediating influence of the DR applied for ISPP for tworeasons, which we elaborate next.

First, congruence between a firm’s IS strategy and the DR ituses for prioritizing its IS portfolio fosters a rule-basedapproach to portfolio prioritization which provides front-linemanagers with the right incentives to design suitable IS-enabled initiatives and disincentivizes (rule-defiant) behavior(e.g., Prendergast 1999). Incentivizing managers is criticalfor effective ISPP and eventually the superior business valueof IS (Kohli and Grover 2008). Second, congruence betweenDR employed by firms during ISPP and their IS strategies islikely to ensure the suitable initiatives are approved for futureimplementation and the unsuitable initiatives are weeded outearlier in the prioritization stages. Often, unsuitable choicesmade early during strategic IS portfolio planning are costly tocorrect later during plan implementation. Thus, we maintainthat the DR used during prioritization has a strong bearing onthe implementation of planned IS portfolios, which eventuallyinfluences the business value realized from these portfolios. Incorporating this new theoretical construct into the businessvalue of IT research (e.g., Kohli and Grover 2008) wouldrequire scholars to carefully choose a unit of analysis suitablefor evaluating the true impact of ISPP DR on the businessvalue realized from IS strategies.

Proposition 2 for Future Research: Decision ration-ale profile is likely to mediate the relationshipbetween planned IS strategies along the ISconservative–innovator continuum and the businessvalue realized from these investments.

Managerial Implications of Our Results

DTs open up the black box of IS decision making bycodifying the tacit DR applied during ISPP in a manner thatcan easily be scrutinized by decision makers. Our findingsenable us to develop recommendations for assisting CIOs inmanaging their IS portfolios (Maizlish and Handler 2005). The first three steps we outline next (see Table 7) follow fromour research design whereas the latter three follow from ourfindings.

Identify initiatives in distinct portfolios: Although DR forprioritizing distinct portfolios (IT hardware/infrastructureportfolios versus IS portfolios) is likely to be different, wepropose that to govern these portfolios effectively, it is criticalfor CIOs to catalogue all initiatives in these portfolios (Kumaret al. 2008). Identifying initiatives in distinct portfoliosenables CIOs to govern them effectively, as they can system-atically track the progress of all their portfolios.

MIS Quarterly Vol. 39 No. 2/June 2015 429

Page 18: P INFORMATION SYSTEMS PORTFOLIO ... - Dr. Prasanna Karhade · Prasanna Karhade Department of Information Systems, Business Statistics and Operations Management, The Hong Kong University

Karhade et al./Patterns in IS Portfolio Prioritization

Table 7. Managerial Implications

Portfolio Management Lifecycle Key Recommendations

Identify initiatives in distinct portfolios Maintain distinct portfolios in hardware (infrastructure) and software(business applications of IS)

Gather portfolio data Define initiatives in a portfolio in an information-rich manner

Define the decision schemes Select decision schemes suitable to the firm strategic objectives

Develop a repository of decision rules CIOs need to articulate their decision rules; DTs we present in ourresearch could serve as a starting point

Apply decision rules Determine nonnegotiable decision rules to apply and identify means todeal with exceptions.

Manage portfolio lifecycles Identify opportunities for continual refinements for IS portfoliomanagement capabilities

Gather portfolio data: Information attributes used to describeinitiatives must be consistently used across all initiativeswithin a portfolio. Ideally, initiatives need to be described inadequate detail to assist CIOs to make well-informed deci-sions. At the same time, providing too much information canbe overwhelming to CIOs, ultimately proving to be counter-productive.

Define the decision scheme: For some portfolios, a binary(yes/no) decision scheme might be suitable whereas otherinstances might necessitate an additional partial fundingdecision outcome. For instance, binary decisions can besimplistic to administer but can create divisive, disincenti-vizing rifts between managers within a firm. Thus, theimplications of choosing between different decision schemesneed to be systematically evaluated.

Develop a repository of decision rules: CIOs need to articu-late their DR in the form of decision rules. These rulescapture the decision attribute(s) that guide CIOs to eitherapprove or reject initiatives within a portfolio. DR typicallytaps into vast amounts of tacit, prior experience that enablesCIOs to identify combinations of attributes that describeinitiatives they believe are likely to be successful. Decisionrules codify tacit DR in the form of if–then–else statements.

In other words, decision rules can tie information attributes tothe decision outcomes. As various decision makers articulatetheir individual DRs in the form of rules, to the extent pos-sible, they develop a repository of decision rules. Moreimportantly, decision makers can compare their DRs, under-stand the point of view of others, and eventually agree upondecision rules to apply during ISPP. Such collective exercisescan enable decision makers to arrive at a set of rules that arecollectively deemed nonnegotiable. Ideally, decision makersneed to arrive at a consensus on nonnegotiable decision rulesto apply before they start prioritizing IS portfolios. DTs wepresent in our study can serve as a starting point in assisting

CIOs, adopting different kinds of IS strategies, in developinga repository of decision rules.

Although codification of tacit DR is likely to assist decisionmakers responsible for ISPP, there are often darker conse-quences to openly sharing all these rules. For instance, if allrules are made explicit, then managers proposing initiativesare likely to game the system by designing initiatives to onlycomply with these codified approval rules. CIOs should exer-cise caution in sharing their rules with managers responsiblefor proposing new initiatives.

Apply decision rules: Having reached a consensus on therepository of nonnegotiable decision rules to apply, inter-actions between CIOs and front-line managers proposinginitiatives during ISPP are likely to be more efficient and fair. To the extent possible, CIOs can consistently apply therepository of nonnegotiable decision rules to prioritize theirportfolios. A repository of explicit decision rules can alsoenable CIOs to systematically handle exceptions to rules. Wepropose that a rule-based approach to ISPP improves thetransparency of decision making and enables CIOs to incenti-vize front-line managers to continually improve theirinitiatives.

Manage portfolio lifecycles: CIOs should strive to improvethe consistency of their decision making, to the extent pos-sible, and educate front-line managers to develop better initia-tives. As portfolio decisions are implemented over time,additional data on the performance of approved initiatives canbecome available to CIOs. This performance data couldguide future decisions but should be used with caution. If ap-proved initiatives are not eventually successful, the associatedpenalties should not be too severe or else managers hoping forfunding will, in future, only craft conservative initiatives. Such a systematic conservative bias is likely to prevent man-agers from realizing the true potential of IS. On the otherhand, if there are no penalties associated with failed initia-

430 MIS Quarterly Vol. 39 No. 2/June 2015

Page 19: P INFORMATION SYSTEMS PORTFOLIO ... - Dr. Prasanna Karhade · Prasanna Karhade Department of Information Systems, Business Statistics and Operations Management, The Hong Kong University

Karhade et al./Patterns in IS Portfolio Prioritization

tives, managers might not cultivate any discipline and arelikely to continue to craft grandiose initiatives withoutexerting effort to manage risks. Thus, CIOs need to appraisethe performance of approved initiatives, but leverage thisperformance data judiciously.

Concluding Remarks

Limitations

Our study has certain limitations. First, our data are obtainedfrom two business units within one large Fortune 50 firm. This could imply that our work suffers from limited generali-zability. To address this limitation, we integrate threeattributes (communicability of DR, consistency in applyingDR, and risk appropriateness of D) for developing theoreticalprofiles of DR tacitly applied during ISPP. Our researchenables us to theorize the linkages between the IS strategyadopted by firms and the corresponding DR applied duringISPP. Insights from our research are thus likely to be appli-cable to firms that can be characterized using the ISconservative–innovator typology. Second, while the cate-gories and specific anchors used as information attributes inthis study (see Table 5) were validated for their comprehen-siveness at our research site, we acknowledge that investi-gations at other firms might require refinements to theseinformation attributes. Information attributes used to charac-terize initiatives in our data can serve as a starting point inguiding other scholars who wish to build on our research.

The limitations identified above, meanwhile, do offer otheradvantages. Our research allows us to test differences in DRacross two business units pursuing different IS strategieswithin a naturally controlled empirical setting. In line withsuggestions from prior research (Sabherwal and Chan 2001),we examine one kind of portfolio, namely business applica-tions of IS, and this focus enables us to control for con-founding factors pertinent to other kinds of IS portfolios.

Conclusion

In this paper, we submit that the IS strategy of a firm is likelyto explain its ISPP DR. We examine differences in the ISPPDR across different IS strategy types. We develop theoreticalprofiles of DR used for ISPP by relying on three attributes: communicability of DR, consistency in applying DR, and therisk appropriateness of DR. Since DR applied during ISPP isoften tacit, unknown even to the decision makers themselves,we adopt DTIM, which is appropriate for discovering this

tacit DR. By analyzing over 150 actual ISPP decisions, wefind support for our theoretical DR profiles. Implications forIS governance practices are developed.

Acknowledgments

We are grateful to senior executives and managers at the researchsite for assistance with this project. We acknowledge financial sup-port from The Hong Kong University of Science and Technology,University of Illinois, and the Leonard C. and Mary Lou Hoeftendowment at the University of Illinois. We would like to thankJohn Burke, Wooje Cho, Joe Mahoney, Arun Rai, and seminar parti-cipants at The City University of Hong Kong and Seoul NationalUniversity for their helpful comments on this paper. We would alsolike to thank the senior editor, Rajiv Kohli, the associate editor, PaulPavlou, and the three reviewers for their helpful comments andexcellent suggestions. The usual disclaimer applies.

References

Aral, S., and Weill, P. 2007. “IS Assets, Organizational Capa-bilities, and Firm Performance: How Resource Allocations andOrganizational Differences Explain Performance Variation,”Organization Science (18:5), pp. 763-780.

Baligh, H. H., Burton, R. M., and Obel, B. 1996. “OrganizationalConsultant: Creating a Useable Theory for OrganizationalDesign,” Management Science (42:12), pp. 1648-1662.

Bardhan, I., Baghchi, S., and Sougstad, R. 2004. “Prioritizing aPortfolio of Information Technology Investment Projects,”Journal of Management Information Systems (21:2), pp. 33-60.

Bardhan, I. R., Kauffman, R. J., and Naranpanawe, S. 2010. “ITProject Portfolio Optimization: A Risk Management Approachto Software Development Governance,” IBM Journal ofResearch and Development (54:2), pp. 2-1.

Boonstra A. 2003. “Structure and Analysis of IS Decision-MakingProcesses,” European Journal of Information Systems (12:3), pp.195-209.

Boynton, A. C., and Zmud, R. W. 1987. “Information TechnologyPlanning in the 1990’s: Directions for Practice and Research”MIS Quarterly (11:1), pp. 59-71.

Broadbent, M., Weill, P., Clair, D. S., and Kearney, A. T. 1999. “The Implications of Information Technology Infrastructure forBusiness Process Redesign,” MIS Quarterly (23:2), pp. 159-182.

Brown, C. V. 1997. “Examining the Emergence of Hybrid ISGovernance Solutions: Evidence from a Single Case Site,”Information Systems Research (8:1), pp. 69-94.

Chen, D., Preston, D. S., Mockler, M., and Teubner, A. 2010. “Information Systems Strategy: Reconceptualization, Measure-ment, and Implications,” MIS Quarterly (34:2), pp. 233-259.

Drazen, R., and Van de Ven, A. 1985. “Alternative Forms of Fit inContingency Theory,” Administrative Science Quarterly (30), pp.514-539.

Earl, M. J. 1989. Management Strategies for Information Tech-nology, Englewood Cliffs, NJ: Prentice-Hall.

MIS Quarterly Vol. 39 No. 2/June 2015 431

Page 20: P INFORMATION SYSTEMS PORTFOLIO ... - Dr. Prasanna Karhade · Prasanna Karhade Department of Information Systems, Business Statistics and Operations Management, The Hong Kong University

Karhade et al./Patterns in IS Portfolio Prioritization

Earl, M. J. 1993. “Experiences in Strategic Information SystemsPlanning,” MIS Quarterly (17:1), pp. 1-24.

Eisenhardt, K. M., and Bourgeois, L. J. 1988. “Politics of StrategicDecision Making in High-Velocity Environments: Toward aMidrange Theory,” Academy of Management Journal (31:4), pp.737-770.

Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., andWitten, I. H. 2009. “The WEKA Data Mining Software: AnUpdate,” ACM SIGKDD Explorations Newsletter (11:1), pp.10-18.

Huber G. P. 1981. “Organizational Decision Making and theDesign of Decision Support Systems,” MIS Quarterly (5:2), pp.1-10.

Iversen, J. H., Mathiassen, L., and Nielsen, P. A. 2004. “ManagingRisk in Software Process Improvement: An Action ResearchApproach,” MIS Quarterly (28:3), pp. 395-433.

Jeffery, M., and Leliveld, I. 2004. “Best Practices in IS PortfolioManagement,” Sloan Management Review (45:3), pp. 41-49.

Jansen, J. J. P., Van Den Bosch, F. A. J., and Volberda, H. W. 2006. “Exploratory Innovation, Exploitative Innovation, and Perfor-mance: Effects of Organizational Antecedents and Environ-mental Moderators,” Management Science (52:11), pp.1661-1674.

Kauffman, R. J., and Sougstad, R. 2008. “Risk Management ofContract Portfolios in IT Services: The Profit-at-Risk Ap-proach.” Journal of Management Information Systems (25:1), pp.17-48.

Ko, K., Kirsch, L. J., and King, W. R. 2005. “Antecedents ofKnowledge Transfer from Consultants to Clients in EnterpriseSystem Implementations,” MIS Quarterly (29:1), pp. 59-85.

Kohli, R. 2007. “Innovating to Create IS-Based New BusinessOpportunities at United Parcel Service,” MIS Quarterly Executive(6:4), pp. 199-210.

Kohli, R., and Grover, V. 2008. “Business Value of IT: An Essayon Expanding Research Directions to Keep Up with the Times,”Journal of the Association for Information Systems (9:1), pp.23-39.

Kumar, R., Ajjan, H., and Niu, Y. 2008. “Information TechnologyPortfolio Management: Literature Review, Framework, andResearch Issues,” Information Resources Management Journal(21:3), pp. 64-87.

Langley, A., Mintzberg, H.,Pitcher, P. Posada, E., Saint-Macary, J. 1995. “Opening Up Decision Making: The View from theBlack Stool,” Organization Science (6:3), pp. 260-279.

Lambert, R. A. 1986. “Executive Effort and Selection of RiskyProjects,” RAND Journal of Economics (17:1), pp. 77-88.

Lyytinen K., Mathiassen L., and Ropponen J., 1998. “AttentionShaping and Software Risk: A Categorical Analysis of FourClassical Approaches,” Information Systems Research (9:3), pp.233-255.

Maizlish, B., and Handler, R. 2005. IS Portfolio Management: Unlocking the Business Value of Technology, New York: JohnWiley & Sons.

Markus, M. L., Majchrzak, A., and Gasser, L. 2002. “A DesignTheory for Systems that Support Emergent Knowledge Pro-cesses,” MIS Quarterly (26:3), pp. 179-212.

March, J. G. 1994. Primer on Decision Making: How DecisionsHappen, New York: Free Press.

March, J. G., and Shapira, Z. 1987. “Managerial Perspectives onRisk and Risk Taking,” Management Science (33:11), pp.1404-1418.

McFarlan, F. W. 1981. “Portfolio Approach to Information Sys-tems,” Harvard Business Review (59:5), pp. 142-150.

Miles, M., and Huberman, A. M. 1984. Qualitative Data Analysis,Beverly Hills, CA: Sage Publications.

Mintzberg, H. 1994. The Rise and Fall of Strategic Planning, NewYork: MacMillan.

Mitchell, V. L., and Zmud, R. W. 2006. “Endogenous Adaptation: The Effects of Technology Position and Planning Modes on IT-Enabled Change,” Decision Sciences (37:3), pp. 325-355.

Mithas, S., Tafti, A., and Mitchell, W. 2013. “How a Firm’s Com-petitive Environment and Digital Strategic Posture InfluenceDigital Business Strategy,” MIS Quarterly (37:2), 511-536.

Nolan, R., and McFarlan, F. W. 2005. “Information Technologyand the Board of Directors,” Harvard Business Review (83:10),pp. 96-106.

Philip, G. 2007. “IS Strategic Planning for Operational Efficiency,”Information Systems Management (24:3), pp. 247-264.

Piccoli, G., and Ives, B. 2005. “IT-Dependent Strategic Initiativesand Sustained Competitive Advantage: A Review and Synthesisof the Literature,” MIS Quarterly (29:4), pp. 747-776.

Prendergast, C. 1999. “The Provision of Incentives in Firms,”Journal of Economic Literature (37:1), pp. 7-63.

Quinlan, J. R. 1986. “Induction of Decision Trees,” MachineLearning (1:1), pp. 81-106.

Quinlan, J. R. 1990. “Decision Trees and Decision Making,” IEEETransactions on Systems, Man, and Cybernetics (20:2), pp.339-346.

Quinlan, J. R. 1993. C4.5: Programs for Machine Learning, SanMateo, CA: Morgan Kauffman Publishers.

Rajegopal, S. 2012. Portfolio Management: How to Innovate andInvest in Successful Projects, Basingstoke, UK: PalgraveMacmillan.

Ramasubbu, N., Mithas, S., Krishnan, M., and Kemerer, C. 2008. “Work Dispersion, Process-Based Learning, and Offshore Soft-ware Development Performance,” MIS Quarterly (32:2), pp.437-458.

Reyck, B. D., Grushka-Cockayne, Y., Lockett, M., Calderini, S. R.,Moura, M., and Sloper, A. 2005. “The Impact of Project Port-folio Management on Information Technology Projects,” Inter-national Journal of Project Management (23:7), pp. 524-537.

Sabherwal, R., and Chan, Y. E. 2001.”Alignment between Businessand IS Strategies: A Study of Prospectors, Analyzers, andDefenders,” Information Systems Research (12:1), pp. 11-33.

Sambamurthy, V., and Zmud, R. W. 1999. “Arrangements forInformation Technology Governance: A Theory of MultipleContingencies,” MIS Quarterly (23:2), pp. 261-290.

Segars, A. H., and Grover, V. 1999. “Profiles of Strategic Informa-tion Systems Planning,” Information Systems Research (10:3),pp. 199-232.

Shannon, C. E., and Weaver, W. 1963. The Mathematical Theoryof Communication, Urbana, IL: University of Illinois Press.

Sherer, S. A., and Alter, S. 2004. “Information Systems Risks andRisk Factors: Are They Mostly About Information Systems?,”

432 MIS Quarterly Vol. 39 No. 2/June 2015

Page 21: P INFORMATION SYSTEMS PORTFOLIO ... - Dr. Prasanna Karhade · Prasanna Karhade Department of Information Systems, Business Statistics and Operations Management, The Hong Kong University

Karhade et al./Patterns in IS Portfolio Prioritization

Communications of the Association for Information Systems(14:1), Article 2.

Straub, D. W., and Welke, R. J. 1998. “Coping with Systems Risk: Security Planning Models for Management Decision Making,”MIS Quarterly (22:4), pp. 441-469.

Susarla, A., Subramanyam, R., and Karhade, P. 2010. “ContractualProvisions to Mitigate Holdup: Evidence from InformationTechnology Outsourcing,” Information Systems Research (21:1),pp. 37-55.

Tessmer, A. C., Shaw, M. J., and Gentry, J. A. 1993. “InductiveLearning for International Financial Analysis: A Layered Ap-proach.” Journal of Management Information Systems (9:4), pp.17-37.

Weill, P., and Ross, J. W. 2004. IS Governance: How TopPerformers Manage IS Decision Rights for Superior Results,Boston: Harvard Business School Press

Xue, Y., Liang, H., and Boulton, W. R. 2008. “Information Tech-nology Governance in Information Technology InvestmentDecision Processes: The Impact of Investment Characteristics,External Environment, and Internal Context,” MIS Quarterly(32:1), pp. 67-96.

About the Authors

Prasanna Karhade is currently an assistant professor in the Depart-ment of Information Systems, Business Statistics and OperationsManagement at The Hong Kong University of Science and Tech-

nology. He earned his Ph.D. from the University of Illinois atUrbana-Champaign in 2009. His research interests include designof formal contracts for governing IT outsourcing relationships, ITgovernance, and the impact of IT on firm innovation. His researchhas been published in Information Systems Research. Prasanna isthe contact author for this article.

Michael J. Shaw is Hoeft Chair of Information Systems in theDepartment of Business Administration at the University of Illinoisat Urbana-Champaign. He has been the editor-in-chief (withJ. Becker) of the journal Information Systems and e-BusinessManagement. He is affiliated with, in addition to his home depart-ment, the National Center for Supercomputing Applications and theInformation Trust Institute. His most recent research has focused onbusiness analytics and on the functions of chief information officers,supply-chain management, and electronic commerce.

Ramanath (Ram) Subramanyam is an associate professor ofBusiness Administration at the University of Illinois at Urbana-Champaign. He earned his Ph.D. from the Ross School of Business,University of Michigan (2004) and a Bachelors of Electronics andCommunication Engineering from NIT, Trichy, India. His researchinterests include IT sourcing governance, management of IS designprocesses and project management, new product development,customer influences on technological product design, and IT-drivensustainability in products and processes. His research has appearedin various journals, including Information Systems Research,Manufacturing & Service Operations Management, IEEE Trans-actions on Software Engineering, and Decision Support Systems.

MIS Quarterly Vol. 39 No. 2/June 2015 433

Page 22: P INFORMATION SYSTEMS PORTFOLIO ... - Dr. Prasanna Karhade · Prasanna Karhade Department of Information Systems, Business Statistics and Operations Management, The Hong Kong University

434 MIS Quarterly Vol. 39 No. 2/June 2015

Page 23: P INFORMATION SYSTEMS PORTFOLIO ... - Dr. Prasanna Karhade · Prasanna Karhade Department of Information Systems, Business Statistics and Operations Management, The Hong Kong University

RESEARCH ARTICLE

PATTERNS IN INFORMATION SYSTEMS PORTFOLIOPRIORITIZATION: EVIDENCE FROM DECISION

TREE INDUCTION

Prasanna KarhadeDepartment of Information Systems, Business Statistics and Operations Management, The Hong Kong University of

Science and Technology, Clear Water Bay, Kowloon, HONG KONG {[email protected]}

Michael J. Shaw and Ramanath SubramanyamDepartment of Business Administration, University of Illinois at Urbana–Champaign,

Champaign, IL 61820 U.S.A. {[email protected]} {[email protected]}

Appendix A

Theory Development: Decision Rationale Profile for Firms thatAdopt Dual IS Conservative- and IS Innovator-Like Strategies

Some firms adopt dual (IS-conservative-like and IS innovator-like) strategies. The behavior of firms that adopt a dual IS strategy is associatedwith the following characteristics: they (1) adhere to dual goals with a simultaneous emphasis on efficiency improvements and cautiousexploration of new opportunities, (2) adopt a relatively formal decision-making structure, and (3) are inherently risk averse. Dual goals arelikely to exert a conflicting influence on the behavior of firms that adopt both IS conservative- and IS innovator-like strategies.

Balancing dual goals can be cognitively demanding and thus we maintain that the complexity of the decision rationale applied by such firmsis likely to be high. Thus, the communicability of this decision rationale is likely to be low. The consistency with which decision rules areapplied by such firms in the process of managing these dual conflicting goals is likely to be low as applying the same rule with a highconsistency would imply that one of the dual goals is most likely not being sufficiently addressed. Acting appropriately, decision makers witha low-risk appetite are expected to approve high-risk initiatives only after ensuring that risk mitigation mechanisms have been designed to lowerthe likelihood of failures (Boynton and Zmud 1987; March and Shapira 1987; Straub and Welke 1998). In summary, the decision rationaleapplied by firms that adopt dual IS conservative- and IS innovator-like strategies is likely to be complex, applied with a low consistency, andlikely to focus on the assessment/mitigation of risks.

Proposition 3: The decision rationale used by firms that adopt dual IS conservative-like and IS innovator-like strategiesfor IS portfolio prioritization is likely to be difficult to communicate, applied with low consistency and focused on riskassessment/mitigation.

MIS Quarterly Vol. 39 No. 2—Appendices/June 2015 A1

Page 24: P INFORMATION SYSTEMS PORTFOLIO ... - Dr. Prasanna Karhade · Prasanna Karhade Department of Information Systems, Business Statistics and Operations Management, The Hong Kong University

Karhade et al./Patterns in IS Portfolio Prioritization

Appendix B

Identifying the IS Strategy

Semi-structured interviews with key informants within different business units at the research site revealed several recurring themes whichenabled us to determine their adopted IS strategies.

IS Conservative Classification: Interviews revealed that executives within one business unit at our research site place a high emphasis onadopting a safe and stable approach to running their business, as one executive noted, “[a] significant portion of our revenues were generatedprimarily based on a set of stable, proven, technologies.” The approach they adopt toward investment in IS tends to be very safe and stable. They largely invest in proven, safe IS and perceive IS as a vehicle to support their business operations in a stable manner. Informants alsorevealed that they are extremely conservative and risk averse when it comes to investments in IS. Given their conservative approach toinvesting in IS, this firm is not in any hurry to adopt new, emerging IS. They only invest in IS that is proven to be a stable, predictabletechnology in their industry. Informants also revealed an emphasis on intensive planning and a top–down formal structure for decision makingwithin their firm. Their conservativeness enables them to ensure that all of their investment is geared toward stable and proven IS thatsystematically improves the efficiency of their business operations. Given the prevalence of such strong themes, this firm was classified asan IS conservative. This research choice was unanimously validated for us by the informants at our research site.

IS Innovator Classification: Interviews revealed that executives within another business unit at our research site place a high emphasis onthe necessary experimentation for their continual growth and perceive IS as a vehicle for their experimentation/growth. Corroborating evidencefor this emphasis on continual growth was also obtained from the annual report:

[We] have been working feverishly to globalize this business….[A] significant fraction of our orders now come fromoutside the U.S.…[New] customers in Country A, B, C are now buying our products….We want to take advantage of anew market of $4 billion in global opportunities....[We] have effectively doubled the market for this great business.

Informants revealed that they are focused on experimentation and take risks when it comes to investments in IS. Given their innovativeapproach to investing in IS, this firm is more likely to quickly respond to signals from their external dynamic environment to adopt new,emerging IS. Informants revealed their reliance on a less formal, more bottom–up approach to decision making within their firm. Given theprevalence of such strong themes emphasizing experimentation with IS for growth, this firm was classified as an IS innovator. This researchchoice was also unanimously validated for us by the informants at our research site.

Dual IS Strategy Classification: Finally, interviews with executives at another business unit at our research site revealed the presence of boththe IS conservative- and IS innovator-like behavior, described above, in their philosophy in the use of IS. This firm was classified as havingdual IS strategies. (Theoretical development for this IS strategy is discussed in Appendix A; data are presented in the fourth section ofAppendix C; results are presented in Appendix D. Finally, we discuss the managerial implications of this IS strategy in the subsection,“Mediating Role of the Decision Rationale Profile,” of the main paper.)

Appendix C

Sample Selection Criteria, Characterizing Initiatives (Information Attributes)

Elimination of Initiatives

To account for various confounding factors, prior research has suggested focusing on only one kind of portfolio (e.g., Earl 1989; Sabherwaland Chan 2001). In accordance with this recommendation, we employed a rigorous selection process to retain only initiatives pertaining tobusiness applications of IS.

Regulatory compliance related proposals (e.g., Sarbanes-Oxley Act) were eliminated.

A2 MIS Quarterly Vol. 39 No. 2—Appendices/June 2015

Page 25: P INFORMATION SYSTEMS PORTFOLIO ... - Dr. Prasanna Karhade · Prasanna Karhade Department of Information Systems, Business Statistics and Operations Management, The Hong Kong University

Karhade et al./Patterns in IS Portfolio Prioritization

A key decision maker at the research site commented on this elimination step as follows:

Managers who really need funds to finish off their older initiatives will sometimes pitch “new” initiatives and say that theseare SOX initiatives …expecting us to readily agree…but SOX initiatives are very different and different forces guide thosechoices….There always is this dark side to planning effort and I am glad you excluded SOX initiatives from your analysis.

Such discussions with key decision makers at the research site provided validation for our sampling criteria.

Next, proposals strictly pertaining to IT infrastructure, identified for us by the decision makers, were also eliminated. In this research study,we intend of examine the influence of the strategic orientation of an organization on the decision rationale it uses for IT governance. At ourresearch site, the IT infrastructure portfolio was governed as an enterprise-wide, shared capability. The theoretical justification for governingIT infrastructure portfolio as an enterprise-wide centralized capability, at our research site, is discussed next.

As multibusiness organizations have to make a choice between granting autonomy to their separate businesses (or business units) and extractingsynergies across their businesses (or business units), prior work has framed this choice as a tradeoff (Weill and Ross 2004). In the context ofthe portfolio of IT investments, we find that organizations can enjoy the benefits of both of these paradigms if they leverage these differentmechanisms for different kinds of IT portfolios. At our research site, we found that the IT infrastructure portfolio was relatively standardizedand there were significant synergies to be extracted across various businesses on the IT infrastructure portfolio. From the IT governancestandpoint, given the high levels of standardization within the IT infrastructure, this IT infrastructure portfolio was managed as an enterprise-wide capability. With regard to these IT infrastructure components, a standardized approach was leveraged across the entire organization toderive synergies from this IT infrastructure portfolio.

IT infrastructure components are systematically different from business applications of IS in three ways. First, approvals for certain ITinfrastructure initiatives do not require comprehensive search (decision) processes (Boonstra 2003). For instance, typically, the decision touse Microsoft Office as the office productivity software does not require elaborate search processes. Second, IT infrastructure componentsare often used in this organization, across the entire enterprise, almost in a “plug-and-play” manner with very little or no customization. Business applications of IS, on the other hand, required systematic customization depending on the various different business needs acrossindividual businesses or business units within the organization. Third, IT infrastructure initiatives enjoy extensive scale advantages such thatthe licensing fees for these hardware/software components that are used in a plug-and-play/standardized manner across the entire enterprisesignificantly decrease with increasing scales/volumes. Typically, standardized vendor contracts kick in and large organizations can get betterdeals and services as they purchase multiple licenses in larger volumes. Centralization of such IT infrastructure capabilities has some strongadvantages in the form of significant cost savings and an enterprise-wide shared capability approach also leverages technology expertise acrossthe company while permitting large and cost-effective contracts with hardware/software vendors. Governing the IT infrastructure portfolioas an enterprise-wide, shared capability was especially considered better at this particular large organization as it operated in multiple countriesand continents and the senior management believed that standardization of the IT infrastructure would significantly reduce coordination costs(Weill and Ross 2004). Given these theoretical reasons, we excluded IT infrastructure initiatives from our consideration set and, in accordancewith prior research (e.g., Earl 1989; Sabherwal and Chan 2001), exclusively focused only on one portfolio pertaining to the set of proposalsthat systematically described business applications of IS.

Information Attributes

Characterizing Benefits

Our key informants indicated that although managers proposing new initiatives were required to richly characterize the benefits associated theirinitiatives, quantifying these benefits with a number was not a requirement. In other words, IT governance decisions did not depend on anumeric measure of benefits. Arguably, a rigorous quantification of benefits associated with initiatives (with a return-on-investment measure)would be a desirable decision-making aid. But often, arriving at such a numeric measure is extremely difficult given the bounded rationalityof the economic actors involved in planning (Simon 1955). Detailed discussions with the decision makers revealed numerous challengesassociated with quantifying the benefits associated with proposed initiatives. Discussions with the CIO, senior management, and members ofthe top management teams revealed that especially in the early planning stages, ROI metrics were not exclusively used as decision-makingcriteria.

These insights revealed that decisions on proposed initiatives are often made on a tacit level by relying on qualitative information on the typesof benefits proposals are designed to deliver. Based on our understanding of the pertinent literature (e.g., Broadbent et al. 1999; Sabherwaland Chan 2001), five kinds of benefits that initiatives could potentially offer were used to create five variables to comprehensively characterize

MIS Quarterly Vol. 39 No. 2—Appendices/June 2015 A3

Page 26: P INFORMATION SYSTEMS PORTFOLIO ... - Dr. Prasanna Karhade · Prasanna Karhade Department of Information Systems, Business Statistics and Operations Management, The Hong Kong University

Karhade et al./Patterns in IS Portfolio Prioritization

benefits associated with initiatives. The comprehensiveness of these five variables was validated for us by the key decision makers at theresearch site. Initiatives were designed such that they could offer multiple types of benefits and thus these five different kinds of benefitsassociated with proposals were not mutually exclusive.

Although all of the initiatives we examine in this study (annual planning cycle for investments spanning the 2006–2007 time period) were ofhigh substantive significance to the organization, not all these initiatives offered strategic benefits. Some of these initiatives were designedspecifically with efficiency improvements in mind. Based on several consultations with members of the CIO’s office and the metrics withinthis organization, such initiatives were classified as initiatives that offered only efficiency improvements and not as initiatives that offeredstrategic benefits. Such proposals were crafted very differently, highlighting the strategic benefits offered by their initiatives, and it was thusvery easy for us to identify such initiatives (in consultation with the members of the metrics groups at the research site) which offered strategicbenefits and efficiency improvements. Discussions with the members of the CIO’s office and the metrics group within this organization guidedus to systematically classify benefits and helped us validate these nuanced distinctions/classifications. Given the dark side to the planningeffort, senior executives feared that there would be a tendency for every manager to claim that all initiatives offered strategic benefits so asto warrant approval and funding. Senior management cautioned managers that all the initiatives claiming to offer strategic benefits couldwarrant additional scrutiny. Thus, we do not find evidence to suggest that managers were claiming benefits that were not genuinely built intothe design of their initiatives.

1. Efficiency improvements: Certain initiatives were designed so as to develop IS that replaced, digitized, or automated manual tasksor business processes (Camillus and Lederer 1985; Philip 2007). These initiatives, focused on exploitation (e.g., March 1991), helpedmanagers automate various business activities and thus operate with higher efficiency.

2. Marketing benefits: Certain initiatives were designed to develop IS that enabled businesses to create, promote, and better positionnew products/services. These initiatives helped managers to effectively market their products (Sabherwal and Chan 2001).

3. Strategic benefits: Certain initiatives were designed to develop IS which were deemed strategic. These initiatives helped managersto develop strategic capabilities to enable them to achieve some strategic advantages (Piccoli and Ives 2005).

4. Efficient interorganizational business processes: Certain initiatives were designed to develop IS that improved the efficiency ofinterorganizational business processes including critical supplier-, customer-facing processes (e.g., Kumar and van Dissel 1996).

5. Cycle Time Reductions: Certain initiatives were designed to develop IS that had the potential to offer business processimprovements, specifically aimed at reducing the business cycle implementation times associated with certain business processes(Broadbent et al. 1999).

Characterizing Risks

Risk assessment is critical for IT governance (Iversen et al. 2004; McFarlan and Nolan 2005). Based on the recommendations from prior work(Lyytinen et al. 1998), we adopt McFarlan’s (1981) approach for assessing the risk of proposed initiatives, resulting in these three measures.

6. Initiative Size: This attribute was measured based on the estimated investment required to implement the initiative. Risk associatedwith an initiative increases with its size (McFarlan 1981). This variable was assigned the following three values: low size(investment less than U.S. $100,000), medium size (investment greater than U.S. $100,000 but less than U.S. $1,000,000), and highsize (investment greater than U.S. $1,000,000). These anchors for the size measure were validated for their suitability based on inputsfrom the senior management and the CIO at the research site.

7. Initiative Structure: Some initiatives by their very definition are well-defined and have high clarity and certainty in terms of theirexpected inputs and outputs. Thus, the corresponding organizational tasks required to implement such initiatives are relativelystraightforward (Eisenhardt 1985). Initiatives whose inputs/outputs are vulnerable to change have low structure. Initiatives of highstructure are less risky when compared to initiatives of low structure (McFarlan 1981). This variable was assigned two values: highstructure (well-defined objectives for the initiative) and low structure (initiative with relatively fluid objectives). The vulnerabilityto change and the extent of clarity of the objectives, which separates low-structured, high-risk initiatives from high-structured, low-risk initiatives, was ascertained by the managers proposing initiatives and validated for us by the members of the senior managementand the CIO at the research site.

This variable was assigned only two values. The decision makers indicated that, from an IT governance standpoint, such ameasurement scheme was satisfactory as it helped them to identify and separate the high-structured, low-risk initiatives from the high-

A4 MIS Quarterly Vol. 39 No. 2—Appendices/June 2015

Page 27: P INFORMATION SYSTEMS PORTFOLIO ... - Dr. Prasanna Karhade · Prasanna Karhade Department of Information Systems, Business Statistics and Operations Management, The Hong Kong University

Karhade et al./Patterns in IS Portfolio Prioritization

risk, low-structured initiatives. Although in theory the degree of inherent structure in a project is a continuous variable, this use ofa two-category variable demonstrates satisficing behavior from the boundedly rational decision makers involved in IT governancetasks.

8. Prior Experience: As the familiarity of an organization with a technology increases, the likelihood of encountering technicalproblems reduces. The higher the prior experience with technologies used in the execution of initiatives, the lower the risk associatedwith such initiatives (McFarlan 1981). This variable was assigned three values: low (initiatives with new, emerging technologieswith low familiarity within the organization), medium (initiatives involving technologies when the familiarity with that technologywas neither high nor low), and high (initiatives involving standard technologies very familiar to the organization).

The design of this variable also provides evidence of the satisficing behavior of the decision makers. Values at the ends of thespectrum for this variable were very easy to identify. Managers had worked with certain mature technologies in the past anddelivered successful projects. Some technologies were nascent and emerging, and managers had not yet adopted these technologieswithin the organization. Identifying initiatives with high/low prior experience with technologies was relatively straightforward, andthus identifying the technologies for which the prior experience was neither high nor low was also easy. From a decision-makingstandpoint for IT governance, the design of this three category variable was deemed satisfactory.

Characterizing Risk Mitigation Mechanisms

Diverse kinds of risk mitigation mechanisms are critical for successful implementation of business initiatives that depend on IS (e.g., Iversenet al. 2004; Nolan and McFarlan 2005; Piccoli and Ives 2005; Sherer and Alter 2004). Prior research points to at least three kinds (e.g.,Baskerville 1993), including (1) internal risk mitigation mechanisms pertaining to software and technological capabilities (e.g., Baskerville1993), (2) process risk mitigation mechanisms pertaining to the management of software development processes and methodologies(Ramasubbu et al. 2008; Sherer and Alter 2004), and (3) external risk mitigation mechanisms concerning the business process redesignimplications of new initiatives (Broadbent et al. 1999). Variables on these categories of decision criteria,1 used for managing risks associatedwith these initiatives are described next.

Internal Risk Mitigation Mechanisms

9. Employ in-house software: Software applications developed in-house potentially embed organizational knowledge (e.g., Earl 1993;Mitchell and Zmud 2006; Saarinen and Vepsalainen 1994) and thus their use in the execution of proposed initiatives can be viewedas a risk mitigating factor. Since in-house software applications embed organizational process knowledge, this familiarity with thetechnological solution to an organizational process problem makes it relatively easy to redeploy this solution in the context of a newinitiative. This variable was assigned a value of 1 if a proposed initiative could leverage a software application developed in-houseor a value of 0 otherwise.

10. Internal Maturity: Managers who have managed certain initiatives in the past (e.g., an e-commerce website to solve some businessproblem in the past) are likely to develop mature plans2 for future initiatives. So although the development work for the subsequentinitiatives is not done, it is easy to see how the proposals for these subsequent initiatives are likely to be considered mature and thusperceived to be less risky (Boonstra 2003). Risks associated with an initiative decrease as the maturity associated with the proposedinitiative increases (Ramasubbu et al. 2008). Uncertainties associated with an initiative are often resolved by dedicating moreresources to develop the plan for a proposed initiative and advancing it further along the software development lifecycle (SDLC)maturity phases. In other words, a proposal or plan of an idea that is more developed and further along the SDLC maturity phases

1One decision criterion (a risk criterion) in most organizations during planning is that of resource availability. For our research site, given the large organizationsize and the availability of a sufficiently large pool of resources at their offshore implementation centers, we observed that the resource availability criterion wasnot a limiting factor.

2For instance, managers at a municipality introduced an Internet application facility for the payment of dog licenses. After the decision to approve the firstinitiative was taken, for subsequent initiatives that are arguably more mature as they were similar to the first initiative, decisions on the related initiatives followedrelatively easily, which led to a wide range of Internet-enabled facilities (Boonstra 2003).

MIS Quarterly Vol. 39 No. 2—Appendices/June 2015 A5

Page 28: P INFORMATION SYSTEMS PORTFOLIO ... - Dr. Prasanna Karhade · Prasanna Karhade Department of Information Systems, Business Statistics and Operations Management, The Hong Kong University

Karhade et al./Patterns in IS Portfolio Prioritization

is likely to be less risky.3 This variable has been assigned three values: low (proposed initiative in its early stages of conception),medium (requirements and goals associated with the initiative are clearly defined), and high (several different future contingencieshave been envisioned and controls have been systematically developed to manage those risks by crafting a complete, matureproposal). In other words, the proposals for some initiatives are more mature and less risky than others as managers proposing theseinitiatives can build on some similar initiatives they have implemented in the past, so the internal maturity of such proposals isrelatively high. These plans, given their high maturity, are likely to be perceived as being less risky.

Process Risk Mitigation Mechanisms

Exploiting potential business opportunities that critically depend on IS involves several organizational tasks in addition to just deploying thesoftware. Process capabilities are often deemed critical in delivering successful initiatives. The CIO and other members of the top managementteams we interviewed validated that the likelihood of success is critically dependent not only on the maturity of the initiative proposals but alsoon the teams (collaborations between the internal and external partners) assembled to manage and implement the initiatives. Potential businessopportunities that critically depend on IS often have a significant impact on the business processes of an organization. Such initiatives whichcritically depend on IS can either constrain or facilitate business process redesign (BPR) initiatives and vice versa (Broadbent et al. 1999). Managing the BPR implications of IT initiatives and vice versa is critical for successfully executing proposed initiatives.

11. Business process redesign completed: Before starting initiatives that critically depend on IS, envisioning process changes, andredesigning work flow activities, exerting effort and planning for such BPR tasks is critical to minimizing process risks (Broadbentet al. 1999). This variable was assigned a value of 1 when BPR planning tasks were completed and these controls were systematicallypresented in the proposal and a value of 0 when the BPR planning tasks were not described in the proposal.

12. Resources for process redesign committed: Identifying organizational resources and committing them for undertaking BPR planningtasks before starting initiatives can be a critical risk mitigation factor (Lambert 1986). For the successful delivery of these newbusiness initiatives that critically depend on IS, the early involvement of the right resources that systematically understand thebusiness process implications and ramifications of these initiatives is critical. This variable was assigned a value of 1 when resourceswere identified and assigned to proposed initiatives for conducting BPR tasks and a value of 0 otherwise.

External Risk Mitigation Mechanisms

13. Employ consultant knowledge: Specialized external consultants/partners can add value to large IT initiatives and integrating theseexternal sources of knowledge with internal expertise can mitigate risks. Consultants can offer expertise in specific areas and, inparticular, their exposure of several different organizational contexts on similar initiatives can be helpful in minimizing the likelihoodof project failure (Ko et al. 2005). For each initiative, this variable was assigned a value of 1 when managers identified and proposedleveraging capabilities from external consultants and a value of 0 otherwise. The identification of external partners who have workedon similar initiatives in the past is a nontrivial task and managers are required to exert significant effort to systematically identifysuch external partners.4 In the context of this large organization, proposals from managers that identified such pertinent, externalpartners that have worked on similar initiatives in the past were considered less risky.

14. Utilize specialized software applications: Organizations can potentially manage successful delivery of large initiatives by procuringspecialized software products. These partial solutions to specialized organizational problems can potentially expedite initiativeprogress and improve likelihood of success (McFarlan 1981). Given the large size of this organization, managers within all thebusiness units could exert significant bargaining power to attract very competitive contracts from multiple software vendors. Giventhe high bargaining power of this large organization, risks of vendor opportunism were relatively easy to mitigate. This identificationof vendors early on in the process of designing an initiative helped managers systematically manage risks and understand how thesebuilding blocks, in the form of specialized software applications, could be leveraged to expedite the development of their initiatives.

3It is critical to point out that we are referring to the maturity of the plan. It is not as if the work on the initiative has been done, but that experienced managersare often able to envision future contingencies (problems they might encounter during development/deployment of the initiative) and develop ex ante controlsfor those risks by crafting mature proposals.

4This mechanism relies on external partners who can introduce new risks by acting opportunistically. However, given that this organization is a large, Fortune50 enterprise, they had a history of developing extensive contracts with well-designed safeguards to mitigate the risks of opportunistic behavior on externalpartners. At the same time, this organization has also been able to craft contracts that incentivize their external partners to excel at their tasks.

A6 MIS Quarterly Vol. 39 No. 2—Appendices/June 2015

Page 29: P INFORMATION SYSTEMS PORTFOLIO ... - Dr. Prasanna Karhade · Prasanna Karhade Department of Information Systems, Business Statistics and Operations Management, The Hong Kong University

Karhade et al./Patterns in IS Portfolio Prioritization

This variable was assigned a value of 1 if the initiative proposed the procurement of specialized software and a value of 0 otherwise.

15. Leverage third-party solutions: Executives can also potentially manage successful delivery of large initiatives by leasing third partytechnologies (e.g., McFarlan 1981) solutions as building blocks. Third party applications model best practices and thus can expeditethe delivery of proposed initiatives, simultaneously improving likelihood of success. This variable was assigned a value of 1 if theproposed initiative recommended leveraging third party software applications and a value of 0 otherwise.

This risk mitigation mechanism is conceptually similar to the purchase of specialized software solutions from external vendors. In the eventof the purchase of licenses, these software components would need to be hosted within the organization and would necessitate additionalmaintenance effort over the duration of the initiative. The lease of third party applications would free the organization from the burden ofhosting software applications as a part of the internal IT infrastructure. Barring these minor differences, both of these risk mitigationmechanisms demonstrated that managers proposing new initiatives had exerted significant effort to identify vendors, craft preliminarycontract/service level agreements, and negotiate preliminary price quotes to systematically manage the risks associated with their initiatives. Both of these mechanisms depended on external vendors, but given the large size of this organization, the likelihood of opportunistic behavioron behalf of the vendors was deemed very low. In essence, both of these mechanisms were considered risk mitigating as they demonstratedthat managers proposing new initiatives, while managing the threat of opportunistic behavior on the part of the external vendors, had exertedsignificant effort to leverage partial solutions as building blocks to assuage the risks associated with their initiatives.

Measure definitions for these 15 attributes were consistently enforced across all proposal submissions before the commencement of planning. Portfolio data analyzed in our study was subjected to at least two rounds of validation before decision making started.

Summarized Views of Portfolio Data

IS Conservative’s Portfolio (Total Number of Initiatives = 72)

Risk Attributes Benefit Attributes

MIS Quarterly Vol. 39 No. 2—Appendices/June 2015 A7

Page 30: P INFORMATION SYSTEMS PORTFOLIO ... - Dr. Prasanna Karhade · Prasanna Karhade Department of Information Systems, Business Statistics and Operations Management, The Hong Kong University

Karhade et al./Patterns in IS Portfolio Prioritization

IS Conservative’s Portfolio: Risk Mitigation Attributes (Total Number of Initiatives = 72)

Internal Risk MitigationMechanisms

External Risk MitigationMechanisms

Process Risks MitigationMechanisms

IS Innovator’s Portfolio (Total Number of Initiatives = 32)

Risk Attributes Benefit Attributes

A8 MIS Quarterly Vol. 39 No. 2—Appendices/June 2015

Page 31: P INFORMATION SYSTEMS PORTFOLIO ... - Dr. Prasanna Karhade · Prasanna Karhade Department of Information Systems, Business Statistics and Operations Management, The Hong Kong University

Karhade et al./Patterns in IS Portfolio Prioritization

IS Innovator’s Portfolio: Risk Mitigation Attributes (Total Number of Initiatives = 32)

Internal Risk MitigationMechanisms

External Risk MitigationMechanisms

Process Risks MitigationMechanisms

Portfolio of a Business Unit with a Dual IS Conservative- and IS Innovator-Like Strategy(Total Number of Initiatives = 57)

Risk Attributes Benefit Attributes

MIS Quarterly Vol. 39 No. 2—Appendices/June 2015 A9

Page 32: P INFORMATION SYSTEMS PORTFOLIO ... - Dr. Prasanna Karhade · Prasanna Karhade Department of Information Systems, Business Statistics and Operations Management, The Hong Kong University

Karhade et al./Patterns in IS Portfolio Prioritization

Portfolio of a Business Unit with a Dual IS Conservative- and IS Innovator-Like Strategy(Total Number of Initiatives = 57)

Internal Risk MitigationMechanisms

External Risk MitigationMechanisms

Process Risk MitigationMechanisms

A10 MIS Quarterly Vol. 39 No. 2—Appendices/June 2015

Page 33: P INFORMATION SYSTEMS PORTFOLIO ... - Dr. Prasanna Karhade · Prasanna Karhade Department of Information Systems, Business Statistics and Operations Management, The Hong Kong University

Karhade et al./Patterns in IS Portfolio Prioritization

Appendix D

Decision Tree for Business Unit that Adopted Dual IS Conservative-and IS Innovative-Like Strategies

References

Baskerville, R. 1993. “Information Systems Security Design Methods: Implications for Information Systems Development,” ACM ComputingSurveys (25:4), pp. 375-414.

Boonstra A. 2003. “Structure and Analysis of IS Decision-Making Processes,” European Journal of Information Systems (12:3), pp. 195-209.Boynton, A. C., and Zmud, R. W. 1987. “Information Technology Planning in the 1990’s: Directions for Practice and Research” MIS

Quarterly (11:1), pp. 59-71.Broadbent, M., Weill, P., Clair, D. S., and Kearney, A. T. 1999. “The Implications of Information Technology Infrastructure for Business

Process Redesign,” MIS Quarterly (23:2), pp. 159-182.Camillus, J. C., and Lederer, A. L. 1985. “Corporate Strategy and the Design of Computerized Information Systems,” Sloan Management

Review (26), pp. 35-42.Earl, M. J. 1989. Management Strategies for Information Technology, Englewood Cliffs, NJ: Prentice-Hall.Eisenhardt, K. M. 1985. “Control: Organizational and Economic Approaches,” Management Science (31:2), pp. 134-149.Iversen, J. H., Mathiassen, L., and Nielsen, P. A. 2004. “Managing Risk in Software Process Improvement: An Action Research Approach,”

MIS Quarterly (28:3), pp. 395-433.

MIS Quarterly Vol. 39 No. 2—Appendices/June 2015 A11

Page 34: P INFORMATION SYSTEMS PORTFOLIO ... - Dr. Prasanna Karhade · Prasanna Karhade Department of Information Systems, Business Statistics and Operations Management, The Hong Kong University

Karhade et al./Patterns in IS Portfolio Prioritization

Ko, K., Kirsch, L. J., and King, W. R. 2005. “Antecedents of Knowledge Transfer from Consultants to Clients in Enterprise SystemImplementations,” MIS Quarterly (29:1), pp. 59-85.

Kumar, K., and van Dissel, H. G. 1996. “Sustainable Collaboration: Managing Conflict and Cooperation in Interorganizational Systems,”MIS Quarterly (20:3), pp. 279-300.

Lambert, R. A. 1986. “Executive Effort and Selection of Risky Projects,” RAND Journal of Economics (17:1), pp. 77-88.Lyytinen, K., Mathiassen, L., and Ropponen, J. 1998. “Attention Shaping and Software Risk: A Categorical Analysis of Four Classic

Approaches,” Information Systems Research (9:3), pp. 233-255.March, J. G. 1991. “Exploration and Exploitation in Organizational Learning,” Organization Science (2:1), pp. 71-87.March, J. G., and Shapira, Z. 1987. “Managerial Perspectives on Risk and Risk Taking,” Management Science (33:11), pp. 1404-1418.McFarlan, F. W. 1981. “Portfolio Approach to Information Systems,” Harvard Business Review (59:5), pp. 142-150.Mitchell, V. L., and Zmud, R. W. 2006. “Endogenous Adaptation: The Effects of Technology Position and Planning Mode on IT-Enabled

Change,” Decision Sciences (37:3), pp. 325-355.Nolan, R., and McFarlan, F. W. 2005. “Information Technology and the Board of Directors,” Harvard Business Review (83:10), pp. 96-106.Philip, G. 2007. “IS Strategic Planning for Operational Efficiency,” Information Systems Management (24:3), pp. 247-264.Piccoli, G., and Ives, B. 2005. “IT-Dependent Strategic Initiatives and Sustained Competitive Advantage: A Review and Synthesis of the

Literature,” MIS Quarterly (29:4), pp. 747-776.Ramasubbu, N., Mithas, S., Krishnan, M., and Kemerer, C. 2008. “Work Dispersion, Process-Based Learning, and Offshore Software

Development Performance,” MIS Quarterly (32:2), pp. 437-458.Saarinen, T., and Vepsäläinen, A. P. J. 1994. “Procurement Strategies for Information Systems,” Journal of Management Information Systems

(11:2), pp. 197-208.Sabherwal, R., and Chan, Y. E. 2001.”Alignment between Business and IS Strategies: A Study of Prospectors, Analyzers, and Defenders,”

Information Systems Research (12:1), pp. 11-33.Sherer, S. A., and Alter, S. 2004. “Information Systems Risks and Risk Factors: Are They Mostly About Information Systems?,” Communi-

cations of the Association for Information Systems (14:1), Article 2.Simon, H. A. 1955. “A Behavioral Model of Rational Choice,” The Quarterly Journal of Economics (69:1), pp. 99-118.Straub, D. W., and Welke, R. J. 1998. “Coping with Systems Risk: Security Planning Models for Management Decision Making,” MIS

Quarterly (22:4), pp. 441-469. Weill, P., and Ross, J. W. 2004. IS Governance: How Top Performers Manage IS Decision Rights for Superior Results, Boston: Harvard

Business School Press.

A12 MIS Quarterly Vol. 39 No. 2—Appendices/June 2015

Page 35: P INFORMATION SYSTEMS PORTFOLIO ... - Dr. Prasanna Karhade · Prasanna Karhade Department of Information Systems, Business Statistics and Operations Management, The Hong Kong University

Copyright of MIS Quarterly is the property of MIS Quarterly and its content may not becopied or emailed to multiple sites or posted to a listserv without the copyright holder'sexpress written permission. However, users may print, download, or email articles forindividual use.