World Development Vol. 74, pp. 197–208, 20150305-750X/� 2015 Published by Elsevier Ltd.

www.elsevier.com/locate/worlddevhttp://dx.doi.org/10.1016/j.worlddev.2015.04.017

Explaining Positive Deviance in Public Sector Reforms in Development

MATT ANDREWS*

Harvard Kennedy School, Cambridge, USA

Summary. — Public sector reforms are commonplace in developing countries. Much of the literature about these reforms reflects ontheir failures. This paper asks about the successes and investigates which of two competing theories best explain why some reformsare positive deviants: “solution- and leader-driven change” (SLDC) and “problem-driven iterative adaptation” (PDIA). The theoriesare used to analyze data emerging from a case survey involving thirty cases from Princeton University’s Innovations for SuccessfulSociety (ISS) program. The bulk of evidence from this study supports a PDIA explanation, but there is reason to believe that SLDChypotheses also have value.� 2015 Published by Elsevier Ltd.

Key words — development, leadership, reform, governance, innovation, case survey

1. INTRODUCTION

Final revision accepted: April 26, 2015

Public sector reforms are commonplace in developingcountries. Much of the literature about these reforms reflectson the failures (or limited successes) of the interventions,and continued weaknesses of governments after reforminterventions are complete (Pritchett, Woolcock, &Andrews, 2013). This paper asks about the successes;instances where reforms have led to more effective solutionsto public sector problems than is normal. One might callsuch successes the “positive deviants” of public sectorreform. Building on the positive deviance approach tounderstanding and facilitating change, the paper aims toexplain the strategies that are associated with these abnor-mally successful interventions (Pascale, Sternin, & Sternin,2010). The goal is not to provide a final test of any one the-ory of effective reform, but rather to offer a systematic,evidence-based analysis that helps to better construct suchtheory.

The first part of the paper discusses past experience withreforms and the fact that successes (instances where reformslead to more functional governments that solve problems)are the exception, not the norm. It refers to such exceptionsas “positive deviants” and explains the underlying rationalebehind the positive deviance approach. This gist of thisapproach holds that understanding the strategies leading topositive deviance can help inform more general success. Build-ing on past work (Andrews, Pritchett, & Woolcock, 2013;Bond & Hulme, 1999), the discussion then proposes compet-ing theories to explain why some reforms are positive deviants.These theories are called “solution- and leader-driven change”(SLDC) and “problem-driven iterative adaptation” (PDIA)(Andrews, 2013a, 2013b):� SLDC proposes that abnormal success results whenreforms are introduced through a disciplined, formal pro-ject process: solutions are fully identified up-front and arethe focus of change; the reform is fully planned out at thestart and implemented as planned; a champion drives theprocess; and a pure-form best practice solution is produced.� PDIA suggests that abnormal success results whenreforms are introduced through an iterative process morereflective of “muddling through”: change is motivated bya problem, not a solution; the reform content emergesthrough a process of experimentation and trial and error;

197

with multiple agents playing different leadership roles; pro-ducing a mixed-form hybrid that is fitted to the peculiarcontext.

The second section reports on an empirical study intendedto shed light on which theory (or parts of the two theories)best explains positive deviance in public sector reforms indevelopment. The study employs a case survey method to syn-thesize evidence in 30 case studies published by PrincetonUniversity’s Innovations in Successful Societies (ISS) pro-gram. The research method and case sample are introducedand the survey results are discussed. The discussion shows thatthe bulk of evidence supports a PDIA explanation, but there isreason to believe that SLDC hypotheses also have value. Itseems that PDIA and SLDC are two viable paths throughwhich positive deviance can emerge, but PDIA is a far widerpath that accommodates and fosters positive deviance morereadily. The conclusion identifies some limits of this study,but also suggests how the approach taken in this paper canbe built upon to promote a better understanding and theoryof why some reforms succeed when most fail—and even toinform reform strategies in future.

2. LEARNING FROM POSITIVE DEVIANCE IN PUBLICSECTOR REFORM IN DEVELOPMENT

Public sector reform has emerged as a staple of develop-ment. Such reform is typically introduced through projectssupported by development agencies. These agencies have seena steadily increasing portfolio of projects since the 1970s. Forexample, total lending volumes for such interventions in theWorld Bank grew from an average annual inflation-adjustedtotal of $1.8 billion during the 1990s to $2.7 billion in the2000s (World Bank, 2012, p. 2). Growth in the number ofWorld Bank projects with public sector reform content hasbeen significant (going from 469 in the 1980s to 3,235 in the2000s (Andrews, 2013a; Moloney, 2009)). Similar patternsshow that these reforms dominate project portfolios in otherdevelopment organizations as well. Public sector reforms areembedded in over half of the operations carried out byBritain’s Department for International Development during

198 WORLD DEVELOPMENT

2004–10. 1 They are also evident in over half of the Asian andAfrican Development Banks’ project portfolios in the late2000s, 2 having comprised less than 10% of interventions priorto the 1990s . 3

The pervasive nature of these reforms is further evidenced inthe variety of affected countries. World Bank projects support-ing these reforms can be identified in over 140 countries(Andrews, 2013a). Similar coverage is provided by agencieslike the United States Agency for International Development(USAID) and other bilateral entities, the International Mone-tary Fund (IMF), and regional development banks. Countriespromote reform agendas apart from these external influencestoo, which further clutters the public sector change discourse.

(a) A ‘norm’ of failure (or limited success)

Mounting evidence shows that these reforms commonlyproduce poor results—either failing to achieve objectives atall or generating changes in forms (like laws and systems)but not having a positive impact on practice or leading tothe resolution of governance problems. Using measures fromthe World Bank Country Policy and Institutional Assessment(CPIA) indicators, for instance, a 2008 World Bank evaluationshowed that many countries failed to improve aspects of gov-ernment quality even after reform projects were completed.The proportion of countries seeing post reform declines orstagnation on these measures ranged from 40% to 60%: forexample, over half of the countries stayed the same or wentbackwards on indicators of “quality of public administration”(World Bank, 2008, p. 46 4).

The best performing area in the 2008 evaluation was publicfinancial management, where about 60% of countries improvedtheir scores. Unfortunately, very few of these improving coun-tries achieved levels on the indicators that suggest they areactually establishing functional systems (Andrews, 2011,

2013a; de Renzio, Andrews, & Mills, 2010). 5 This means thatmany of the countries have produced better laws and processesthrough reforms, but they still commonly struggle with imple-menting and using the new laws and processes. As a result,problems fester: money still flows slowly after reforms in mostcountries, actual spending does not reflect plans or budgets,leakage is high, and resources fail to produce results.

The data used to make these assessments are always open tochallenge, but a variety of other studies show similar things.For instance, a 2011 study found even more disappointment(World Bank, 2011, pp. 68–76). Fewer than 40% of the 80countries receiving World Bank support for public sectorreform during 2007–09 registered improved CPIA governancescores in that period. A quarter of these countries actually sawsuch scores decline, while more than a third stayed the same.The quality of public administration was higher after reformsin only 13% of reforming countries, dropping in about thesame-sized group. Andrews (2013a) shows that the sameobservations can be made using multiple data sources, andthat these observations resonate with stories emerging fromcase-study analysis (and case survey work). These case-basedstudies indicate that there are mixed and disappointing resultsin a range of other institutional reform areas, including priva-tization (Boubakri, Cosset, & Guedhami, 2009), deregulation(Busenitz, Gomez, & Spencer, 2000), public financial manage-ment (Andrews, 2010; de Renzio et al., 2010), health systemmodernization (World Bank, 2009), and financial liberaliza-tion (Karikari, 2010; Obtsfeld, 2009). The studies illustrate,for instance, that many countries do not have more efficientservice delivery or better trade volumes or more stable finan-cial sectors after they privatize industries, introduce new trade

regulations, and liberalize financial systems. The emergingstory across such studies is loud and clear and is even acceptedby donor organizations (World Bank, 2012): in most cases pub-lic sector reforms do not lead to more functional governments.

(b) The existence of positive deviants

However, research also points to the existence of more suc-cessful reform experiences that do lead to more functionalgovernments. In these instances, reforms facilitate the estab-lishment of governments that solve problems and achieve thekind of functionality needed to produce public value; newpublic financial management systems actually foster betterresource use, administrative reforms foster better service deliv-ery, trade reforms generate higher volumes of trade, and soforth. These experiences could be called positive outliers; giventhat they produce results that are better than the norm. “Pos-itive deviance” is another term that describes such experiences.The term has been used in various literatures but entered thedevelopment domain because of the work of Pascale et al.(2010).

These authors argue that positive deviance is observable inevery community or field, where some agents find better solu-tions to problems than their peers even though they have sim-ilar resources as their peers and face similar challenges andobstacles. Given such belief, the positive deviance approachhas emerged as a way of identifying workable solutions todevelopment’s toughest problems. It emphasizes the impor-tance of learning from the positive deviants within the contextswhere failure is more normal; and focuses especially on learn-ing about the strategies adopted to find and fit effective solu-tions.

The importance of this kind of learning cannot be over-stated in the international development domain. This impor-tance is reflected in a number of studies that have tried topromote such learning in the past decades. Many of thesestudies try to explain “pockets of productivity” or “islandsof excellence” in government organizations in developingcountries (Leonard, 2010). These include studies like Grindleand Thomas (1991), Leonard (1991), Schneider (1991),Grindle (1997), Tendler (1997), Uphoff, Esman, and Krishna(1998), Heredia and Schneider (2002), Grindle (2004), Joshand Moore (2004), Owusu (2006), and Bebbington andMcCourt (2007). These studies actually investigate differentmanifestations of what is being called positive deviance in thispaper (Leonard, 2010). Some focus on oddly successful orga-nizations, others on successful policy interventions, and yetothers on successful reforms themselves. In most cases the suc-cesses one sees emerged from some or other change process,however, so it is appropriate in all cases to ask how suchchange (or reform) came about and was consolidated to fostermore effective government (where Leonard sees success as theimprovement in state capability to sustainably generate publicgoods).

Recent publications have built on this vibrant (but relativelysmall) set of studies. For instance, Andrews (2013a) uses ablend of case study and survey methods to identify the strate-gies common to more successful institutional reforms in a vari-ety of arenas. Similarly, the recent World Bank evaluation ofpublic sector reform (World Bank, 2008) looks at project suc-cesses across public sector reform types and countries to gleanlessons about “what works and why”. In the same vein,Rodrik (2003) develops narratives of the few (but impressive)high-growth experiences in the past 50 years. The 2008 SpenceGrowth Commission also tried to identify commonalities inthe strategies that led to high growth in countries like South

EXPLAINING POSITIVE DEVIANCE IN PUBLIC SECTOR REFORMS IN DEVELOPMENT 199

Korea, Singapore, and Turkey (Brady & Spence, 2010). All ofthese studies implicitly support the view that positive deviantsexist in the field of development (and reform) and that study-ing such deviants can foster a better understanding of develop-ment and change.

(c) Explaining positive deviance: competing theories of“successful” public sector reform

This is not to say that the studies have converged on simpleexplanations or even on the same explanations for positivedeviance in reforms. In fact, there are many different perspec-tives on why some public sector reforms seem to be more suc-cessful than others (and why some public organizationsproduce more public value after reforms than others). To illus-trate this, Leonard (2010) summarizes the literature on “pock-ets of productivity”’ to show what he calls “five implicitmeta-hypotheses.” These relate to context, organizationalleadership and structure, process, and the nature of the taskbeing undertaken. Leonard breaks the meta-hypotheses downinto over 50 stand-alone hypotheses that reflect how hard it isto tell any uncomplicated and coherent story.

In the interests of providing just such a story, however, thecurrent paper pulls together a selection of hypotheses aboutthe process of reform to generate competing theories of “suc-cessful” public sector reform. The hypotheses emerge particu-larly from the recent World Bank and Growth Commissionreports (Brady & Spence, 2010; World Bank, 2008) and thework of Andrews and others (Andrews, 2013a; Andrewset al., 2013; Pritchett et al., 2013) and earlier studies on thetopic of “process” theory (Bond & Hulme, 1999). Thehypotheses are responses to basic questions: What motivatesreform? How do reforms get implemented? Who leads the pro-cess? What do the “new” government and governance struc-tures look like? A first theory combining such answers iscalled solution- and leader-driven change (SLDC) and a sec-ond is called problem-driven iterative adaptation (PDIA). Inmany ways these approaches are like the “blueprint” and“process” approaches proposed by earlier work (like Bond &Hulme, 1999).

SLDC proposes that abnormal success results from reformsthat are introduced through a disciplined, formal project pro-cess: solutions are fully identified up-front and are the focus ofchange; the reform is fully planned out at the start and imple-mented as planned; a champion drives the process; and apure-form best practice solution is produced.� The emphasis on clearly identified solutions as a motiva-tion for change reflects the Growth Commission view thatsuccessful reform requires an initial decision to choose the“correct model” (Brady & Spence, 2010, p. 6).“Solutions” are provided across the development spectrum(often in indicator sets 6 but also in reform prescriptions(Andrews, 2005)) and are argued to guide and motivatereforms (inspiring change because there is a “solution” toadopt and a promise of gains). Such thinking yields a sim-ple hypothesis and expectation of what one would observeat the start of a reform process: H1. Successful reform ismotivated by the promise of a solution. One expects reformsolutions to be fully identified at the more successful reform’sstart and that reforms will be motivated by such solution, andfocused on implementing the solution.� The focus on pre-planning and implementing reforms asplanned reflects the idea that thorough project specificationfosters project success. This idea is implicit in the way orga-nizations like the World Bank stress project preparationand highly detailed ex ante programing. The 2008 World

Bank public sector reform evaluation states that this kindof pre-planning only works if the project is actually imple-mented according to plan, however, and suggests that moresuccessful reforms have this characteristic (World Bank,2008). This view is echoed in the Growth Commission’sobservations that growth-enhancing reform requires havingclear objectives and an authorizing environment in whichsuch objectives can be systematically implemented. 7

These observations and views yield the following hypothe-sis: H2. Successful reforms will be implemented accordingto a specified plan of action. One would expect to find thatthe reform content of these successful reforms is fully plannedout at the start of the change process, and that the reformsare implemented as planned.� It is commonly held that this kind of reform is successfulwhen driven by high-level individuals in positions ofauthority and influence. The perspective is reflected in apractical emphasis on reform “champions” in organizationslike the World Bank (where a loan can only be accessed if asenior official signs off, like the Minister of Finance). It isalso explicit in the Growth Commission’s work, whichnotes that such leaders are central in most of the instanceswhere governments have fostered high levels of growth oversustained periods (Brady & Spence, 2010). According to thecommission, such individuals facilitate these achievementswhen they make decisions that introduce the right policiesand then ensure that the policies get implemented. 8 Theargument is also that these strong leaders manage politicalstruggles around public sector change. Such change is typ-ically challenged from multiple quarters and some degree ofstrong central leadership is considered vital to build sup-porting coalitions for reform among narrow elites. Thereare many reasons why individuals would emerge to providesuch political leadership, and foster change, as discussed invarious recent texts (for instance, Bueno de Mesquita &Smith, 2012). The motivations that lie behind this behaviorare very important to understand and consider. One candraw a basic hypothesis from such thought without gettinginto the details of different explanations of these motiva-tions, however: H3. Successful reform is led by a champion;some high-level individual with authority. As a result, onewould expect to find a single individual in a high-level positionidentified consistently as the “leader” of the more successfulreform.� Given a belief in “correct” models in many parts ofdevelopment (as echoed in Brady and Spence (2010)) onehypothesis is that that positive deviants are reformers thatactually manage to adopt the best practice solutions theyfocus on replicating. This line of argument is implicit inthe way development organizations routinely evaluatereform progress as compliance with “good” or “best” prac-tice scripts. The implied argument is that adopting suchscripts will lead to functionality and success, which framesa basic hypothesis: H4. Successful reforms produce the“right” best practice solutions commonly argued as neces-sary to foster good governance. One expects to find thatfinal successful reform products will resemble pure-form bestpractice solutions identified at the start of reform.

These four arguments are combined in the first column ofTable 1, which suggests what a solution- and leader-drivenchange (SLDC) theory looks like. It is a simplified theory thatmay even be criticized as a straw man representation of theimplicit logic behind structured project interventions. This isarguably not an accurate perspective, however, as others referto similar constructs. Bond and Hulme (1999, p. 1340), forinstance, speak of the “blueprint” approach which is described

Table 1. Eight hypotheses, and two theories, to explain positive deviance in public sector reforms

Key question Solution and leader driven change (SLDC) Problem driven iterative adaptation (PDIA)

A. What motivates reform? H1. Successful reform is motivated by the promise of asolutionOne expects reform solutions to be fully identified at the

reform’s start and that reforms will focus on

implementing solutions

H5. Successful reform is motivated by a problem(performance deficiency).One expects that a problem will be clearly identified at

the start of reform, promoting an appetite for change

and a process of change

B. How do reforms getimplemented?

H2. Successful reform is implemented according to aspecified plan of actionOne expects to find that reform content is fully planned

out at the start of the change process, and is

implemented as planned

H6. Successful reform emerges through a process ofexperimentation as agents find and fit content tocontextOne expects to find reform content emerging through a

process of experimentation and trial and error

C. Who leads the reformprocess?

H3. Successful reform is led by a champion; somehigh-level individual with authorityOne expects to find a single individual in a high-level

position identified consistently as “leader” of the reform

H7. Successful reform is led by a group of agents whoprovide various functions required to make reformworkOne expects to find multiple individuals identified as

playing key functional roles in the reform process

D. What do the “new”

government/ governancestructures look like?

H4. Successful reforms produce the “right” bestpractice solutions commonly argued as necessary tofoster good governanceOne expects final reform products to resemble pure-

form best practice solutions identified at reform’s start

H8. Successful reforms produce hybrid solutions thatblend ideas from inside and outside the context; allfitted to the contextOne expects to find that final reform products are

mixed-form products of many influences; including best

practices and internal ideas

200 WORLD DEVELOPMENT

as follows: “Prescribed steps lead through the stages of theproject cycle; expert design and control activities; detailedplanning at the beginning specifies objectives, targets, outputs,resources, and schedules; local institutions are bypassed if theyhave inadequate capacity and the job of management is toimplement as closely as possible to plan.”

The four SLDC hypotheses are thus argued to representestablished and entrenched views on what makes for effectivereform in development (and reflect actual behaviors in thedevelopment community—especially where such is dominatedby reductionist economists, engineers, or infrastructureexperts who believe in the value of rational and disciplinedprocess and the idea that there are “answers” to development’sproblems). These four hypotheses are contrasted with ideasproposed to combine into an alternative theory explainingpositive deviance in reform—problem-driven iterative adapta-tion (PDIA).

PDIA suggests that abnormal success results when reformsare introduced through an iterative process more reflectiveof “muddling through” (Lindblom, 1959). As a theory, itrelates to many other studies and arguments (as discussed inAndrews et al., 2013) and the participatory work ofChambers (1997), Korten (1980) and Uphoff (1985) as wellas the managerial work of authors like Rondinelli (1993).Given such roots, PDIA proposes that successful change isusually motivated by a problem, not a solution; the reformcontent emerges through a process of experimentation andtrial and error; with multiple agents playing different leader-ship roles; producing a mixed-form hybrid that is fitted tothe peculiar context.� The first argument is that institutional reforms are moresuccessful when motivated by problems that local peoplecare about. The idea is that problems create an urgencyneeded to foster change, an awareness of potential loss thatoutweighs the expected loss associated with change andreform (Cameron, 1986; Seo & Creed, 2002; Vis & vanKersbergen, 2007). A problem focus also points to theweaknesses of extant structures, guiding reformers in howto de-institutionalize such (Greenwood, Suddaby, &Hinings, 2002). A problem focus further motivates a search

for viable alternatives to replace faulty incumbents; contex-tualizing the search to ensure that “solutions” fit contextualrealities. Such thinking yields a hypothesis and expectationof what one would observe at the start of a more successfulreform process: H5. Successful reform is motivated by aproblem (or performance deficiency). One expects that aproblem will be clearly identified at the start of reform, pro-moting an appetite for change and a problem-driven processof change.� The approach also emphasizes “purposive muddling” inthe reform process (Andrews, 2013a). This proposes thatreforms are incremental, emerging via multiple stepsthrough which reformers learn about what works andwhy, build capacities to introduce new mechanisms, andachieve short-term results to grow support for change(McCay, 2002). This “finding and fitting” involves a local-ized focus on problems and contextual realities, such thatreform choices address matters that are politically relevantand viable given capacities and constraints (Rose, 2003).Solutions are expected to emerge through processes of dis-cussion, translation, and experimentation (Dorado, 2005;Garud & Karnøe, 2003; Ostrom, 2008). This perspectiveyields the hypothesis: H6. Successful reform emergesthrough a process of experimentation as agents find andfit content to context. One expects to find reform contentemerging through a process of experimentation and trialand error.� A third piece of this perspective is that institutionalreform requires the engagement of a broad set of agents(Andrews, 2008; Andrews, McConnell, & Wescott, 2010).Individual champions are not enough, and even smallgroups of centrally located actors are insufficient.Multiple functions are needed to foster an awareness ofthe need for change, introduce new ideas to drive reform,motivate and authorize adjustment, and more (Hackman& Walton, 1986). These functions typically come from dif-ferent agents at different positions in reform networks(Greenwood & Suddaby, 2006). Given this, reforms arelikely to emerge and diffuse successfully only where diversesets of players are mobilized, including elites, peripheral

EXPLAINING POSITIVE DEVIANCE IN PUBLIC SECTOR REFORMS IN DEVELOPMENT 201

agents, and distributed implementers. The hypothesisemerging from such thought is simple: H7. Successfulreform is led by a group of agents who provide variousfunctions required to make reform work. One expects tofind multiple individuals identified as playing key functionalroles in the reform process.� This approach emphasizes finding and fitting solutions tocontext, and notes that processes succeed when allowingsolutions to emerge through a process of bricolage, ashybrid blends of different attributes—with some externalcontent and some internal content—rather than pure formreplicas of any one idea (Dacin, Goodstein & Scott, 2002;Ostrom, 2008). Ostrom (2008, p. 47) summarizes such view;“There must be the generation of new alternatives, [and]selection among new and old combinations of attributesthat are successful in a particular environment.” The argu-ment is that “new alternative” hybrids will lead to function-ality and success, which informs a basic hypothesis: H8.Successful reforms produce hybrid solutions that blendideas from inside and outside the context and are fitted tothe context. One expects to find that final reform productsare mixed-form products of many influences; including bestpractices and internal ideas.

3. EXAMINING POSITIVE DEVIANCE: A CASE SUR-VEY STUDY

The four PDIA hypotheses are organized into a single the-ory in Table 1, alongside the column showing the SLDChypotheses. As presented, SLDC and PDIA are intended toprovide competing explanations for how public sector reformsin development achieve success. In respect of this paper, thequestion is whether one of the two theories best explains theemergence of positive deviants—cases of abnormal success.It is difficult to test this, however, partly because the bulk ofempirical evidence about public sector reforms in develop-ment—both failures and success—comes in the form of casestudies. These studies provide detailed renditions of experi-ences in particular reform engagements. They do not alloweasy generalized testing of theories, however, or even garner-ing of lessons across cases. The case survey method hasemerged to fill such gap. It was devised as a means of general-izing from case studies using a closed–ended questionnaire(Lucas, 1974; Yin & Heald, 1975) to facilitate the analysis of“original qualitative case studies in a rigorous, highly struc-tured . . . way” (Newig & Fritsch, 2009, p. 5). The current sec-tion discusses an application of this method designed to fostera better understanding of positive deviance in public sectorreform in development.

(a) Research method and sample

The case survey method is a form of meta-analysis. Itrequires one to select a sample of relevant cases, develop acoding scheme to convert qualitative descriptions into quanti-fied variables, code each case (using multiple coders), and thenanalyze the coded data (Bullock & Tubbs, 1987; Larsson,1993; Larsson & Lubatkin, 2001). It has been used acrossthe social sciences, including in the literature on public sectorreform. Probably the most prominent application in this liter-ature was by Wolf (1993), who conducted a case survey ofbureaucratic effectiveness in US cabinet agencies. The studyused coded data to test competing theories of effectivenessacross various agencies, which is a similar approach to thattaken in this paper.

The sample used in the current paper emerged from a pur-posive sampling process focused on selecting cases of publicsector reform in developing nations that: (i) are consideredabnormally successful (because they fostered functionalimprovements in the way governments work), (ii) have enoughinformation to allow analysis, and (iii) cover a variety of pub-lic sector reform types. In this respect, 30 studies were selectedfrom Princeton University’s repository of Innovations for Suc-cessful Societies (ISS) cases. These cases were chosen by ISSstaff because they come from countries “once considered frag-ile states” and have generated “sustained improvement ininstitutional performance and economic growth.” 9 Givensuch selection process, the ISS reforms were expected to beabnormally successful (given their contexts) in fostering func-tional improvements in public organizations. Forty cases wereidentified initially, after discussions with ISS staff (intended toensure that all cases meant the criteria listed above). Twograduate students were then asked to rate each case and checkwhether they met the criteria (with questions asked about thedegree and nature of “success” 10 and the information densityin the cases). In this process, the students were tasked withensuring that the cases did not inflate success but rather pro-vided real stories with specific evidence of functional improve-ments in the governments involved (given potential biastoward inflating the “success” in each story). 11

The sample size was decreased to 30 cases after this process.The selected cases were ones in which both raters agreed thatthe story exhibited important “success” (functional improve-ment was evident) and was told in sufficient detail to allowanalysis. The final list is shown in Annex 1 and included exam-ples of reform in revenue agencies, local governments, nationallevel policymaking, anti-corruption reform, civil service sys-tems, and beyond.

The obvious limit to this sampling approach is that there areno cases of failure or even questionable success (at last given theselection criteria). The lack of “counterfactual” cases meansthat the study here cannot be seen as an explicit “test” of whatexplains positive deviance. Rather, it is a step in determining atheory to understand such. Another limit relates to the potentialsampling bias inherent in drawing all cases from one publicationsource, where one expects a common method of identifyingcases, doing research, and writing the final story. It is difficultto know precisely what kinds of bias to expect. However, it isapparent that the ISS cases are organized into “focus areas”and this might suggest that they are biased to telling storiesabout how best practice solutions represented in these areasare introduced (like independent revenue agencies, forinstance). 12 There is also a common charge that published caseslike those in the ISS repository tell stories about heroic figuresand simplify the narrative to center on such person. 13 If true,one would expect a bias in favor of the SLDC approach toexplaining success (where individual leaders drive the process).One might also expect case writers to adopt a linear story-tellingapproach that could also bias the narratives toward telling anSLDC type story (where a hero introduces a solution througha top-down process that can be easily reproduced).

To facilitate measurement and analysis in each case, theauthor constructed a survey questionnaire based (mostly) onthe hypotheses formulated above. The major questions were

closed ended and used 3 or 5 point scales. 14 The questionnairealso allowed raters an opportunity to provide open-endeddescriptions of the rationale behind their rating decisions.The questionnaire was pretested and then used for codingthe full sample, which was done by 12 raters. The raters wereall graduate students with a background in public sectorreform but with no experience in the particular cases. At least

202 WORLD DEVELOPMENT

two raters coded all the cases. The inter-rater reliability wascalculated using a simple measure of percent agreement andvaried from 50% to 90% (the low percent agreement questionsare used with caveats). Coding discrepancies were resolved bythe lead author by computing the arithmetic mean in each caseto obtain a single coding for each variable and case. Suchapproach is considered appropriate for this type of research,given that there is actual value in capturing and reflectingthe subjective differences in opinion of “expert” coders(Newig & Fritsch, 2009).

The author adopted two approaches to further foster relia-bility and validity in the final codes. First, the authorcross-referenced final codings with the qualitative explanationsprovided by raters. This allowed the opportunity to ensurethat the codes were congruent with the written rationales givenby each rater. Second, the author sought expert judgment ofthird parties who had been involved in a sub-set of the cases(10 in total). These third parties were asked whether finalcodes made sense given their experience. In all ten cases thethird parties agreed with over 90% of the measures. Whereasthese kinds of checks allow a higher degree of confidence inthe data, they do not address all potential issues with reliabil-ity and validity of the data, which must be seriously consideredgiven the subjective nature of the coding approach (and, inevi-tably, of the cases themselves). Basic statistical techniqueswere used to assess the coded data. These includedmid-point calculations, frequency counts, and bivariate corre-lation. Whereas more rigorous methods are required to testany theory, these simple measures are commonly used andappropriate in case surveys—especially those intent on con-tributing to new theory development (Herek, Janis, & Huth,1987; Mintzberg, Raisinghani, & Theoret, 1976; Newig &Fritsch, 2009; Yin & Heald, 1975).

Table 2. Case survey evidence explains positive de

Key question Solution and leader driven change (SLDC)

A. What motivatesreform?

H1. “Reform content was fully known at the st6 cases scored 4 or above.20 cases scored below 2.5.Average out of 5: 2.4a

B. How do reforms getimplemented?

H2. “The reform content was fully planned outstart and was implemented as planned”

4 scored 4 or above.18 scored below 2.5.Average out of 5: 2.3c

C. Who leads the reformprocess?

H3. “Was there a clear leader involved in the cprocess?”

12 cases yes and identified the same person as “

leader”

D. What do the “new”

government/governancestructures look like?

H4. “The final reform product reflects best pracdesigns from western or other developing countonly.”There were only 2 cases where the final reform prreflected a “best practice”

Source: Author’s analysis.a Standard deviation was 1.27.b Standard deviation was 0.77.c Standard deviation was 1.16.d Standard deviation was 0.98.e Standard deviation was 12.8.f Standard deviation was 0.56.

(b) Evidence and discussion

Table 2 shows the frequency counts and averages emergingfrom analyses of all thirty cases. These are organized as theyrelate to the four questions posed earlier (by row) and thetwo alternative theories discussed in the prior section (by col-umn). For instance, the first line synthesizes evidence about“what motivates reform” in these positive deviance cases.Two competing hypotheses were evaluated. The first hypothe-sis was that successful reforms would be motivated by the pres-ence of a “solution” (the SLDC hypothesis) and the fifthhypothesis was that these reforms would be motivated by somekind of problem (the PDIA hypothesis). Evidence is more sup-portive of the latter PDIA hypothesis, with raters giving 19cases scores over 4 (out of 5) when asked if “the reform wasmotivated by some kind of problem in the context.” The aver-age score was 4.2 on this question, and only 3 cases scoredbelow 2.5. 15 In contrast, raters only scored one case as a 5when asked if the “reform content was fully known at the start”(something one would expect if the reform was motivated by asolution). Raters gave 20 cases scores below 2.5 when assessingthis question, and 7 scored it the minimum score (1) to indicatemajor disagreement with the statement. 16 The average scorewas 2.4 out of 5. This suggests that a majority of cases consid-ered positive deviants were motivated by problems (not solu-tions as suggested in the SLDC theory).

One can see the same kind of story in respect of the twohypotheses about implementation. Raters gave 13 cases scoresof 4 or higher (out of 5) when asked if “the reform contentemerged through the process, with experimentation and trialand error.” 17 A further 9 cases were given scores above 3(meaning there was evidence of experimentation andemergence). The average score was 3.4, which was a full point

viance in public sector reform in development

Problem driven iterative adaptation (PDIA)

art” H5. “The reform was motivated by some kind of problem inthe context”19 cases scored 4 or above.3 cases scored below 2.5.Average out of 5: 4.2b

at the H6. “The reform content emerged through the process, withexperimentation and trial and error”

13 scored 4 or above.8 scored below 2.5.Average out of 5: 3.4d

hange

the

H7. “Was there a clear leader involved in the change process?”

18 cases said “no” or raters identified multiple people as “theleader”.Also, in all cases raters identified multiple agents providingfunctions commonly associated with “leaders”. Average agentsper case: 19.e

ticeries

oduct

H8. “The final reform design incorporated ideas from a rangeof different areas.”There were 28 cases where the final reform products were a mixof sources.The average number of sources seen to influence reformdesigns: 2.35.f

EXPLAINING POSITIVE DEVIANCE IN PUBLIC SECTOR REFORMS IN DEVELOPMENT 203

higher than the average score registered in response to thequestion, “Was the reform content fully planned out at thestart of the change process, and implemented as planned?”Raters combined to give 18 countries fewer than 2.5 pointswhen registering answers to this question, suggesting that mostof the cases had explicit evidence showing that reformsemerged through the process and were not pre-designed (asthe SLDC approach would suggest). 18 The box plot showsthat there is no overlap between the interquartile ranges ineach case; scores for preplanned implementation were signifi-cantly lower than those for “emergent” implementation—thePDIA explanation.

This PDIA perspective was supported in other areas of thesurvey, where raters noted that 19 of the cases had high levelsof “learning” that contributed to the final solution (and werethus not predesigned). Raters of a similar number of cases(20) agreed with the statement that “information collectedthrough monitoring and evaluation mechanisms (formal orinformal) was fed back into the change process and allowedadjustments to the reform.”

The evidence in support of question C (“who leads thereform process”) is more nuanced. Raters were asked whetherthere was a “clear leader involved in the change process” andthen they were asked to name this leader. The level ofinter-rater agreement on these questions was just 50%, whichis surprisingly low given the assumed bias of ISS cases dis-cussed earlier (and the expectation that published cases com-monly focus on telling stories about outstandingprotagonists). 19 Only 12 sets of raters agreed that there wasa clear leader in the case, and identified the same person asthat leader. In eight cases one rater said there was a leaderand one said there was no leader. In four cases both raters saidthere were many leaders, identifying a group of leaders inresponse to the question. In three cases, both raters said therewas no single leader. In two cases both raters said there was aleader but identified different people as “the leader”. Theraters were also asked to identify who provided functions com-monly associated with “leaders” (Andrews, 2013; Hackman &Walton, 1986). These included “substantive”, “procedural”,and “maintenance” contributions. 20 In all cases, raters identi-fied multiple agents providing these functions, with an averageof 19 agents per case. Interestingly, where they were identified,the “clear leader” was seen to provide (at least primarily) avery narrow set of functional contributions—authorizingchange, motivating the change, and convening groups (mostcommonly teams) to make the change happen. The other lead-ership functions were played by members of a broader group.

While the evidence about leadership is nuanced, therefore, itseems more supportive of a PDIA explanation than an SLDCexplanation for how positive deviance comes about. The samecan be said for evidence shown in the last row of Table 2;related to the final product of the reform process. The SLDCapproach leads one to expect that successful reforms will ulti-mately resemble a pure-form version of international bestpractice (considered the “right” solution and assessed in com-mon indicator sets used by development organizations). A suc-cessful public budgeting system will include a multi-yearframework akin to that accorded an “A” in the multi-donorPublic Expenditure and Financial Accountability (PEFA)framework, for instance, and have a competitive bidding pro-cess in the procurement regime (like that propagated by theUnited Nations). A successful revenue reform will producean independent revenue agency with a specific set of checksand balances and accountability mechanisms. In contrast,the PDIA approach would have one expect that final reformproducts look more like hybrids, combinations of many

influences into locally fitted (and often peculiar) forms (thatfunction regardless). A functional budgeting system may cover1 year or many years. Similarly, a functional procurement sys-tem might incorporate elements of competitive bidding along-side other purchasing modalities. A functional revenue systemmight not have an independent agency at all.

To assess if evidence spoke more readily to one of theseexplanations, raters were asked about the influences theysaw on final reform products. They could indicate if therewas strong evidence that the reform product had been influ-enced by external western best practice, developing countrybest practice (emerging from south to south collaborations),past practices within the country, and emerging ideas withinthe country. In their responses to this set of questions, onlytwo of the sets of raters agreed that the final product was influ-enced by external best practice only. This means that 28 sets ofraters agreed that there was mixed influence on the final pro-duct—what one would expect from a hybrid. The averagenumber of sources that raters showed was influencing reformdesigns was 2.35, and the dominant influence was “emergingpractices within the country” (this was the dominant influencein 14 cases, and a secondary influence in five other cases). Thissuggests that most positive deviants were hybrid solutions thatemerged from internal processes and were heavily influencedby emerging internal ideas.

(c) PDIA and SLDC as one theory or two separateexplanations?

Given the above, it appears that evidence arising from thecase survey is more supportive of a problem-driven iterativeadaptation (PDIA) explanation than a solution andleader-driven change (SLDC) explanation: Abnormally suc-cessful reforms (or positive deviants) seem to be motivatedby problems more than solutions; The positive deviantreforms appear to emerge from a process of experimentationand learning more often than through a predesigned, plannedout process; These positive outliers seem to result from theengagement of many leaders more commonly than from theleadership of one outstanding individual; The successes pro-duce hybrids more often than pure-form best practices.

Whereas there is evidence to suggest that the above conclu-sions might be strong takeaways from this study, however, itis clear that the story is not open and closed. There are data sug-gesting that at least some aspects of SLDC contribute to theexplanation of positive deviance. A number of raters did findthat solutions existed at the start of some reforms, for instance,and a number of raters also found that reform processes werepre-planned and followed the plan. Furthermore, nearly halfof the raters agreed that a single leader was driving the process.

The study shows how such SLDC characteristics might over-lap with PDIA characteristics to facilitate a blended approachof producing positive deviant reforms. This is most evidentwhen looking at the way raters assessed questions asking aboutthe interaction of problems and (potential) solutions in motivat-ing reforms. Indicating that problems alone do not motivatereform, 18 pairs of raters agreed that, “There was a problempeople wanted to address, and there was a proposal for action.”In contrast, only seven pairs of raters agreed that, “There was aproblem people wanted to address, even though no one reallyknew what to do.” 21 This suggests that even where there wasa problem motivating reform, some proposal had to be on thetable to guide action. This argument echoes theories of institu-tional change that emphasize the importance of some disruptionto a system and the presence of some viable alternative to thestatus quo (Andrews, 2013a). Such argument is supported by

204 WORLD DEVELOPMENT

the fact that raters of 13 cases agreed with the comment that,“reform content was partially known at the start.” 22 Combinedwith the 6 cases where reform content was fully known at thestart, these data show that over half of the cases began withsome level of knowledge about solutions. Such knowledgemight have been an important motivation for reform, evenwhere problems also motivated intervention.

One could see the potential blend of SLDC and PDIA argu-ments working in other ways as well. It is apparent, for instance,that some interventions in this ISS sample began with at leastpartial knowledge of a solution and simultaneously involvedexperimental implementation processes. These cases all seemto have generated new hybrids (as authors like Ostrom (2008)would predict). Such is the example of many cases across the lit-erature, where flexible design processes allow an initial idea tomorph into something new that can be fitted to complex con-texts (Andrews & Moynihan, 2002; Andrews, 2005).

However, the blended theory one might imagine when con-sidering this argument falls short of a full-blown, shared the-ory. In a sense it is more like a PDIA-lite version, wherechange is motivated by both problems and a potential solutionbut the flexible process of implementation still means that theproposed solution gets shaped to fit the context. One is nolonger talking about a blended theory if this interpretation isaccepted, but then it still becomes important to think abouthow SLDC and PDIA relate.

Interestingly, when one examines bivariate correlations, it isactually apparent that in most cases the SLDC and PDIAhypotheses stand as separate explanations—as one wouldexpect in two different theories. Correlations between the vari-ables were calculated using Kendall’s tau-b (with Fisher’s exacttest used to test the significance of the relationships), 23 and anumber of important observations arose. First, there was a pos-itive correlation of 0.65 (significant at the 10% level) between thevariable capturing whether reforms were fully known and thevariable reflecting whether reforms were preplanned (as perhypotheses 1 and 2, in SLDC). This does not seem like a highcorrelation or level of significance, but prior studies suggest thatin this kind of analysis any correlation above 0.45 should beconsidered high and any significance of 90 or higher should be

considered “significant”. 24 Given this, the high, positive, andsignificant correlation observed suggests the existence of a setof cases where these two SLDC hypotheses interacted to explainsuccess. The cases in this sample included, Nigeria’s FederalInland Revenue Service, Sergio Fajardo and Medellin, PublicSector Reform in Georgia, and Police Reform in Lesotho.

Similarly, the first two PDIA variables (reflecting a problemmotivation and emergent, experimental process, as perhypotheses 5 and 6) had a high correlation (0.63) that was sig-nificant at the 5% level. 25 These statistics show that a set ofpositive deviants had both a problem motivation and an emer-gent, experimental implementation process. This sub-setincluded the following cases (with others): Improving DecisionMaking at the Center of Government: Liberia’s Cabinet Sec-retariat 2009–12; Matching Goodwill with National Priorities:Liberia’s Philanthropy Secretariat, 2008–12; Building Capac-ity, Changing Norms: Rapid Results in Madagascar, 2005–09; Promoting Accountability, Monitoring Services: TextbookProcurement and Delivery, The Philippines, 2002–05;Rebuilding the Civil Service After War: Rwanda After theGenocide, 1998–2009; and The Promise of Imihigo: Decentral-ized Service Delivery in Rwanda, 2006–10.

This evidence thus suggests two dominant paths towardachieving positive deviance in public sector reform in develop-ment. One path (which seems to have fewer cases associatedwith it) is SLDC, where one needs a driving idea and a clear

plan for implementation. The other (wider) path is PDIA,where a problem drives change and a flexible process allowsexperimentation, learning, and adjustment. For instance, therewas a negative relationship between the variable capturingwhether interventions were problem driven (hypothesis 5, inPDIA) and the variables reflecting whether reforms weresolution-driven and preplanned (hypotheses 1 and 2, inSLDC). 26 The correlation between the variable showingdegree of emergence and experimentation (another key PDIAvariable, as per hypothesis 6) and the SLDC variable for pre-planned implementation (hypothesis 2) was also negative

(�0.59) and significant at the 10% level. 27 This suggests thatthere is a big difference between SLDC reforms that are moti-vated by a solution and follow a predetermined path and PDIAreforms that begin with a strong focus on solving problems(and only a partial focus on some or other potential solution)and are allowed to emerge through experimentation. This evi-dence seems to clash with earlier comments about a potentialthird path—or PDIA lite—where one sees some blending ofthe approaches. There is no clash in evidence, however, in thatthe evidence that PDIA and SLDC are two paths is not abso-lute—with correlations below one. This suggests that in mostcases the two pathways are different but there are examplesof soft blending where a pre-designed reform becomes moreflexible as it proceeds (as a PDIA-lite intervention).

4. CONCLUSION

It serves as useful to remind readers that the case surveyevidence used in this study was drawn from an analysis ofthirty narratives of public sector institutional reform pub-lished by the Innovations in Successful Society (ISS) programat Princeton University. The ISS cases are examples ofreforms that have yielded functional improvement in someof the toughest contexts in development, where reform failureis far more normal. As such, they are considered abnormalsuccesses—or positive deviants—and are the kinds of reformexperiences where one would hope to learn a great deal. Thecurrent paper intends to contribute to a better understandingof this kind of positive deviance in public sector reform indeveloping countries. It focuses on learning about the pro-cesses through which such deviants arise and are imple-mented. Rather than testing a particular theory, it has settwo theories in competition with each other and askedwhether evidence from the case survey study supports oneor the other (or parts of both).

As presented, case survey evidence seems more supportive ofproblem-driven iterative adaptation (PDIA) as a theoryexplaining positive deviance. In support of major PDIAhypotheses, the majority of reforms in the sample seem to havebeen motivated by a problem, implemented through flexibleprocesses involving experimentation, and led by groups. Alsosupporting PDIA is the fact that over 90% of these reformsultimately produced hybrid products that did not resemblepure-form best practices. This strong evidence is particularlyimpressive given potential bias in the ISS sample toward anSLDC narrative (as discussed). However, there is also evidencethat solution- and leader-driven change (SLDC) has somevalue in the discussion about positive deviance. There weresome reforms that seem to have been motivated by clearlydefined solutions, implemented according to plan, led by a sin-gle leader, and ultimately proved successful in introducing afunctional “best practice” product.

This evidence suggests that SLDC and PDIA offer two dom-inant paths for producing positive deviance in public sector

EXPLAINING POSITIVE DEVIANCE IN PUBLIC SECTOR REFORMS IN DEVELOPMENT 205

reform in developing countries. The PDIA path appears muchwider than the SLDC path, seemingly explaining more of thecases than SLDC. Future work should explore why this is so,and especially whether different strategies fit different types ofpublic sector reform challenges. It may be that PDIA is theapproach required when reforms are in particularly complexareas, contextual impediments are severe and difficult to nav-igate, there are no clear solutions, and political challenges areparticularly difficult to manage. The current study was notdesigned to test this, but the idea has some support in the factthat most of the initiatives that seemed to follow a PDIA tra-jectory also related to more complex and contested reformtypes (like civil service reform in Rwanda and procurementreform in the Philippines). Similarly, it could be that SLDCworks best when reforms are more technical, do not demandcontextual fit, and/or are more likely to appeal to a top downleader (and fall under his or her authorization to pushthrough). Once again, the current study was not designed totest this but does suggest it may be a valid path for futureresearch (revenue and policing reforms fell into this category,for instance, which seem to fit the description provided).

It is also interesting that some cases of positive deviance seemto be explained through a softened version of PDIA (PDIA-lite);where positive deviants are motivated by both problems and aproposed (partially identified) solution but where flexible imple-mentation processes still ensure solutions are shaped to the con-text. Future research needs to examine whether this kind ofPDIA-lite intervention is more common than an extreme version.This would be a very useful finding for those in the developmentcommunity who might be eager to experiment with PDIAapproaches but need to do so within the parameters of a tradi-tional project preparation process (that incentivizes a more directSLDC orientation). If PDIA can start with both a problem andsolution motivation it might be possible to fit such approach intoa results based lending instrument—as long as there is flexibilityand learning in implementation.

The evidence from this study needs to be qualified, however,and applied to practice with care. The study could be strength-ened by facilitating more scrutiny of political factors affectingpublic sector reform and implementation in government, forinstance. As discussed earlier, there are well-established argu-ments about the motivations that could underpin political deci-sions to pursue and support reform that this study has notexplicitly explores (see, for instance, Lemos, & de Oliveira,2004; Moore, 2014). These arguments would be helpful inaddressing some of the questions about leadership emergingin this work, as well as the questions about a match betweenSLDC and PDIA and different reform types. It would be useful

to investigate if reforms that others find politically favored (likepolicing and tax reform) are indeed more amenable to top-downSLDC approaches, for instance. Beyond this, the study is fur-ther limited because of a lack of counterfactual cases. It isunclear whether elements of PDIA or SLDC would be moreor less present in less successful reform cases. Future researchcould build on the work in this paper by expanding the sampleto include both positive deviants and more-normal reform fail-ures. Adopting a larger and more varied case sample in futureresearch could also help in managing potential bias one mustexpect when drawing cases from one source (as in this study).Bias could be further mitigated in future work by drawing onsome of the lessons about coding emerging in this study. In par-ticular, the variables used to capture leadership proved difficultto craft—partly because raters disagreed on basic questions(like whether there was a single leader). It is important to specifyexactly what one is looking for in this kind of study and it wouldbe better to ask more explicitly about who provided what func-tional contributions to the reform (as was asked in a secondaryset of questions in the current study).

Future studies are definitely needed before one can explainpositive deviance in public sector reform in developing coun-tries. These studies could expand on the theoretical frame-works considered for assessment (not just the samplevariation). There is significant scope to build more on worklike Leonard (2010) and on past work like Bond and Hulme(1999) in the “process projects” approach of the 1980s, andof course people like Hirschman (1967). Even without theseadditional views, however, the current paper has contributedto a better understanding of such deviance. It yields the lessonthat two paths exist to produce an abnormally successfulreform, but one is much wider than the other and will proba-bly generate more success in more cases. This latter path is onein which reforms take shape through a process calledproblem-driven iterative adaptation (PDIA) where they aremotivated by problems, emerge through experimentationand learning, are led by groups, and ultimately produce newhybrid solutions that make governments work better than theydid before reforms began. In reflecting on such path, one isreminded of Albert O. Hirschman’s writing on implementa-tion in development (Hirschman, 1967, p. 35) and the impor-tance of thinking about development projects as journeys:

“The term “implementation” understates the complexity of the task ofcarrying out projects that are affected by a high degree of initial igno-rance and uncertainty. Here “project implementation” may often meanin fact a long voyage of discovery in the most varied domains, fromtechnology to politics.”

NOTES

1. Spending on governance accounted for about 20% of the Departmentfor International Development’s (DFID) activities, while more than 20%of the spending focused on economic reforms that tended to involveinterventions at the interface of the public and private sectors. Beyondthis, DFID documents note that institutional reforms are common insectoral engagements (like water supply and sanitation, health andeducation). Department for International Development (2011).

2. Asian Development Bank (2009, 2011), African Development Bank(2012).

3. Governance operations in the African Development Bank during1967–2006 accounted for 15% of all loans. Most took place after themid-1990s (African Development Bank, 2012).

4. Table 5.1 on page 46 shows that 42% of countries with lending for civilservice reform actually saw improvements in CPIA scores during 1999–2006. The other 58% saw stagnant or declining scores.

5. The CPIA guidelines note that a score of 4.5 reflects functionality inthe system. Scores below this level indicate that countries are establishingprocesses and systems but that these are still not fully functional. Fewerthan 20% of countries break through this score barrier, however, evenafter decades of reform (Andrews, 2013a).

6. These inform countries about the “right” policies needed to fosterbusiness activity, or manage public finances, or tackle corruption, andbeyond. The Washington Consensus was an example (see Gore,2000).

206 WORLD DEVELOPMENT

7. Brady and Spence (2010) note, for instance, that plans were vital in drivingdevelopment processes in Singapore and South Korea. These plans are arguedto have provided both a direction for reform and acted as a disciplining factor inthe implementation process. Leaders could assess progress according to plan,for instance, and accountability for action was clear.

8. Brady and Spence (2010, 7).

9. http://www.princeton.edu/successfulsocieties/aboutiss/.

10. It is difficult to establish a common and clear perspective on whatconstitutes “success” in reforms. VanLandingham, Wellman, andAndrews (2005) and Roberts and Andrews (2005) show that reformscan appear successful in changing what governments look like, but couldalso fail to produce results. de Renzio et al. (2010) and Andrews (2013a)advocate looking at whether reforms foster changes that help governmentsbecome more functional.

11. The raters were asked to score each case on “whether it could becalled a success”. They were also asked to answer whether the case heldvalue for academics or practitioners examining successful reform. Therewas sufficient agreement across raters that all thirty cases used for the finalanalysis could be called successful.

12. http://www.princeton.edu/successfulsocieties/content/.

13. Barzelay, 2007.

14. Similar to the approach taken in Beierle and Cayford (2002), Newigand Fritsch (2009), Wolf (1993) and Yin and Heald (1975).

15. There was a high level of agreement on answers to this question (93%).This meant that both raters agreed on the exact coding in 50% of the casesand the raters only differed by one point in a further 43% of the cases.

16. There was a high level of agreement on answers to this question(90%).

17. The level of inter-rater agreement was 77%. Twelve sets of ratersagreed on the exact score; 11 differed by one point.

18. The level of inter-rater agreement was 70%. Eight sets of ratersagreed on the exact score; 13 differed by one point.

19. Fifteen of the 30 sets of raters agreed that either there was a leader(and named the same leader) or that there was no leader. There wasdisagreement in the other 14 sets of raters; meaning that one rateridentified a “leader” and the other did not, or the raters identified differentagents as the leader.

20. Substantive contributions include activities like “identifying prob-lems” and “identifying solutions”. Procedural contributions include“providing authority” and “motivating”. Maintenance contributionsinclude “convening” and “connecting” to allow engagement that fosterschange.

21. There was a high rater agreement for this question (93%).

22. In respect of this question, 13 out of 30 sets of raters scored above 4,6 scored below 2, and the average was 3.08 out of 5. The inter-rateragreement was 70%. Interestingly, in the cases where coders noted that thereform content was fully known, low scores were given for the question“was reform content only partially known”: 4 answered “2” and threeanswered “1.5”.

23. Kendall’s tau-b correlation coefficient is based on the number ofconcordant and discordant pairs of observations in the table, using acorrection for ties. It is an appropriate nonparametric measure ofcorrelation for ordinal data (Beierle & Konisky, 2000; Bullock & Tubbs,1987). Fisher’s exact test measures the probability of observing acontingency table that shows at least as much association betweenvariables as that actually observed.

24. Beierle and Konisky (2000, p. 596). It should be noted that thespearman correlation was 0.81.

25. The spearman correlation was higher, at 0.82.

26. The spearman correlations were �0.43 and �0.40.

27. The spearman correlations were �0.33 and �0.38.

REFERENCES

African Development Bank. (2012). Governance strategic directions andaction plan 2008–2012. Tunis: African Development Bank.

Andrews, M. (2005). Voice mechanisms and local government fiscaloutcomes: How does civic pressure and participation influenceaccountability?. In A. Shah (Ed.), Public expenditure analysis(pp. 217–241). Washington, DC: World Bank.

Andrews, M. (2008). Effective political engagement. In S. Odugbemi, & T.Jacobson (Eds.), Governance reform under real world conditions(pp. 95–111). Washington, DC: World Bank.

Andrews, M. (2010). How far have public financial managementreforms come in Africa? HKS working paper series RWP10-018,May 2010.

Andrews, M. (2011). Which organizational attributes are amenable toexternal reform? An empirical study of African public financialmanagement. International Public Management Journal, 5(2), 135–154.

Andrews, M. (2013a). The limits of institutional reform in development.New York, NY: Cambridge University Press.

Andrews, M. (2013b). How do governments get great? HKS working paperseries RWP13-020, April 2013.

Andrews, M., McConnell, J., & Wescott, A. (2010). Development asleadership led change. Washington, DC: World Bank.

Andrews, M., & Moynihan, D. (2002). Why reforms do not always haveto “work” to succeed. Public Performance & Management Review, 25,282–297.

Andrews, M., Pritchett, L., & Woolcock, M. (2013). Escaping capabilitytraps through Problem Driven Iterative Adaptation (PDIA). WorldDevelopment, 51, 234–244.

Asian Development Bank. (2009). ADB support for public sector reforms inthe Pacific: Enhance results through ownership, capacity, and continuity.Manila: ADB.

Asian Development Bank. (2011). Development effectiveness review.Manila: ADB.

Barzelay, M. (2007). Learning from second-hand experience: Methodol-ogy for extrapolation-oriented case research. Governance, 20(3),521–543.

Bebbington, A., & McCourt, W. (Eds.) (2007). Development success:Statecraft in the south. Basingstoke: Palgrave Macmillan.

Beierle, T. C., & Cayford, J. (2002). Democracy in practice: Publicparticipation in environmental decisions. Washington, DC: Resourcesfor the Future.

Beierle, T. C., & Konisky, D. M. (2000). Values, conflict, and trust inparticipatory environmental planning. Journal of Policy Analysis andManagement, 19(4), 587–602.

EXPLAINING POSITIVE DEVIANCE IN PUBLIC SECTOR REFORMS IN DEVELOPMENT 207

Bond, R., & Hulme, D. (1999). Process approaches to development:Theory and Sri Lankan practice. World Development, 27(8),1339–1358.

Boubakri, N., Cosset, J. C., & Guedhami, O. (2009). From state to privateownership: Issues from strategic industries. Journal of Banking andFinance, 33(2), 367–379.

Brady, D., & Spence, M. (2010). Leadership and politics: A perspectivefrom the commission on growth and development. In D. Brady, & M.Spence (Eds.), Leadership and growth (pp. 1–17). Washington, DC:World Bank.

Bueno de Mesquita, B., & Smith, A. (2012). The dictator’s handbook: Whybad behavior is almost always good politics. New York: Public Affairs.

Bullock, R., & Tubbs, M. (1987). The case meta-analysis method for OD.Research in Organizational Change and Development, 1, 171–228.

Busenitz, L. W., Gomez, C., & Spencer, J. W. (2000). Country institu-tional profiles: Unlocking entrepreneurial phenomena. Academy ofManagement Journal, 43(5), 994–1003.

Cameron, K. S. (1986). Effectiveness as paradox: Consensus and conflictin conceptions of organizational effectiveness. Management Science,32, 539–553.

Chambers, R. (1997). Whose reality counts? Putting the first last. London:IT Publications.

Dacin, M. T., Goodstein, J., & Scott, W. R. (2002). Institutional theoryand institutional change: Introduction to the special research forum.Academy of Management Journal, 45(1), 45–57.

Department for International Development. (2011). Governance portfolioreview summary. London: DFID.

de Renzio, P., Andrews, M., & Mills, Z. (2010). Evaluation of donorsupport to Public Financial Management (PFM) reform in developingcountries. London: Overseas Development Institute.

Dorado, S. (2005). Institutional entrepreneurship, partaking, and conven-ing. Organization Studies, 26(3), 385–414.

Garud, R., & Karnøe, P. (2003). Bricolage versus breakthrough:distributed and embedded agency in technology entrepreneurship.Research Policy, 32, 277–300.

Gore, C. (2000). The rise and fall of the Washington consensus as aparadigm for developing countries. World Development, 28(5),789–804.

Greenwood, R., & Suddaby, R. (2006). Institutional entrepreneurship inmature fields: The Big Five accounting firms. Academy of ManagementJournal, 49, 27–48.

Greenwood, R., Suddaby, R., & Hinings, C. R. (2002). Theorising change:The role of professional associations in the transformation ofinstitutional fields. Academy of Management Journal, 45(1), 58–80.

Grindle, M. (1997). Divergent cultures? When PUBLIC ORGANIZA-TIONS PERFORM WELL IN DEVELOPING COUNTRIES. WorldDevelopment, 25(4), 481–495.

Grindle, M. (2004). Despite the odds: The contentious politics of educationreform. Princeton: Princeton University Press.

Grindle, M., & Thomas, J. (1991). Public choice and policy change: Thepolitical economy of reform in developing countries. Baltimore: TheJohns Hopkins University Press.

Hackman, J. R., & Walton, R. E. (1986). Leading groups in organizations.In P. S. Goodman (Ed.), Designing effective work groups (pp. 72–119).San Francisco: Jossey-Bass.

Heredia, B., & Schneider, B. R. (2002). The political economy ofadministrative reform in developing countries. In B. R. Schneider, &B. Heredia (Eds.), Reinventing leviathan: The politics of administrativereform in developing countries (pp. 1–29). Miami: North-South CenterPress.

Herek, G. M., Janis, I. L., & Huth, P. (1987). Decision making duringinternational crises—Is quality of process related to outcome?. Journalof Conflict Resolution, 31(2), 94–112.

Hirschman, A. (1967). Development projects observed. Washington, DC:Brookings.

Josh, A., & Moore, M. (2004). Institutionalised co-production: unortho-dox public service delivery in challenging environments. Journal ofDevelopment Studies, 40(4), 31–49.

Karikari, J. A. (2010). Governance, financial liberalization, and financialdevelopment in Sub-Saharan Africa. African Economic Conference,November 2010. Available at: <http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1733322>.

Korten, D. C. (1980). Community organization and rural development: Alearning process approach. Public Administration Review (September/October).

Larsson, R. (1993). Case survey methodology: Quantitative analysis ofpatterns across case studies. The Academy of Management Journal,36(6), 1515–1546.

Larsson, R., & Lubatkin, M. (2001). Achieving acculturation in mergersand acquisitions: An international case survey. Human Relations,54(12), 1573–1607.

Lemos, M., & de Oliveira, J. (2004). Can water reform survive politics?Institutional change and river basin management in Ceara, NortheastBrazil. World Development, 32(12), 2121–2137.

Leonard, D. (1991). African successes: Four public managers of Kenyanrural development. Berkeley: University of California Press.

Leonard, D. (2010). ‘Pockets’ of effective agencies in weak governancestates: Where are they likely and why does it matter?. PublicAdministration and Development, 30(2), 91–101.

Lindblom, C. (1959). The science of ‘muddling through’. Public Admin-istration Review, 19, 79–88.

Lucas, W. A. (1974). The case survey method: Aggregating case experience.Santa Monica.

McCay, B. J. (2002). Emergence of institutions for the commons:Contexts, situations, and events. In E. Ostrom (Ed.), The drama ofthe commons (pp. 361–399). Washington, DC: National ResearchCouncil.

Mintzberg, H., Raisinghani, D., & Theoret, A. (1976). The structure of“unstructured” decision processes. Administrative Science Quarterly,21(2), 246–275.

Moloney, K. (2009). Public administration and governance: a sector-levelanalysis of World Bank aid. International Review of AdministrativeSciences, 75(4), 609–627.

Moore, M. (2014). Revenue reform and state building in anglophoneAfrica. World Development, 60, 99–112.

Newig, J., & Fritsch, O. (2009). The case survey method and applicationsin political science. Paper presented at 2009 APSA meeting.

Obtsfeld, M. (2009). International finance and growth in developingcountries: What have we learned?. IMF Staff Papers, 56, 63–111.

Ostrom, E. (2008). Design principles of robust property-rights institutions:What have we learned?. In K. G. Ingram, & Y.-H. Hong (Eds.),Property rights and land policies (pp. 25–51). Cambridge, MA: LincolnInstitute of Land Policy.

Owusu, F. (2006). On public organizations in Ghana: What differentiatesgood performers from poor performers?. African Development Review,18, 471–485.

Pascale, R., Sternin, J., & Sternin, M. (2010). The power of positivedeviance: How unlikely innovators solve the world’s toughest problems.Cambridge: Harvard Business Review Press.

Pritchett, L., Woolcock, M., & Andrews, M. (2013). Looking like a state:Techniques of persistent failure in state capability for implementation.The Journal of Development Studies, 49(1), 1–18.

Roberts, J., & Andrews, M. (2005). Something funny happened on theway to reform success. . .The case of budget reform implementation inGhana. International Journal of Public Administration, 28(3–4),291–311.

Rodrik, D. (2003). In search of prosperity: Analytic narratives on economicgrowth. Princeton: Princeton University Press.

Rondinelli, D. A. (1993). Development projects as policy experiments: Anadaptive approach to development administration (2nd ed.). London:Methuen.

Rose, R. (2003). What’s wrong with best practice policies-and whyrelevant practices are better. On target? Government by measurement.London: House of Commons Public Administration Select CommitteeHC 62-II, 2003, pp. 307–317.

Schneider, B. R. (1991). Politics within the state: Elite bureaucrats andindustrial policy in authoritarian Brazil. Pittsburgh: University ofPittsburgh Press.

Seo, M. G., & Creed, W. E. D. (2002). Institutional contradictions, praxisand institutional change: A dialectical perspective. Academy ofManagement Review, 27(2), 222–247.

Tendler, J. (1997). Good government in the tropics. Baltimore: JohnsHopkins University Press.

Uphoff, N. (1985). Fitting projects to people. In M. Cernea (Ed.), Puttingpeople first: Sociological variables in rural development. New York:Oxford University Press.

Uphoff, N., Esman, M. J., & Krishna, A. (1998). Reasons for success:Learning from instructive experiences in rural development. WestHartford, Conn.: Kumarian Press.

208 WORLD DEVELOPMENT

VanLandingham, G., Wellman, M., & Andrews, M. (2005). Useful, butnot a panacea—Performance-based program budgeting in Florida.International Journal of Public Administration, 28(3–4), 233–253.

Vis, B., & van Kersbergen, K. (2007). Why and how do political actorspursue risky reforms?. Journal of Theoretical Politics, 19(2), 153–172.

Wolf, P. (1993). A case survey of bureaucratic effectiveness in US cabinetagencies: Preliminary results. Journal of Public Administration,Research and Theory, 3(2), 161–181.

World Bank. (2008). Public sector reform: What works and why?.Washington, DC: World Bank.

ANNEX 1THE THIRTY ISS CA

Case title

A Change Agent in the Tax Office: Nigeria’s Federal Inland Revenue ServiA promise kept: How Sierra Leone’s president introduced free health care,Against the Odds: Attempting Reform in Suharto’s Indonesia, 1967–98Building Capacity, Changing Norms: Rapid Results in Madagascar, 2005–0Building civil service capacity in post conflict Liberia, 2006–11Building Civilian Police Capacity: Post-Conflict Liberia, 2003–11Clearing The Jungle Raj, Bihar State, India, 2005–09Creating an Affordable Public Service: Tanzania, 1995–98Decentralizing Authority After Suharto: Indonesia’s “Big Bang,” 1998–201Defusing a Volatile City, Igniting Reforms: Joko Widodo and Surakarta, IDelivering on the hop of the Rose Revolution: Public sector reform in GeoDisseminating the Power of Information: Kenya Open Data Initiative, 2011Empowering Operational Staff: Land Registration in Sarawak, Malaysia, 2Energizing the civil service: Managing at the top, Bangladesh, 2006–11Focusing on Priority Goals: Strategic Planning in Lithuania, 2000–04’ andFighting Reform Fatigue in Lithuania, 2006–12”

From Central Planning to Performance Contracts, New Public ManagemenFrom Fear to Hope in Colombia: Sergio Fajardo and Medellın, 2004–07Improving decision making at the center of government: Liberia’s cabinet sInstilling Order and Accountability: Standard Operating Procedures at IndMatching Goodwill with National Priorities: Liberia’s philanthropy secretaPromoting Accountability, Monitoring Services: Textbook Procurement anRebuilding the Civil Service After War: Rwanda After the Genocide, 1998–Reining in a Rogue Agency: Police Reform in Lesotho, 1997–2010Restoring Police Service with a Community Vision: TanzaniaRestructuring Service Delivery: Johannesburg, South Africa, 1996–01Reworking the revenue service in South AfricaShifting the cabinet into high gear: Agile policymaking in Rwanda, 2008–1Strengthening Public Administration: Brazil 1995–98The Promise of Imihigo: Decentralized Service Delivery in Rwanda, 2006–1Tying Performance Management to Service Delivery: Public Sector Reform

World Bank. (2009). Improving effectiveness and outcomes for the poor inhealth, nutrition, and population: An evaluation of World Bank groupsupport since 1997. Washington, DC: World Bank.

World Bank. (2011). IEG annual report 2011. Washington, DC: WorldBank.

World Bank. (2012). World Bank approach to public sector management2011–2020: Better results from public sector institutions (draft).Washington, DC: World Bank.

Yin, R., & Heald, K. (1975). Using the case method to analyze policystudies. Administrative Science Quarterly, 20(3), 371–381.

SES USED IN THIS STUDY

ce, 2004–092009–10

9

0ndonesia, 2005–11rgia: 2004–09–12

006-2009

“Improving the Quality of Decision Making:

t in Mongolia, 1996–2009

ecretariat 2009–12onesia’s Ministry of Finance, 2006–07riat, 2008–12d Delivery, The Philippines, 2002–052009

2

0in Malaysia, 2009–11

ScienceDirectAvailable online at www.sciencedirect.com