Reviewing the role of external actors · International development actors are, increasingly,...

Strengthening public investment managementReviewing the role of external actorsMark Miller and Tom Hart

Report

September 2017

Overseas Development Institute203 Blackfriars RoadLondon SE1 8NJ

Tel. +44 (0) 20 7922 0300 Fax. +44 (0) 20 7922 0399 E-mail: [email protected]

www.odi.org www.odi.org/facebook www.odi.org/twitter

Readers are encouraged to reproduce material from ODI Reports for their own publications, as long as they are not being sold commercially. As copyright holder, ODI requests due acknowledgement and a copy of the publication. For online use, we ask readers to link to the original resource on the ODI website. The views presented in this paper are those of the author(s) and do not necessarily represent the views of ODI.

© Overseas Development Institute 2017. This work is licensed under a Creative Commons Attribution-NonCommercial Licence (CC BY-NC 4.0).

Cover photo: Construction workers on site. Arne Hoel / World Bank

Strengthening public investment management: reviewing the role of external actors 3

Acknowledgements

We thank Simon Groom, Philipp Krause, Annette Mummert, Anand Rajaram, Kirsten Richter-Raschke and Ossmann Svenja for insightful discussions and comments in the preparation of this paper. This paper was generously supported by Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ) GmbH on behalf of the German Federal Ministry of Economic Cooperation and Development (BMZ). The views and analysis are the responsibility of the authors alone. We would also like to thank Richard Hughes who provided programming support, Julia Hanne for overseeing the production of the document, Angela Hawke and Alasdair Deas for editing and proofreading, and Garth Stewart for document design.

Contents

Acknowledgements 3

1. Introduction 7

2. What is ‘public investment management’? 8

2.1. Changing perspectives on public investment 8

2.2. Changing perspectives on the management of public investment 10

2.3. The rise of public investment management 11

2.4. Summary 13

3. What do public investment management diagnostics tell us? 14

3.1. Introduction 14

3.2. The scope of public investment management diagnostics 14

3.3. Approaches to diagnostics 18

3.4. Summary 21

4. Where next for international engagement in this area? 22

4.1. Emerging risks and measures to mitigate those risks 22

4.2. Moving from diagnostics to reform 25

4.3. Drawing lessons from the public investment management literature for other sectors and policy areas 28

4.4. Principles for undertaking diagnostics 28

Annex 1: Diagnostic questions for assessing the functionality of PIM systems 30

References 32

4 ODI Report


Boxes

Box 1. International Monetary Fund’s Public Investment Management Assessment (PIMA) 17

Box 2. Supporting room for manoeuvre in the change process in Uganda’s roads sector 28

List of boxes, figures and tables

Figures

Figure 1. Public Investment 1970-2011 (percentage of GDP, weighted by purchasing power parity) 8

Figure 2. Time and cost overruns for investment projects in selected countries 12

Figure 3. Key features of a public investment management system 16

Figure 4. Public Investment Management Assessment score by country income 19

Figure 5. Public investment institutional scores by country group 20

Figure 6. A ‘blueprint’ approach to public investment management reform 26

Tables

Table 1. Key components of public investment management (PIM) diagnostics 18

Table 2. Assessing the functionality of public investment management systems 24

Table 3. Assessing the fiscal sustainability of the overall level of public investment 25

Table 4. Diagnostic questions for assessing infrastructure operation 25

Acronyms

GDP Gross domestic product

IMF International Monetary Fund

OECD Organisation for Economic Co-operation and Development

MDG Millennium Development Goal

PEFA Public Expenditure and Financial Accountability

PIM Public investment management

PIMA Public Investment Management Assessment

PIMI Public Investment Management Index

PIP Public investment programme

PPP Public–private partnership

6 ODI Report


1. Introduction

Public investment in infrastructure is, once again, at the heart of the development agenda. With interest rates continuing to languish at record lows, there is growing support for increased public investment in both the developed and developing worlds to support economic growth (IMF, 2014). There has also been a shift in thinking about the role of the state in infrastructure provision in recent years.

While greater support for private involvement in the provision of infrastructure services has improved operational efficiency, the levels of investment generated have fallen short of expectations (Helm and Mayer, 2016). There is, therefore, greater recognition of the role the state might play in filling this gap.

This is not, however, the first wave of enthusiasm for public investment in infrastructure. Previous surges in capital spending have often led to increased levels of debt without delivering the expected returns to economic growth (Warner, 2014). There is evidence that public investment in infrastructure does not tend to be done particularly well in either developed or developing countries (Flyvbjerg et al., 2002; Ansar et al., 2014), although the problems are often most acute where the infrastructure needs are greatest.

Public investment management (PIM) institutions have come to be seen as the missing piece of the puzzle (Rajaram et al., 2010, 2014; Dabla-Norris et al., 2012; IMF, 2015a). Increasing public investment has the potential to increase economic growth; but the returns to investment depend on the quality of institutions overseeing that investment.

International development actors are, increasingly, supporting programmes to reform ‘public investment management systems’ with a view to improving the efficiency of investment. These reform efforts are being

built, predominantly, around PIM diagnostic frameworks developed by the World Bank and adapted by the International Monetary Fund (IMF). The World Bank has engaged with more than 60 countries on PIM issues in recent years. The IMF, meanwhile, has pledged to increase support for PIM and use its Public Investment Management Assessment (PIMA) tool to help countries identify reform priorities and develop capacity-building strategies (IMF, 2015b). Consultation is now underway on a revised and harmonised public investment diagnostic tool.

The interest of the international community in improving value for money from investment spending is a welcome development. As international organisations look to standardise approaches to measuring PIM, this report provides a timely stocktake of the evolution of ideas in this field and explores how international engagement around PIM could be improved upon in future.

The report is structured as follows.

• Section 2 looks at the origins of the international community’s current interest in PIM and demonstrates the shift in thinking around public investment over recent decades.

• Section 3 looks in detail at the two diagnostic frameworks that have been most widely used in developing countries, the World Bank’s Unified Framework for Public Investment Management and the IMF’s PIMA, setting out their evolution, differences in approach and weaknesses.

• The final section looks ahead, with suggestions on how to maximise the impact of this growing interest in PIM.

2. What is ‘public investment management’?

1 Public investment is sometimes used in a wider sense to mean investment in human capital such as education and health spending, or financial investments by government institutions such as sovereign wealth funds. However, in this paper it is only used in this narrower sense, which is typically measured by general government gross fixed capital formation (in national accounts) or spending on fixed assets (in government finance statistics).

2 See IMF (2014: 92), Table 3.2: Economy Group Composition for details of which countries are in each category. This categorisation ‘is not based on strict criteria, economic or otherwise, and it has evolved over time. The objective is to facilitate analysis by providing a reasonably meaningful method of organizing data’ (ibid.: 161). The categories do not strictly follow the World Bank low-, middle- and high-income country categorisation. There are a number of high-income countries included in the emerging market group of economies and a number of lower-middle-income countries included in the group of developing countries.

Public investment is government spending on the creation or improvement of physical assets, including both economic infrastructure (such as airports, roads, railways, water and sewerage systems, electricity utilities and telecommunications) and social infrastructure (such as schools, hospitals and prisons) (IMF, 2015a).1

This section explores the evolution of ideas around PIM across the international community. It sets out trends in the levels of public investment over time, and shows why public investment is enjoying a renewed emphasis in fiscal policy debates. It then looks at how perspectives on the management of investment have changed; with a shift from a narrow focus on appraisal and the integration of capital and recurrent expenditure to a broader emphasis on the production and management of public assets.

2.1. Changing perspectives on public investment

2.1.1. Fiscal policy perspectives

Interest in the processes for managing public investment has waxed and waned with the level of investment. After booming in the 1970s, public investment fell precipitously in low-income countries in the 1980s, before a partial recovery since 2005 (see Figure 1).

Perspectives on public investment have evolved in line with changes in the development community’s understanding of economic development more broadly. From the 1950s to the 1970s, the quantity of capital investment was seen as central to economic development. For example, the Harrod–Domar growth model, used

Figure 1. Public Investment 1970-2011 (percentage of GDP, weighted by purchasing power parity)2

0

2

4

6

8

10

12

Publ

ic in

vest

men

t as

% o

f GDP

(PPP

wei

ghte

d)

Advanced economies Emerging market economies Developing countries

Source: IMF (2014).

8 ODI Report


widely by development economists, suggested a linear relationship between the amount of capital investment in the economy and the level of growth (Easterly, 1997).3

This growth model was later amended by Walter Rostow (1962) who argued that development was choked off by a lack of savings and that donors could promote growth by closing the financing gap. The focus of the World Bank for its first three decades was on supporting investment in infrastructure (roads, dams, ports, water and sanitation systems) as a critical need for development, with engineers forming a strong core of the Bank’s staff. The World Bank was also an early proponent of social cost–benefit analysis as a way to appraise and select high-return projects. In the 1980s and early 1990s, many governments in low-income countries had invested heavily in infrastructure, but did not have the economic growth to show for it. In many cases, this had led to crippling debt burdens. It was difficult to maintain the view that there was a linear relationship between investment and growth when overwhelming evidence showed that the quantity of capital spending, on its own, could not explain the varied growth levels of different countries (Easterly, 1997).

The consensus view in Washington D.C. was that development failings had been the result of poor policies, rather than insufficient investment (see, for example, Balassa, 1986; Krueger, 1990; Williamson, 1990). Part of this critique related to policies that impeded the effective functioning of markets, but another key component of getting the ‘policies right’ was ensuring fiscal discipline.

International organisations called for fiscal retrenchment as part of structural adjustment programmes, and investment spending was often a relatively easier target for cuts than ongoing government operations. It is easier to reduce capital than current expenditure, simply by allowing capital assets to depreciate more quickly through reduced maintenance expenditure, or by cancelling a few large infrastructure projects (see Hemming and Ter-Minassian, 2004).

By the late 1990s, poverty reduction had become the primary objective of most organisations working in development (Simson, 2012). Advocacy campaigns helped to rally support for increased public spending, which was to be financed in part by debt relief and increased aid flows. Increases in expenditure were to focus on poverty alleviation and, in particular, spending on health, education and other social development goals. These perspectives were summarised in the Millennium Development Goals (MDGs), which set targets for poverty reduction and social development. Donor agencies monitored certain ‘poverty reducing expenditures’ (primarily in the social sectors) in a number of countries that were eligible for debt relief.

The mid-2000s saw a growing critique of the MDGs, which created a ‘false dichotomy’ between poverty

3 This model continues to be used in certain countries where there is a legacy of central economic planning.

reduction and economic growth (Paternostro et al., 2007). A number of authors (Killick, 2004; Paternostro et al., 2007; World Bank, 2005) suggested that a narrow focus on spending in the social sectors might be crowding out spending on infrastructure, as well as on agriculture and private-sector development, that could alleviate poverty by increasing long-term economic growth.

In the 2000s, the fiscal policy agenda shifted away from concerns with macroeconomic stability to how fiscal policy could promote economic growth (World Bank, 2006, 2007). Developing country governments, especially in Latin America, felt the fiscal reforms they were undertaking were too biased towards fiscal restraint. They argued that the macroeconomic stabilisation framework recommended by the IMF placed undue restrictions on their capacity to undertake public investments, at the cost of long-term growth (Rajaram et al., 2014: 4-5).

Concerns over the low levels of public spending in infrastructure have not been limited to low-income and developing countries. Such concerns have been echoed more recently in the debates over whether and how fiscal stimulus could promote growth in OECD countries following the financial crisis of 2008 (Berg et al., 2016). The IMF has also sided with these arguments, stating that ‘the time is right for an infrastructure push’ (IMF, 2014: 75).

2.1.2. Infrastructure policy perspectivesRunning in parallel to these developments were evolving views on the appropriate role of government in the provision of infrastructure services. The 1994 World Development Report noted that ‘[m]ajor investments have been made in infrastructure stocks, but in too many developing countries these assets are not generating the quantity or the quality of services demanded.’ This signalled two major departures from much of the previous infrastructure debate.

• The first departure was the idea that the objective should be the provision of infrastructure services, rather than the construction of infrastructure assets. Building roads or power stations, therefore, is not the goal of public policy; instead, the goal is to improve transportation or access to electricity. Rather than concentrating on individual projects, the report called for a stronger focus on the institutions that regulate and operate the provision of infrastructure services.

• The second key departure was that government institutions and their incentives were seen as being at the root of poor service provision. The proposed solution was to increase private involvement in the financing, ownership and management of infrastructure assets. The report advocated competition, private-sector

management practices and greater use of public–private partnerships (PPPs) to finance infrastructure. The role of government is seen primarily as a policy-maker and regulator, rather than an investor and operator.

The key take-away message was that governments needed to undertake institutional reform and that the private sector and markets would take the lead in filling the gaps in infrastructure service provision (World Bank, 1994).

The optimism of the 1994 World Development Report about private investment has been tempered somewhat in recent years. Despite a wave of institutional reforms that have increased private participation in infrastructure, and resulting efficiency gains, the amount of private investment has fallen short of expectations (Estache and Fay, 2009). This problem is particularly acute in places where the levels of access to infrastructure are the lowest (ibid.).

There is now a greater acknowledgement of the importance of both public and private sectors in infrastructure provision: while there is a more nuanced recognition of the relative strengths of the private sector, few would argue that this sector alone can fill the infrastructure gap. Today’s infrastructure gaps are thus expected to be filled by a mixture of public investment, private investment and public–private investment. Although the Harrod–Domar growth model has been largely consigned to history, the notion of financing gaps continues to play a powerful role in policy debates on infrastructure. The run-up to the Third International Conference on Financing for Development in Addis Ababa in 2015 saw a wave of calculations of infrastructure gaps and calls for different financing sources to fill them. Such calculations have been made for developed, as well as developing, countries.4

2.2. Changing perspectives on the management of public investment

As ideas have evolved on the role of the state and fiscal policy in infrastructure provision, so too have perspectives on how the state can best manage investment. In the 1950s and 1960s, there was limited engagement with the specific processes whereby governments managed their public investment projects. Governments prepared national development plans, targeted capital investment rates were set and donors provided aid to close the financing gap in capital investment.

Today, nobody would argue that it is the quantity of investment spending alone that determines the outcomes; how that money is spent matters too. Running in tandem with efforts to close infrastructure financing gaps has been a growing interest in how to improve the institutions that manage the investment. However, the understanding of which institutions matter and why they matter continues to evolve.

4 See Ruiz Nunez and Wei (2015) for an overview of infrastructure financing estimates and an attempt to calculate this for all developing countries on a consistent basis. Estache and Fay (2009) provide a brief overview of the different methodologies for calculating infrastructure financing gaps, and their limitations.

2.2.1. Investment appraisal and public investment programmes

The late 1960s saw significant efforts to develop social cost–benefit analyses that could evaluate public investments in developing countries (see Little and Mirrlees, 1991, for a useful summary). Originating in welfare economics, these analyses emerged from debates among economists about the best techniques to evaluate economic costs and benefits in developing countries. They aimed, in part, to improve the quality of evaluations undertaken by donors; but efforts were also made to support the use of cost–benefit analysis by developing country governments. Many staff from aid-receiving countries were trained in social cost–benefit analysis techniques during the 1970s and 1980s (Rajaram et al., 2014).

In the 1980s, the World Bank recommended multi-year rolling public investment programmes (PIPs) as a way to link micro-project appraisal with macroeconomic objectives. The idea was to create a pipeline of well-prepared projects that had been appraised using cost–benefit analysis and were ready for selection through the annual budget process (Schiavo-Campo and Tommasi, 1999).

The World Bank also provided analytical support to these programmes through a series of public investment reviews, which reviewed the stock of investment projects to help governments prioritise their limited fiscal resources. By the 1990s, public investment reviews had been replaced by public expenditure reviews in keeping with the broader focus on overall spending and the wider budgetary process.

2.2.2. Medium-term integrated budgeting In the 1980s and 1990s, the attention of the international financial institutions shifted towards the more general fiscal and budgetary problems facing developing countries. This was a result of the decline in public investment, the other economic and fiscal problems many developing countries faced in these ‘lost decades’, and the focus on social spending in the era of the MDGs. A shift away from the provision of finance through specific projects and towards budget support also raised concerns about overall systems of budgetary and fiduciary management. Public investment was not identified as in need of targeted reforms.

The lessons learned from the economic and fiscal problems of the 1980s and 1990s, and the attempts to combat these, were summed up in a series of publications towards the end of the 1990s. The World Bank’s Public Expenditure Management Handbook aimed to ‘[provide] a framework for thinking about how governments can attain sound budget performance and [give] guidance on the key elements of a well-performing public expenditure management system.’ (World Bank, 1998: 1.) These handbooks can be understood as part of the

10 ODI Report


‘post-Washington Consensus’ and the broader recognition that ‘institutions matter’ for the provision of services.

It is worth noting that none of the key ‘weaknesses in resource allocation and use’ identified in the Handbook relate specifically to public infrastructure investment, although several more general problems would clearly have an impact upon such investment5 (World Bank, 1998: 5). Similarly, Schick (1998) devotes no special attention to PIM, instead focusing on achieving the objectives of aggregate fiscal discipline, allocative efficiency and operational efficiency. Public investment was not identified as posing any special challenges, but was subsumed under these broader notions of allocative and operational efficiency.

The key problem identified as affecting public investment was the practice of ‘dual budgeting’,6 where recurrent and development budgets are prepared separately. While the aim of capital budgets and PIPs was to provide a mechanism to manage investment projects more efficiently, in practice they have been associated with several budget weaknesses.

There is often, for example, a lower hurdle to incorporate projects presumed to be ‘productive’ in the development budget than to approve increases in recurrent spending that are presumed ‘unproductive. This leads to a form of ‘opportunistic budgeting’, where current expenditures are labelled as capital investments (Schick, 1998: 68), and to an expansionary bias towards spending, as new projects are inserted without considering their future and recurrent implications. PIPs, therefore, tended to ‘neglect medium- to longer-term operations and maintenance expenditure needs’ (Brumby et al., 2013: 578.).

The ‘inexorable growth of “investment projects” also results in an ‘annual budget process [that] allocates extremely limited domestic resources to keep too many projects and activities alive’ (World Bank, 1998: 5). PIPs tended to be too bottom up (i.e. driven by the process of selecting projects) and not sufficiently constrained by top-down resource limits.

This problem of dual budgeting was just one example of, and perhaps ‘the single most important culprit in’, the broader failure to link policy, planning and budgeting (World Bank, 1998: 56). Dual budgeting was, in turn, identified as perhaps ‘the single most important factor contributing to poor budgeting outcomes at the macro, strategic and operational levels in developing countries’ (World Bank, 1998: 31).

5 For example: poor planning; no links between policy-making, planning and budgeting; inadequate funding of operations and maintenance; little relationship between budget as formulated and budget as executed; and unreliability in the flow of budgeted funds to agencies and to lower levels of government. See World Bank (1998: 5), Box 1.

6 Similarly, Schick (Schick, 1998:39) identifies ‘enclave budgeting’ as one of the key allocative efficiency problems in developing countries: attempting to protect certain priorities by establishing special funds, separate investment budgets and other devices that wall off ‘enclaves’ from the rest of the budget.

7 Some PIM issues covered in the Public Expenditure Management Handbook are project selection (criteria for selection) and review (at appraisal stage), but these are not discussed extensively.

As a result, PFM reforms focused not on public investment in particular, but on efforts to solve this more general problem of linking policy, planning and budgeting. Medium-term expenditure frameworks and centrally set top-down budget ceilings were championed to integrate both capital and recurrent expenditure and bring a more medium-term, resource-constrained and policy-based perspective to budgeting.7 The role of the centre was to set the overall spending limits and ministries were to prioritise across both recurrent and capital spending. PIPs were still prepared and could serve as useful informational tools by describing all the projects that were underway in the line ministries. However, they were no longer supported by international organisations as the main link between public spending and development objectives.

2.3. The rise of public investment management

2.3.1. The particular challenges of public investmentGovernment objections to severe fiscal restraint have often been countered by strong arguments that increased public investment may lead to waste. Pritchett (2000) pointed out that spending inefficiencies mean that the return from a dollar of spending on infrastructure is not necessarily equivalent to an increment of a dollar in the public capital stock. This problem is likely to be particularly acute in those developing countries where infrastructure is most needed, but institutions are weakest. There is, therefore, a risk that countries could find themselves saddled with debt as a result of infrastructure investment, but without the expected improvements in infrastructure services.

The policy importance of more efficient investment and of addressing the challenges to such investment have given rise to a burgeoning literature on PIM. This recognises explicitly that investment spending is ‘different’ and may require specific management processes (Fainboim et al., 2013).

Spending on large-scale public investment projects, for example, often has large budgetary implications. Capital investment is often ‘one off’ (making it hard to estimate costs) and technically complex to manage, both in terms of procurement and implementation. Where the costs of projects overrun, they can create considerable fiscal risks for a government. Spending on investment is generally ‘lumpy’: projects tend to need significant up-front financing rather than smooth and regular payments over their lifespan. Public investment also creates lasting assets

that have to be maintained. This means that decisions made today on whether to go ahead with a project have implications for future financing obligations related to operation and maintenance.

Large-scale investment projects are not only technically difficult to manage, but they also pose particular political problems. Infrastructure is built in specific locations and neither the costs nor the benefits of investments are evenly spread. Large dams, for example, may require the resettlement of citizens. There is often strong resistance to projects that might increase air and noise pollution. At the same time, investments can benefit local economies in the form of greater employment, for example, and local politicians may compete to attract investment to win votes; even if the returns to other investments might be greater from a national perspective. Reconciling these divergent political interests poses a real challenge for effective investment in infrastructure.

Public investment is also particularly prone to corruption and waste. Where politicians cannot make credible commitments to their potential supporters, political support is gained by clientelism, such as spending on narrowly targeted public infrastructure and excessive rent-seeking, or what has become known as ‘pork barrel’ politics.8 This logic also applies to ‘white elephants’ – investment projects with costs that massively exceed their

8 The appropriation of government spending for local projects secured solely or primarily to bring money to a political representative’s district.

9 The projects are located in 20 countries on five continents, including both developed and developing nations, and were completed between 1927 and 1998.

10 All the projects studied in developing countries were railway projects. Average cost escalation for railway projects in Europe was 34.2%, and 40.8% in North America.

over-estimated benefits. Inefficient projects demonstrate the credibility of politicians’ promises to their supporters, who know full well that these loss-making projects would not be maintained by opposing politicians: ‘it is not just that politicians are bad at picking winners, they actually pick known losers’ (Robinson and Torvik, 2005).

An emerging body of empirical evidence also suggests that investment spending is not being particularly well managed in either developed or developing economies. In a sample of 258 transportation projects in both developed and developing countries, Flyvbjerg et al. (2002) found evidence of persistent time delays and average cost overruns of 27.6%, as well as over-estimation of benefits.9 These overruns rose to 64.6% for projects in developing countries (although the sample of just 16 projects was much smaller and all were railways).10 A more recent study of 245 large hydropower dams in 65 countries found cost overruns averaging 90% (Ansar et al., 2014). The Construction Sector Transparency Initiative (CoST, 2011) has also reviewed cost and time overruns in a number of developed and developing countries, as shown in Figure 2.

2.3.2. A public investment management systemRajaram et al. first conceptualised the idea of a PIM system in their ‘public investment management diagnostic framework’ (Rajaram et al., 2010, 2014). Their approach

Figure 2. Time and cost overruns for investment projects in selected countries

0 20 40 60 80 100 120 140

Ethiopia

Average overrun (%)

Tanzania

Malawi

Zambia

Viet Nam

Philippines

Guatemala

UK

Cost overrun Time overrun

Source: CoST (2011).

12 ODI Report


draws together and builds upon three previous strands of thinking on investment management:

• the project appraisals in the 1970s, emphasising the importance of linking investment choices to policy and high-quality investment appraisal for selecting efficient projects,

• the public expenditure management literature of the 1990s, acknowledging the importance of integrated budgeting to ensure that sufficient resources are provided not only for the construction of assets, but also for their maintenance and operation, and

• infrastructure policy debates, noting that the returns to investment depend not only upon the construction of the asset, but also on how that asset delivers services. The innovation of this framework is to bring these

different strands together and conceptualise PIM as a system to produce productive public assets, from the planning of an asset right through to its operation. It

recognises that decisions taken throughout the whole life cycle of an asset affect the efficiency of investment, and examines these processes as a PIM system, rather than simply examining appraisal procedures or the management of the budget in isolation.

2.4. SummaryThere has been a resurgence in interest about the role of public investment in promoting economic growth, as shown by rising levels of public investment in developing countries. However, similar surges of public investment in the past have seen wasteful spending, rising debt and little of the anticipated economic growth.

Today, there is far greater interest in the literature not just in the quantity of investment spending, but also in how that spending is managed. The idea of a PIM system has emerged to improve returns to public investment through institutional reforms. The literature on this system fills a gap in the literature on public expenditure management in the late 1990s about the specific institutional procedures needed to manage capital spending.

3. What do public investment management diagnostics tell us?

11 PEFA has become the standard tool for assessing the overall quality of PFM systems. It consists of 31 performance indicators across seven broad areas of the PFM system: Budget reliability; Transparency of public finances; Management of assets and liabilities; Policy-based fiscal strategy and budgeting; Predictability and control in budget execution; Accounting and reporting; External scrutiny and audit.

12 Performance indicator 11 covers PIM, with dimensions that include the economic analysis of investment proposals, investment project selection, investment project costing and investment project monitoring.

3.1. IntroductionOne distinctive feature of the literature on PIM is that it is based predominantly around certain ‘diagnostic frameworks’. These aim to assess the quality of a PIM system, to identify the processes or institutions that are weak and to suggest potential reforms (Rajaram et al., 2010).

The World Bank’s diagnostic framework for PIM was the first instrument to explore these issues, but it has been followed by many other tools. In 2015, the IMF launched its PIM assessment at the Financing for Development conference in Addis Ababa. The OECD has also developed a self-assessment tool for OECD countries (OECD, 2015). The revised Public Expenditure Financial Accountability (PEFA) assessment11 framework launched in 2016 also introduced a new pillar to assess the management of assets and liabilities, which includes new indicators to assess PIM.12 A revised PIM assessment (PIMA) is also being prepared, which aims to harmonise these various frameworks.

This section examines how these frameworks have evolved, focusing on the World Bank and IMF frameworks as these have been most commonly used in developing countries. It provides an overview of these diagnostics, looking at how they conceptualise PIM and the key features that affect the efficiency of investment. It also explores differences in the scope and diagnostic approaches of these frameworks.

3.2. The scope of public investment management diagnostics

3.2.1. A focus on the institutions that underpin government-funded investment

The concept of a PIM system is still relatively new and fluid. Not surprisingly, the diagnostics used to assess these systems are also still evolving. Nevertheless, they have a clear focus on the institutions that manage the direct government financing of infrastructure.

PIM diagnostics are concerned with the institutions that underpin the decisions made about government-funded investment. It is worth noting that public spending is not the only policy tool that governments can use to influence the quality of infrastructure services. In certain sectors, such as mobile telecommunications, the government plays primarily a regulatory role to influence the quality of private provision of services.

The IMF’s PIMA does make reference to the way in which infrastructure companies are regulated. However, for the most part, these instruments are not concerned with how regulatory frameworks can support privately financed investment or the operation of assets.

PIM diagnostics have emerged primarily from a desire to improve value for money in public spending on infrastructure. However, the efficiency of investment spending is not only a function of government institutions. In many low- and middle-income countries, supply-side constraints in the local construction sector can be a major source of inefficiencies and bottlenecks in public investment. Scarce technical and managerial resources in the private sector are fixed in the short run.

14 ODI Report


Where the constraints on these resources are binding and no substitutes can be found, increasing the amount of investment spending leads to increased investment costs without increasing the quantity of assets produced. Efforts to scale up investment, therefore, become increasingly inefficient (Collier, 2009).

Certain governments have made active efforts to promote the productivity of the construction sector as part of a wider industrial strategy. However, this is not within the scope of this type of diagnostic.

3.2.2. World Bank diagnostic framework: a system linked to the life cycle of an asset

The initial World Bank diagnostic framework for public investment was structured around the life cycle of an asset. It explored what it might take for any country’s PIM system to function effectively: that is to say, first, to select good projects for public funding and, second, once they are selected, to implement and operate investments effectively. Here, the approach identifies eight ‘must-have’ features for the effective functioning of any PIM system (summarised in Figure 3).

1. Investment guidance, project development and preliminary screening: Governments should provide broad strategic guidance for public investment projects and programmes. There should be a first level of screening of project concepts to ensure they are consistent with these priorities, and those that are not consistent should be rejected.

2. Formal project appraisal: Following the initial screening of projects for consistency with strategic priorities, there should be more rigorous appraisal of the cost–benefit ratio or cost-effectiveness of projects. The decision whether or not to proceed with a project should be based on formal and well-publicised technical guidance.13

3. Independent review of appraisal: This is needed as a check on the optimism bias (underestimated costs and overestimated benefits) that is prevalent in project proposals, especially when these are prepared by the line ministry or agency that will implement the project. There should be a formal review process, which can be undertaken by the ministry of finance, or by an agency with an arm’s-length relationship to government, such as a university or a policy research institute.

4. Project selection and budgeting: This is the point at which projects are incorporated into the budget and government funds are allocated to projects. The fiscal

13 This process should include a pre-feasibility study that identifies relevant alternatives before a full feasibility study. This should include cost–benefit or cost-effectiveness analysis, preliminary design and environmental and social impact assessments. The evaluation techniques used in this process should be appropriate to both the size of the project and to the level of government capacity.

14 Where the budget process is annual, it is particularly useful to have a total project cost management system. It does not necessarily have to be linked to a multi-annual budgeting system.

framework and the annual budget should determine the overall envelope for public investment, and the recurrent budget should adjust to provide future recurrent funding to operate and maintain the new assets created by the selected projects.

5. Project implementation: Once projects are selected and budgeted, they need to be completed on time and on budget. While accurate cost estimates at the project appraisal stage are clearly essential, governments should also take other steps to implement projects efficiently and prevent time and cost overruns. These should include: a total project cost-management system and/or multi-year budgeting for complex projects14 to anticipate and control the budgetary needs throughout implementation (rather than accounting by separate contracts or stages and tracking against annual appropriations); arrangements for efficient and fair procurement; and guidelines, plans and capacities to manage and monitor project implementation.

6. Project adjustment: Governments should have an active review process to monitor project implementation and address problems. A project-review process should typically be part of the annual budget process, where implementing agencies submit progress reports. The review process should have some flexibility to allow changes in the disbursement profile to take account of changes in project circumstances and, potentially, to allow reallocations of funding across projects that are moving at different speeds. This should also include a mechanism to restructure or even halt a project if costs escalate or time delays reach a point where the project is no longer beneficial.

7. Facility operation: Once a project is completed, there should be a process to ensure that the facility is ready for operation and that services can be delivered. Asset registers should be maintained and asset values recorded and updated.

8. Project evaluation: There should be a basic completion review for all projects, setting out whether the project was finished within the specified budget and time frame, and whether the outputs were delivered as specified. For a smaller sample of projects, there should also be more in-depth evaluation that compares the costs and benefits of the project with those in the project design. These evaluation processes enable learning and feedback that can support improvements in investment management over time.

These ‘must-have’ features signal a clear departure from the largely public finance-oriented reforms supported by external actors in the late 1990s. Their support for institutional reform aimed to put in place resource-constrained, medium-term expenditure frameworks, in the expectation that sound, prioritised expenditure policy would follow. At the same time, it was assumed that performance budgeting techniques would take care of value for money.

In Rajaram et al.’s (2010) conceptualisation of a PIM system, however, the budget plays a critical but supportive role in a wider set of institutions required to develop and implement good policy. Projects should only be financed when enough funding is available, but decisions on whether projects should be approved as being eligible for public financing are managed explicitly outside the budget process.

3.2.3. The IMF approach: a greater focus on budgetary institutions

The framework for the IMF’s PIMA was developed initially as part of a study that tested and quantifed the potential benefits of ‘Making Public Investment More Efficient’ (IMF, 2015a). Since that time, the framework has been used in a number of countries to undertake dedicated assessments.

Certain indicators used to assess the health of PIM institutions are relatively consistent with the ‘must-haves’ set out by the World Bank. For example, the PIMA looks

at national and sectoral planning, the appraisal of projects, the vetting of appraisals, the budget process, project management and procurement (see Box 1).

Throughout the assessment, however, there is a much stronger emphasis on budgetary and fiduciary concerns than seen in the World Bank’s framework. Indeed, the IMF describes PIM institutions as a sub-set of budgetary institutions (although it is unclear how some of the indicators like ‘project management’ are consistent with that definition). The assessment looks, for example, at the comprehensiveness and unity of the budget, the rules related to the transfer of items from one financial account to another (virements) and the carry-over of funds. It also examines how assets are accounted for, but not necessarily how they are operated.

More consideration is also given to the institutional features that might be needed to ensure the fiscal sustainability of overall investment spending. There are measures that look at fiscal rules, the management of fiscal risks in PPPs, oversight of sub-national government borrowing and the financial performance of state-owned enterprises. In this respect, PIMA aims to forge a link between overall concerns about the relationship between investment and debt sustainability and the factors explored by Rajaram et al. that affect project-level performance (IMF, 2015a). More generally, the assessment demonstrates continuity with the sorts of issues set out in the public expenditure handbooks of the late 1990s.

Figure 3. Key features of a public investment management system

Consistency in project preparation

1 2 3 4 5 6 7 8

Authority to screenand reject projects

An effective budget and procurement process to support implementation and operation

Key to credible selection

Guid

ance

Appr

aisa

l

Inde

pend

ent r

evie

w

Sele

ctio

n

Impl

emen

tatio

n

Adju

stm

ent

Oper

atio

n

Eval

uatio

n

Link to a development strategy

Maintain assetregister, operate

and maintain asset

Evaluation toimproveguidance

Source: Rajaram et al. (2014: 22).

16 ODI Report


3.2.4. Risks of omitting ‘downstream’ constraints to efficient investment

This positioning of PIM as a subset of public financial management has gained some currency: the most recent update of the PEFA framework, for example, introduces an indicator on PIM. However, framing PIM as a subset of

budgetary institutions runs the risk of overlooking critical ‘downstream’ aspects of the implementation, maintenance and operation of investment assets that are connected loosely to issues of budgeting and fiduciary management.

These ‘downstream’ factors that affect the efficiency of investment feel relatively underdeveloped in existing

Box 1. International Monetary Fund’s Public Investment Management Assessment

The IMF’s Public Investment Management Assessment (PIMA) measures the strength of 15 institutional elements needed for the effective management of public investment, across the three stages of planning, allocation and implementation.

Planning sustainable levels of public investment requires institutions that ensure public investment is fiscally sustainable and there is effective coordination across sectors, levels of government and between the public and private sectors. The PIMA assesses whether countries have:

• fiscal principles or rules that ensure overall levels of public investment are adequate, predictable and sustainable

• national and sectoral plans that ensure public investment decisions are based on clear and realistic priorities, cost estimates and objectives for each sector

• central–local coordination arrangements that integrate public investment plans across levels of government, provide certainty about funding from the central government and ensure sustainable levels of sub-national borrowing

• management of public–private partnerships, which ensures effective evaluation, selection and monitoring of PPP projects and liabilities

• regulation of infrastructure companies, which promotes open and competitive markets for the provision of infrastructure services, objective pricing of infrastructure outputs and effective oversight of infrastructure company investment plans.

Allocation of investment to the right sectors and projects requires comprehensive, unified and medium-term capital budgeting, as well as objective criteria and competitive procedures for the appraisal and selection of particular investment projects. The PIMA assesses whether countries have:

• multi-year budgeting that provides transparent and predictable levels of investment by ministry, programme and project over the medium term

• budget comprehensiveness that ensures all public investment, regardless of the funding channel, is authorised by the legislature and disclosed in the budget documentation

• budget unity that ensures decisions about individual projects take account of both their immediate capital cost and future operating and maintenance costs

• project appraisal that ensures project proposals are subject to a published appraisal using a standard methodology and taking account of potential risks

• project selection that ensures projects are vetted systematically, selected on the basis of transparent criteria and included in a pipeline of approved projects.

Implementation of projects on time and on budget requires institutions that ensure projects are fully funded, transparently monitored and effectively managed. The PIMA assesses whether countries have:

• protection of investment to ensure that project appropriations are sufficient to cover total project costs and cannot be diverted at the discretion of the executive

• availability of funding that allows for the planning and commitment of investment projects based on reliable forecasts and timely cash flows from the treasury

• transparency of budget execution that ensures major investment projects are tendered in a competitive and transparent process, monitored during project implementation and independently audited

• project management that identifies an accountable project manager working in accordance with approved implementation plans, and that provides standardised procedures and guidelines for project adjustments

• monitoring of public assets that ensures assets are properly recorded and reported and that their depreciation is recognised in financial statements.

Source: IMF (2015a: 20-22).

diagnostic frameworks compared with the ‘upstream’ processes of planning and appraisal. The World Bank’s diagnostic framework for PIM unpacks ‘must-have’ requirements to screen projects, appraise projects and have appraisals independently reviewed before the selection of projects. At the same time, however, issues of project management, procurement, management of contracts and the monitoring of project performance are grouped under a single ‘must-have’ feature for project implementation. The questions asked to support a diagnostic assessment of project implementation are also considerably lighter than in upstream areas.15

The IMF’s PIMA framework is even more strongly weighted in favour of the upstream elements. Table 1 compares the IMF PIMA and World Bank framework for PIM and shows that measures related to project adjustment and facility operation (beyond accounting for assets) are not really covered in either of them. The PIMA also lacks a review of processes to evaluate and learn from past projects.

Linked to this, while finance ministries are understandably concerned about inefficiencies in public spending, ‘budgetary institutions’ may not be the binding constraints to greater investment efficiency. For example, a World Bank (2012) study in Indonesia looked at a sample of projects over a period of two years and followed the processes through which their budgets were created, contracts were issued and projects were implemented. The study showed that processes for land acquisition were the critical point of project delay, rather than the processes overseen by finance ministries.

3.3. Approaches to diagnostics

3.3.1. The World Bank’s ‘must-have’ institutional features

The World Bank’s diagnostic framework for PIM is based on the premise that eight ‘must-have’ institutional features are required for the effective functioning of a PIM system. Rajaram et al. do not prescribe targeted levels of sophistication for each of these ‘must-have’ features, but suggest that a certain level of basic functionality is required in each of these eight areas:

‘With regard to the ‘must have’ features, the emphasis is on the basic processes and controls (linked at

15 The diagnostic questions for evaluating project implementation in the World Bank framework for PIM are as follows.

• What is the completion rate of the PIP (annual average over the past five years), defined as the annual public investment budget divided by the estimated cost to complete the current PIP? How does this completion rate differ across key sectors – education, health, water supply and sanitation, roads and power – for example?

• Do ministries undertake procurement plans in line with good practice (for example, using competitive tendering)? And, if so, do they implement procurement plans effectively?

16 ‘Climbing the fence’ refers to situations where projects receive financing through the budget without following agreed procedures for project appraisal.

appropriate stages to broader budget processes) that are likely to yield the greatest assurance of efficiency in public investment decisions. The approach does not seek to identify best practice, as exemplified perhaps by a sophisticated high-level OECD or Chile-like system, but rather to identify the bare-bones institutional features that would minimize major risks and provide an effective process for managing public investments. (Rajaram et al., 2010: 4)’

The World Bank framework allows a certain degree of flexibility to explore potential constraints around each of the eight ‘must-haves’. A set of open questions at each stage of the framework assesses how well the investment management system is functioning. Questions are then designed to explore, for example, whether appraisals are resulting in any projects actually being rejected, or whether certain projects can avoid formal processes by ‘climbing the fence’.16

Table 1. Key components of public investment management diagnostics

IMF PIMA World Bank framework

A. Planning sustainable levels of public investment1. Fiscal principles or rules2. National and sectoral planning3. Central–local coordination4. Public–private partnerships5. Regulation of infrastructure

companies

B. Ensuring public investment is allocated to the right sectors and projects 6. Multi-year budgeting7. Budget comprehensiveness8. Budget unity9. Project appraisal10. Project selection

C. Implementing projects on time and on budget11. Protection of investment12. Availability of funding13. Transparency of budget execution14. Management of project

implementation15. Monitoring of public assets

Eight PIM ‘must haves’1. Investment guidance, project

development and preliminary screening

2. Formal project appraisal3. Independent review of

appraisal4. Project selection and budgeting5. Project implementation6. Project adjustment7. Facility operation8. Basic completion review and

evaluation

18 ODI Report


This framework was used widely by the World Bank to structure country-level dialogue, lending and analytical work on PIM, with diagnostics studies often undertaken in conjunction with public expenditure reviews. Many of the findings of this work were synthesised into a broader guidebook on PIM (Rajaram et al., 2014), which set out certain emerging themes (e.g. the importance of appraisal and procurement) and lessons for country reforms.

3.3.2. The IMF’s introduction of a standardised indicator framework

The IMF’s initial PIMA framework, in contrast, measures the quality of PIM institutions by identifying ‘three key design features’ for each of the 15 ‘PIM institutions’ it includes (as shown in Box 1: five each across the key stages of planning, allocating and implementing public investment). The quality of each of these key design features is scored (fully met, partly met or not met), both in terms of the presence of the institution (its form) and whether it is working effectively (its function).

The PIMA framework builds upon the work of Dabla-Norris et al. (2012), who developed an index of PIM institutions (the Public Investment Management Index, PIMI), which allowed for ‘benchmarking across regions and country groups and for nuanced policy relevant analysis and identification of specific areas where reform efforts could be prioritized.’

The parameters and measurement of PIM institutions indicate a wider push to create standardised indicators of the quality of public sector institutions: most notably through PEFA, but more recently through the Debt Management Performance Assessments and the Tax Administration Diagnostic Assessment Tool.

Part of the motivation for coming up with such indicators is that it allows for cross-country statistical analysis. PIMA scores have been used to establish an empirical relationship between certain ‘PIM institutions’ and the efficiency of public investment (IMF, 2015a).

Making Public Investment More Efficient examined how the overall scores related to public investment performance.17 18 The PIMA results have a high correlation with country income, as shown in Figure 4. PIMA has documented large differences in the efficiency of public investment across countries, defined as how well a certain level of public capital stock translates into effective access to infrastructure, such as roads, water or electricity.

The IMF’s analysis also found that a higher PIMA score was associated with greater efficiency of investment, lower levels of public investment and lesser reliance

17 The countries in the sample are: seven advanced economies (Finland, Germany, Japan, South Korea, Spain, the United Kingdom and the United States); nine emerging markets (Algeria, Brazil, Chile, India, Jordan, the Philippines, Qatar, Romania and South Africa); and nine low-income developing countries (Bolivia, Cambodia, Ethiopia, Ghana, Nepal, Niger, Senegal, Sri Lanka and Uganda).

18 Measured as the absolute difference between the budgeted and actual levels of government capital expenditure.

on PPPs. There were also more stable levels of overall investment and greater stability of the sectoral composition of investment, more credible capital budgets and lower perceived levels of rent-seeking and corruption. Stronger PIM implementation institutions were also found to reduce the volatility of public investment.

Significant differences were found in how countries at different income levels were performing on different measures of institutional strength (Figure 5). After years of donor support for development of plans and medium-term expenditure frameworks, low-income countries and emerging markets received higher ratings for preparing plans and multi-year budgeting than advanced economies; while low-income countries were found to lag significantly behind advanced economies when it came to actually making funds available and managing the implementation of projects (IMF, 2015a: 23).

Figure 4. Public Investment Management Assessment score by country income

10

9

8

7

6

Mean

AEs EMs

Inst

itutio

nal s

tren

gth

5

4

3

2

1

0

LIDCs

Source: IMF (2015a). AE = advanced economy, EM = emerging market;

LIDC = low-income developing country. The boxes show the spread be-

tween the maximum and minimum scores and the mean. ‘Institutional

Strength’ refers to the IMF’s PIMA score which is measured using an

index with the range 1 to 10.

However, a comparison of the PIMA with the PIM index PIMI developed several years earlier (Dabla-Norris et al., 2012) suggests caution in placing too much weight on existing scores. Despite being an attempt to measure very similar processes, the PIMA scores and the PIMI scores are not well correlated, with an R2 of only 0.24.19 In addition, the IMF reports that a principal component analysis found that the results of the two measures were not correlated (IMF, 2015a: 57). This means, essentially, that the indicators that explain the main sources of variation between the countries in the two indexes are not correlated with each other. This means that the two indexes give different explanations for variations in the measured quality of PIM across countries.

One key problem for this comparison is that the PIMA and the PIMI have an overlap of only 10 countries.20

However, it is concerning that two indexes with similar aims have such a low correlation, even for such a small group of countries.

As described above, these quantitative indexes attempt to take complex processes and procedures and then, subjectively, score them. The lack of correlation between

19 In statistical analysis, the coefficient of determination is denoted as R2 and is used to give an indication of the goodness of fit of a model. An R2 close to zero suggests a weak fit.

20 Bolivia, Brazil, Cambodia, Ethiopia, Ghana, Jordan, Philippines, Senegal, South Africa, Uganda.

21 This is a possibility as the initial assessments used in these indices were based on questionnaires rather than in-depth country-level assessments.

the two suggests that either the subjective scoring on one or both of the indexes is not reliable (measurement error)21 or that the indexes are measuring processes and institutions that are sufficiently different to result in different scores. This latter case would indicate that there is a lack of agreement about which processes are important for effective public investment.

3.3.3. Challenges of quantification related to standardised indicator sets in diagnostics

There is a wide literature that critically examines the growing number of governance indicators. Such indicators are often criticised for being overly normative (Davis et al., 2012) or too subjective (Thomas, 2010; Langbein and Knack, 2010).

A fundamental problem with many diagnostics is that they provide common indicators to measure a country’s system against an ideal-type system. Therefore, public sector diagnostics do not start by asking ‘What are the most serious constraints to a pre-specified policy problem?’ (e.g. constraints to growth in Rodrik, 2010), but start instead by asking ‘How far is the public sector from the

Figure 5. Public investment institutional scores by country group

1. Fiscal rules2. National and sectoral planning

3. Central–local coordination

4. Management of PPPs

5. Company regulation

6. Multi-year budgeting

7. Budget comprehensiveness10. Project selection

11. Protection of investment

12. Availability of funding

13. Transparency of execution

14. Project management

15. Monitoring of assets

8. Budget unity9. Project appraisal

6–10 Allocating

1–5 Planning11–15 Implementing

AEs EMs LIDCs

Source: IMF (2015a: 23).

20 ODI Report


idealised system embodied in our diagnostic framework?’ The resulting assessment is, in effect, a ‘gap analysis’ of the actual system relative to the ideal-type system embodied in the diagnostic framework. The largest gaps from the ideal-type system may not in fact also be the most pressing problems in how a PIM system operates in a specific country.

Furthermore, once indicators become important, they may generate incentives for ‘gaming’, where governments (and their partners) target superficial changes that will show up as improved rankings in order to gain external legitimacy. This then reduces the usefulness of such rankings as they no longer measure true differences in how institutions function across countries (Høyland et al., 2012). As international organisations move towards agreement on a standard set of indicators to diagnose PIM systems, this can provide incentives for governments to try to ‘catch up’ with their neighbours and spur reforms. But there is also a risk of diverting focus (and incentives) away from the outcomes desired – such as an improvement in the competitiveness of procurement – and instead focusing solely on how to close the gap between the ideal-type system and the actual system.

This makes it all the more difficult to prioritise reform: instead of focusing on the country-specific challenges and on institutional arrangements that are likely to have the largest impact on infrastructure outcomes for that specific setting, reform may instead focus on the areas where it is

easiest to improve the scores. If these are not the same, and countries need to show donors or other outsiders that they are making progress, there is a strong incentive to divert attention away from effective reforms to focus on easier reforms that may have less impact.

3.4. SummaryPIM diagnostics have played a very useful role in improving understanding on the potential drivers of inefficient investment. The World Bank’s diagnostic framework showed how the allocative and operational efficiency of investment spending is a function of a broader set of institutions than systems of public financial management.

There are, however, two clear directions in the way that these instruments are evolving.

• First, there is a shift of focus towards seeing PIM as a subset of budgetary institutions. This risks diverting focus away from critical downstream processes that can affect the value for money achieved by investment spending.

• Second, there has been a move towards developing standardised indicators to be used as part of a diagnostic framework. These are likely to have an impact on incentives for reforms.

4. Where next for international engagement in this area?

This final section looks ahead and considers how international support for PIM is likely to evolve. It sets out:

• potential risks arising from the move to a standardised indicator-based diagnostic and some suggestions on how to mitigate those risks

• considerations for using diagnostics to support reforms; • some existing gaps in knowledge that should be

addressed by further research • lessons for other sectors that could be drawn from the

evolving debate on PIM, and • some principles on how diagnostics might be applied at

the country level.

4.1. Emerging risks and measures to mitigate those risks

4.1.1. Getting value for money from public spending on infrastructure is not just about public investment management institutions

There is a risk that reforms to a fixed set of institutional features agreed by international organisations in a standardised diagnostic framework becomes the default approach of the international community to improving value for money in infrastructure. Not all problems of spending efficiency are related to the specific government institutions that frame the selection and execution of investment. This is particularly true if PIM is increasingly seen as simply a subset of budgetary institutions.

Value for money may also be constrained by limitations on the supply side (e.g. insufficient engineers, market structures for suppliers of assets, etc.). The kind of institutions commonly identified in the literature on industrial policy may be more relevant for alleviating these kind of constraints than the institutional features scrutinised in the PIM diagnostics. Collier (2009: i120), for

example, documents how the planning ministry worked with the construction sector in Botswana to alleviate supply-side constraints:

‘The rapid growth of government revenue from diamonds and the resulting scope for public investment created massive demands for construction, but this occurred in a context of initially very limited domestic capacity. In addition to a rolling overall economic development plan, the government had an annual plan-within-a-plan specifically for the construction sector. Each year it would extract from the overall plan the implications for the construction sector, convene the local construction firms, and discuss whether implementation was feasible. If it was judged infeasible, the government would ascertain the problems facing local construction firms, attempt to alleviate them, and also revise the plan so that it was consistent with local capacities. It also flattened the supply curve by the use of foreign construction firms and imported skills.’

Similarly, Rodrik (1995) argues that South Korea’s economic success owes much to government interventions to solve investment coordination failures. Warner (2014) suggests that large returns to infrastructure investment in South Korea in the 1960s were the result of investment that responded to bottlenecks identified through close collaboration between government and the private sector. Elsewhere, a government’s understanding of the needs of the private sector, without being ‘captured’ by that sector, has been called ‘embedded autonomy’ (Evans, 1995).

Strengthening a pre-defined set of ‘public-investment management institutions’ may not then always be the best way to improve returns in investment spending. Bottlenecks can arise from the supply side of infrastructure too. This then calls for efforts to strengthen public investment to be part of a broader discussion on how

22 ODI Report


to improve infrastructure outcomes. Infrastructure specialists and engineers need to be involved as much as fiscal economists. It may also involve developing a better understanding of how the construction industry is functioning and its relationship with government (section 4.2 provides some examples of this type of engagement in the context of Uganda).

4.1.2. Crowding out other analytical workA diagnostic of PIM institutions is one of a number of analytical tools that can be used by international actors to support more efficient investment. However, caution may be needed to ensure that other analytical work is not crowded out by standardised PIM diagnostics.

External actors, can, for example, provide useful analysis of the efficiency and equity of spending policies in infrastructure sectors: indeed the World Bank has undertaken a number of diagnostics in conjunction with public expenditure reviews. Studies such as the Africa Infrastructure Country Diagnostic (Foster and Briceño-Garmendia, 2010) are also useful for providing a sense of the constraints to improved infrastructure outcomes more broadly. There may also be a role for distinct analytical products where state-owned enterprises or development banks play a major role in promoting infrastructure investment.

Attempts to measure the performance of institutions should not crowd out direct measures of the performance of projects. Pulling together data on contracts as well as the costs and benefits of projects internationally is a global public good that has the potential to support good preparation of projects worldwide. Flyvbjerg (2008) and Kahneman (2011) have emphasised the importance of evidence if project appraisals are to counter ‘optimism bias’.

There is also scope to use contractual data for experiments on the effectiveness of different procurement mechanisms. Lewis-Faupel et al. (2014), for example, found that the greater transparency provided by e-procurement improved quality and reduced time overruns in India and Indonesia by reducing the cost of acquiring tender information. At the same time, the increased number of suppliers reduced the potential for collusion between contractors and government officials.

Similarly, local government e-procurement in Bangladesh led to cost reductions of around 12% (Abdallah, 2015).

Existing diagnostics have provided plenty of insight on common institutional weaknesses, but there is less information on how countries have found solutions to problems over time. The World Bank’s The Power of Public Investment Management provides a useful stocktake of the Bank’s own assessments and what comes across clearly is that there are recurring problems that seem to emerge across countries of similar types (Rajaram et al., 2014). Synthesising knowledge on how countries have solved some of these problems could help to generate ideas and menus of options for other governments grappling with common issues. In many low-income countries, for example, absorptive constraints are identified as a key barrier to investing more and to getting good value for money from that investment. It would be useful to understand how countries have alleviated these constraints.

More could also be done to build up a knowledge base around the institutional features that support effective execution of investments. This could include developing a set of questions to better understand downstream processes such as systems of project management, procurement and contract management. Processes for acquiring and managing land and undertaking social and environmental safeguard assessments can also have a major bearing on the value for money from investments, but have not been explored to the same extent.

4.1.3. Keeping the focus on how systems work and not just how they look

As indicators to measure investment management become standardised, there is a need to ensure that diagnostics continue to identify functional weaknesses in systems, and not just institutional forms. A good understanding of whether systems to select and implement projects are actually functioning requires focusing in on actual investment projects that have been planned and delivered. Table 2 draws extensively from the diagnostic framework set out in Rajaram et al. (2014: 30-33; 34-37) and sets out some of the types of questions that officials might want to ask when scrutinising a sample of projects in a particular sector or ministry.

22 It should be noted here that it is unrealistic to expect all projects to be subject to some kind of formal economic analysis. The kind of analysis required should be proportional to the scale or riskiness of the project. In practice, this may mean that there are simple rules such as thresholds that define what kind of analysis is needed (e.g. all projects over $1 million in value require a full economic appraisal).

23 Checks on optimism bias can come from an independent check, but also from techniques such as ‘reference class forecasting’, essentially creating a distribution of the costs and benefits of the actual outcomes from comparable projects. The aim is create an ‘outside view’, as the ‘inside view’ of the project promoter is systematically biased (Flyvbjerg, 2008; Kahneman, 2011).

Table 2. Assessing the functionality of public investment management systems

Feature of public investment management

Key functions Examples of diagnostic questions

Investment guidance and screening

Only projects consistent with the broader government strategy/ vision proceed for formal appraisal.

• Is there guidance which sets out criteria by which projects could conceivably be rejected prior to appraisal?

• Is there evidence of projects being developed that are rejected prior to appraisal?

Project appraisal23 Projects are subjected to standard formal appraisal processes, with a check on systematic optimism bias in appraisal.24

Looking at a sample of project appraisals:

• Did the projects appraised follow standard appraisal procedures?

• Did project appraisals differ depending on the size of the project?

• In the sector under consideration, did appraisals use economic appraisal? Are assumptions in appraisal clearly stated?

• What proportion of project appraisals were rejected/sent back for further analysis? Was there a clear rationale for the rejection[s] that was consistent with appraisal procedures?

Project selection and budgeting

Projects that have not been approved through appraisal procedures are not eligible for financing.

Funding of projects is consistent with the overall resource envelope.

• What proportion of projects sampled were appraised prior to incorporation in the budget?

• Were projects that were incorporated in the budget ready to be implemented?

• Has there been any delays in approving annual budgets for investment projects?

• Are budget allocations for projects consistent with those set out in project appraisals? Are existing investment projects under arrears? Have projects been subject to penalties for delayed payments?

• Are all donor counterpart funding requirements being met?

Project implementation Projects completed on time, on budget and to specification.

Contracting process for projects is functionally competitive to support the achievement of these goals.

• What is the average percentage cost overrun (in inflation adjusted terms) on major projects compared to appraisal; compared to original (and amended) budget?

• What is the average percentage time overrun on major projects?

• What is the average percentage execution rate of projects?

• Funding: Are budgeted funds being released to projects on time and in full? Are ongoing projects receiving sufficient funds for their completion? Is there information available on the total cost of ongoing projects and the amounts that have been spent on the projects to date?

• Procurement: Were contracts awarded on the basis of competitive bidding? How many bids were received for projects? Was the procurement of projects subject to delay? Has there been renegotiation of contracts since the bids were made?

• Monitoring: Is information available on the progress of projects against milestones/reporting frameworks set out in project plans? Are mechanisms in place to verify whether completed projects have been implemented in line with planned specifications?

Project adjustment Project implementation is reviewed and allows for purposeful adjustments to disbursements or termination of projects when external circumstances change.

• Where costs of projects have escalated relative to initial plans, what checks/processes have been in place to approve changes?

• Are project appraisals revisited after their costs increase? Or benefits reduced?

Basic completion review and evaluation

Performance is monitored. Lessons are learned and actions taken to improve performance.

• Are completed projects evaluated and estimated costs and benefits compared relative to appraisal?

• Is there evidence that major problems identified in the analysis are being addressed?

• Is there clarity on the actors with responsibility to address these problems?

24 ODI Report


Table 2 sets out questions that could be usefully asked in an analysis of specific projects. However, a further key function is to ensure that overall levels of public investment are fiscally sustainable. This requires a quite different set of questions, as set out in Table 3.

A final ‘must have’ in the initial Rajaram et al. (2014) diagnostic that is not covered in Tables 2 and 3 is the operation of existing assets.

It is clear that the returns to investment are going to depend on how assets are operated and used. However, this does not fit neatly around the project cycle and it may be more difficult to fully assess how systems are functioning. Questions such as ‘Does an asset register exist?’ are, on their own, unlikely to diagnose whether or not assets are being properly maintained.

A key challenge in understanding whether systems for operating and maintaining existing assets are functioning effectively is that there is likely to be a separate system for each sector. The operation and maintenance of roads might depend upon the management of a road fund. In the energy sector, it raises questions of sector regulation and the pricing of energy. Where state-owned enterprises are involved in the management of existing assets, it raises questions of corporate governance of the state-owned enterprises sector. Nevertheless, some key questions which can be asked to guide diagnosis of the operation of existing assets are set out in Table 4.

An integrated version of Tables 2 to 4 is included at Annex 1.

4.2. Moving from diagnostics to reform

4.2.1. Managing the politics of public investmentDiagnostics are useful for telling us which capacities are weak and in need of reform. But we cannot assume that a government counterpart will necessarily have the willingness or capability to tackle identified weaknesses. More thought is needed on how external actors might usefully engage with the political economy of capital spending.

Cross-country evidence finds that countries with low-quality governance and limited political checks and balances have higher public investment as a proportion of GDP (Keefer and Knack, 2007; Grigoli and Mills, 2014). However, this higher public investment does not generally translate into productive infrastructure. This arises from the inherent nature of capital spending: because it is a multi-year process, from the design of projects to their selection, financing and implementation, it is plagued by commitment problems.

One manifestation of this commitment problem, as noted in Section 2, is seen when politicians cannot make credible commitments to their potential supporters. Political support is then gained by clientelism, such as spending on narrowly targeted public infrastructure and excessive rent-seeking, or ‘pork barrel’ politics. In such a context, infrastructure does not support economic growth because that is not the aim of the spending. It is instead clientelist spending which aims to maintain support for the government.

Table 3. Assessing the fiscal sustainability of the overall level of public investment



Making prudent decisions on the overall level of public investment spending

National infrastructure stock is maintained or expanded, subject to borrowing being consistent with macroeconomic stability.

• Is consideration of the aggregate size of investment integrated into the setting of the overall fiscal framework?

• Do ‘top down’ expenditure ceilings reflect all existing investment commitments? • Do ceilings reflect recurrent commitments resulting from completed projects?• Are policy-makers presented with options on how to allocate fiscal space, including on

new investments?• What is the estimated cost of completing the investment portfolio as a percentage of the

existing investment budget?

Managing potential fiscal risks emerging from investment

Identification and management of potential fiscal risks emerging from management of infrastructure.

• Is information available on contingent liabilities in the infrastructure sector, including through PPPs, state-owned enterprises and subnational borrowing?

• Are there processes in place to use this information to inform decisions on how to manage these risks?

• What is the size of these liabilities relative to GDP?

Table 4. Diagnostic questions for assessing infrastructure operation



Facility operation Once constructed, facilities are operating and providing services.Affordable and accessible infrastructure services are provided.

• What proportion of completed projects are providing services?

• What is the delay between project completion and projects becoming operational?

• Are infrastructure coverage and quality indicators improving?

A second manifestation is seen when politicians cannot commit to agreements with each other, leaving projects uncompleted. Williams (2017) finds that approximately one-third of local government capital projects (e.g. schools, boreholes) in Ghana, consuming nearly one-fifth of all capital spending, are never completed. This happens because political leaders are unable to sustain bargains over project distribution over multiple years. As a result, projects are started but never finished as funds are allocated to new projects before ongoing projects are completed. Districts that are politically and ethnically homogenous, and that can therefore more easily make and keep commitments, have significantly higher completion rates.

Given that poor infrastructure outcomes are related to underlying political incentives, a reform strategy that relies on a narrow political or technocratic leadership to push through improvements to PIM processes and techniques may not work. Instead, reforms need to find creative ways to address, or work around, the more fundamental incentive problems that underlie technical weaknesses.

A simple institutional change that promoted collective action has been seen in Ghana, where some projects were financed by an aid-funded grant with one simple rule: existing projects must be completed before new funds are disbursed. Aid-financed projects had significantly higher completion rates as a result. This simple institutional change to the grant rules seemed to be sufficient to induce bargaining and solve the collective-action problem (Williams, 2017).

Examples of broader institutional reforms that broadened inclusivity can be seen in Kenya. The transition to democracy diminished the ethnic favouritism that had dominated the allocation of road building in Kenya under previous periods of dictatorship (Burgess et al., 2015).

Cross-country studies also hypothesise that, as in Kenya, it is the very dispersal of decision-making power in more democratic institutions that leads to better, and more stable, decisions being made. Constraints on the executive and other checks and balances put decisions under greater scrutiny and require decision-makers to obtain the cooperation and approval of others (Burgess et al., 2015; Grigoli and Mills, 2014).

The result is that instead of investing in narrowly targeted public investment for clientelist reasons, political leaders invest in public investments that are shared across a wider segment of the population – and it is such genuine public goods that can promote economic growth. This is a case of broad institutional reform leading to better outcomes. Of course, it is not realistic to promote democracy simply as part of a public investment reform programme, but this example suggests that aiming to improve technical systems alone may not be enough.

4.2.2. Effective support for reformThe politics of public investment magnify the challenges of reform and emphasise the need for careful use of diagnostics. They should be used as an avenue to open dialogue with the government and identify problems, not to mechanistically turn the identified weaknesses into blueprints for reform.

A stylised PIM reform process based on this ‘blueprint approach’ is set out in Figure 6. First, the diagnostic is used to identify those areas where a PIM system does not live up to the standard embodied in the diagnostic. Reforms are then mapped from the diagnostic, proposing introduction of the sort of processes and institutions that the diagnostic has identified as missing (all too often based on an idealised ‘best practice’, rather than an analysis of what is appropriate

Figure 6. A ‘blueprint’ approach to public investment management reform

Diagnosticsprovide

measure of weak

institutions

Provide technical assistance/

training/money to target improved scores in those

weak PIM institutions

Improveef�ciency of investment

Source: Author’s own

26 ODI Report


to the context). Technical assistance and training are then provided to implement these reforms, with the expectation that the efficiency of investment will increase.

As well as being insensitive to the underlying political incentives, a ‘blueprint’ approach to PIM reforms also carries a large opportunity cost: if government is focused on reforms that do not solve the fundamental problems, its scarce resources are not being focused on the areas that could improve poor performance.

The experiences of how PEFA assessments have been misused in designing reforms carries some cautionary lessons (Hadley and Miller, 2016).

• It can lead to problems of ‘isomorphism’ – countries imitating best-practice institutional forms, without actually reforming how the system is working. Andrews (2010) demonstrates that a common set of public financial management reforms have been implemented in a wide variety of different countries, and it is unlikely they all faced the same underlying problems.

• It can concentrate efforts on making changes to the areas captured by standardised indicators, even where they do not address the most pressing challenges. For example, a key finding of much of the work to date on PIM has been that institutions in low-income countries are particularly weak in the area of project implementation; yet this primarily requires change in downstream institutions rather than the upstream agencies that are the focus of PIMA indicators.

• It could contribute to ‘capability overload’, encouraging institutional reforms that far exceed the domestic capacities to undertake them.

There is a danger in assuming that reform happens through a ‘top-down’ process, whereby a reform plan is drawn-up and then implemented by a committed leadership. Andrews (2015) distinguishes between two broad approaches to public sector reform: (1) ‘solution- and leader-driven change’ (SLDC), a top-down process where leaders design reforms and drive their implementation (referred to as ‘top-down reform’ from here on); and (2) ‘problem driven iterative adaptation’ 24 (PDIA), where reform is a process of muddling through, motivated by locally identified problems, finding solutions through experimentation and learning

24 ‘PDIA combines four key principles of engagement into a way of thinking about and doing development work in the face of complexity: (i) Focus on specific problems in particular local contexts, as nominated and prioritized by local actors; (ii) Foster active, ongoing experimental iterations with new ideas, gathering lessons from these iterations to turn ideas into solutions; (iii) Establish an ‘authorizing environment’ for decision making that encourages experimentation and ‘positive deviance’; and (iv) Engage broad sets of agents to ensure that reforms are viable, legitimate and relevant—that is, politically supportable and practically implementable.’ (Andrews et al., 2015: 20).

and being led by multiple agents. He finds in a sample of successful reforms that the bulk of these occurred through PDIA-type reform processes rather than through SLDC-type processes.

To support such processes, external actors, rather than just playing the role of technical advisors, also need to play the role of ‘facilitators’ who can convene actors and broker agreements (Williamson, 2015). This role as an honest broker is as important, if not more important, as the role as a provider of technical advice. External support may be needed to build consensus on what the problem is and what reforms can address it. There also needs to be wide understanding of what the reform ‘problem’ is, to drive action across a range of government officials in different ministries and agencies, rather than relying on a single ‘reform champion’ to drive change across government.

The kind of exploratory diagnostic provided by the World Bank PIM framework provides an avenue for this sort of analysis. The diagnostic process itself can be used to facilitate discussion across finance ministries and the other key agencies involved in delivering infrastructure. These include the relevant sector ministries and agencies (e.g. roads, water, energy), as well as planning or economic ministries. This can help drive shared understanding of problems and shared commitment to reform.

In addition, if detailed local knowledge is needed to identify reforms that are compatible with local circumstances, an appropriate role for outsiders, who are less likely to possess this detailed local knowledge than domestic actors, is to broker and stimulate discussion of possible reforms. Intelligent use of examples from other countries, which are then adapted to local context, can be important (Krause, 2013), as uncritical or insincere adoption of institutional forms is likely to fail.

A study of the Ugandan roads sector in 2009 offers some concrete examples of initiatives to support ‘room for manoeuvre in the change process’, as summarised in Box 2. These suggestions were made to support the newly established Uganda National Roads Agency to improve the efficiency of spending on roads, where vested interests were reluctant to see the end of collusive practices. These suggestions are not necessarily transferrable, but give a flavour of some of the ideas that could be used to facilitate ‘difficult change processes, turning potential realignments of interest into actual realignments.’

4.3. Drawing lessons from the public investment management literature for other sectors and policy areas

The PIM literature could provide useful insights for other sectors. In particular, it points to the fact that generating value for money in public spending is about much more than public financial management systems. The ‘must-have’ features identified by Rajaram et al. (2014) are not necessarily specific to public investment. Indeed, they could be applied more generally to the development and implementation of good spending policies:

• having in place an overall strategy to guide the formulation of specific policy initiatives

• appraising those policies before consideration in the budget • ensuring that those policies are independently challenged

and scrutinised • making sure that policies are properly budgeted for • having in place proper structures for policy

implementation

• reviewing the progress of policy implementation and revisiting the suitability of the policy design if necessary, and

• evaluating the effectiveness of policy implementation to support learning in the design of future policies.

Specific efforts to strengthen processes to ensure better value for money could also have positive spillover effects for policy-making in other sectors. For example, the UK’s Green Book for appraisal and evaluation is not just for investment projects, but is also about ensuring good spending policy (HM Treasury, 2013). Business cases are developed for all large new spending initiatives, not just specifically for infrastructure. The development of skills required to undertake economic analysis for appraising and evaluating expenditure policies are also highly relevant for evaluating changes in tax policy.

4.4. Principles for undertaking diagnosticsAt the country level, a number of principles could help to guide the quality of future diagnostic work.

Box 2. Supporting room for manoeuvre in the change process in Uganda’s roads sector

A political economy analysis of the Ugandan roads sector provided a set of suggestions on how external actors might increase ‘room for manoeuvre’ in making change.

Communications

External actors can bring communications professionals together with ‘overworked engineering and procurement professionals’ to communicate how the selection and implementation of investment projects affect infrastructure services. Where impressive work is being done, support can be sustained for that work by communicating in newspapers read by urban elites, but also through other channels that reach a wider audience.

Brokering dialogue

Mutual recriminations often surround failures to deliver investments: finance ministries may not provide enough money; line ministries may under-forecast costs of projects; and contractors may not do what they said they were going to do. This kind of adversarial communication is unhelpful for identifying solutions to problems. The right kind of third party can ‘sponsor a dialogue or “bang heads together” as appropriate.’

Countervailing networks

Where a small number of firms are capturing all government contracts at inflated prices, there may be limited incentive for change among those benefitting from the current system. However, there may be companies

who are losing out as a result of such arrangements. Individual companies may be unwilling to ‘put their head above the parapet’ to challenge these arrangements. A well-positioned honest broker for reform could help to build coalitions of actors who would benefit from ending collusive practices and counter those who lobby for maintenance of the status quo.

Lowering barriers to collective action

As in many countries, Ugandan firms are incentivised to bid low to win work in the expectation that they can escalate costs once a contract has been won. Technical improvements to contract management, on their own, are unlikely to fix such problems. To turn this logic around, some type of step-change will be required where firms can be assured that they will not be penalised for costing work accurately in their bids and that contractors will bear the risk of under-pricing work.

Infant industry support

Industry membership organisations (e.g. institutes of engineers or construction professionals, trade associations) can serve as a potentially useful pro-reform network. Providing support to these organisations to link up with counterparts in neighbouring countries can help in the sharing of ideas, building common standards and increasing the perceived value of membership of such organisations. Source: Booth and Golooba-Mutebi (2009, 2015).

28 ODI Report


• Applying diagnostics to an analysis of projects in specific sectors is more likely to yield insights into how systems are operating (and not just how they look). Where possible, an analysis of the performance of a sample of projects throughout their life cycle is a good way of trying to understand how systems are actually working. By focusing on projects, rather than just the institutional structures for managing projects, a clearer picture is likely to emerge as to whether systems are actually working. A project-level analysis can also play a useful role in identifying specific bottlenecks to project implementation.

• Diagnostic frameworks can provide useful structures to organise analysis, but should not be seen as straitjackets. There is a danger that, as standardised measures are introduced, PIM comes to be understood as a pre-defined set of institutions. Where diagnostics are used flexibly, they can be more easily tailored to specific problems.

• As much effort needs to go into identifying appropriate solutions as into identifying constraints to performance. Good institutional diagnostics will tell you how systems are working, but will not necessarily reveal why systems work the way they do. In many cases, PIM problems are collective action problems. ‘Inefficient’ investment may not be seen as a problem by certain government actors who benefit from those inefficiencies. Changing formal procedures may not actually change the decisions that are made if the underlying problem is not widely understood across government and if incentives for government actors are not changed.

• Motivating and supporting processes of change is vital for successful outcomes. This requires giving some thought as to where the pressure for change is likely to come from. In some cases, clients may have the willingness and authority to take on recommendations from a diagnostic and lead a process of reform on that basis; this cannot, however, always be assumed. External actors could play a useful role in bringing together different actors to develop a common understanding of a problem and possible solutions. Structuring discussions around obstacles to the delivery of infrastructure-related services may be more motivating than technical descriptions of ‘public investment management institutions’.

Ultimately, any diagnostic tool can be used poorly or well. What matters most is whether it contributes to an analysis that engages the government in fixing parts of the problem. A diagnostic will be used well if it stimulates discussion of problems and options for how they might be fixed. Doing this well requires relating a general diagnostic to a government’s priorities and the way in which local institutions work.

If diagnostics can be used to achieve this, they can enhance the cause of reform. However, if they are used only to provide mechanical, blueprint reforms that ignore the local context, they will distract attention from the areas where reforms are really needed and set back the cause of improved PIM.

Annex 1: Diagnostic questions for assessing the functionality of PIM systems



Making prudent decisions on the overall level of public investment spending

National infrastructure stock is maintained or expanded, subject to borrowing being consistent with macroeconomic stability.

• Is consideration of the aggregate size of investment integrated into the setting of the overall fiscal framework?

• Do ‘top down’ expenditure ceilings reflect all existing investment commitments?

• Do ceilings reflect recurrent commitments resulting from completed projects?

• Are policy-makers presented with options on how to allocate fiscal space, including on new investments?

• What is the estimated cost of completing the investment portfolio as a percentage of the existing investment budget?

Managing potential fiscal risks emerging from investment

Identification and management of potential fiscal risks emerging from management of infrastructure.

• Is information available on contingent liabilities in the infrastructure sector, including through PPPs, state-owned enterprises and subnational borrowing?

• Are there processes in place to use this information to inform decisions on how to manage these risks?

• What is the size of these liabilities relative to GDP?

Investment guidance and screening

Only projects consistent with the broader government strategy/ vision proceed for formal appraisal.

• Is there guidance which sets out criteria by which projects could conceivably be rejected prior to appraisal?

• Is there evidence of projects being developed that are rejected prior to appraisal?

Project appraisal Projects are subjected to standard formal appraisal processes, with a check on systematic optimism bias in appraisal.

Looking at a sample of project appraisals:

• Did the projects appraised follow standard appraisal procedures?

• Did project appraisals differ depending on the size of the project?

• In the sector under consideration, did appraisals use economic appraisal? Are assumptions in appraisal clearly stated?

• What proportion of project appraisals were rejected/sent back for further analysis? Was there a clear rationale for the rejection[s] that was consistent with appraisal procedures?

Project selection and budgeting

Projects that have not been approved through appraisal procedures are not eligible for financing.

Funding of projects is consistent with the overall resource envelope.

• What proportion of projects sampled were appraised prior to incorporation in the budget?

• Were projects that were incorporated in the budget ready to be implemented?

• Has there been any delays in approving annual budgets for investment projects?

• Are budget allocations for projects consistent with those set out in project appraisals? Are existing investment projects under arrears? Have projects been subject to penalties for delayed payments?

• Are all donor counterpart funding requirements being met?

Project implementation Projects completed on time, on budget and to specification.

Contracting process for projects is functionally competitive to support the achievement of these goals.

• What is the average percentage cost overrun (in inflation adjusted terms) on major projects compared to appraisal; compared to original (and amended) budget?

• What is the average percentage time overrun on major projects?

• What is the average percentage execution rate of projects?

• Funding: Are budgeted funds being released to projects on time and in full? Are ongoing projects receiving sufficient funds for their completion? Is there information available on the total cost of ongoing projects and the amounts that have been spent on the projects to date?

• Procurement: Were contracts awarded on the basis of competitive bidding? How many bids were received for projects? Was the procurement of projects subject to delay? Has there been renegotiation of contracts since the bids were made?

• Monitoring: Is information available on the progress of projects against milestones/reporting frameworks set out in project plans? Are mechanisms in place to verify whether completed projects have been implemented in line with planned specifications?

30 ODI Report




Project adjustment Project implementation is reviewed and allows for purposeful adjustments to disbursements or termination of projects when external circumstances change.

• Where costs of projects have escalated relative to initial plans, what checks/processes have been in place to approve changes?

• Are project appraisals revisited after their costs increase? Or benefits reduced?

Basic completion review and evaluation

Performance is monitored. Lessons are learned and actions taken to improve performance.

• Are completed projects evaluated and estimated costs and benefits compared relative to appraisal?

• Is there evidence that major problems identified in the analysis are being addressed?

• Is there clarity on the actors with responsibility to address these problems?

Facility operation Once constructed, facilities are operating and providing services.

Affordable and accessible infrastructure services are provided.

• What proportion of completed projects are providing services?

• What is the delay between project completion and projects becoming operational?

• Are infrastructure coverage and quality indicators improving?

ReferencesAbdallah, W. (2015) Effect of Electronic Public Procurement: Evidence from Bangladesh. IGC Working Paper. London:

International Growth Centre at the London School of Economic and Political Science.Andrews, M. (2010) ‘How far have public finance management reforms come in Africa?’ Cambridge, MA: Harvard

Kennedy School.Andrews, M. (2015) ‘Explaining positive deviance in public sector reforms in development’. World Development, 74,

197-208.Andrews, M., Pritchett, L., Woolcock, M. (2015) ‘The Challenge of Building (Real) State Capability’. CID Working Paper

No. 306. Cambridge MA: Center for International Development at Harvard University.Ansar, A., Flyvbjerg, B., Budzier, A., Lunn, D. (2014) ‘Should we build more large dams? The actual costs of hydropower

megaproject development’. Energy Policy, 69, pp.43-56.Balassa, B. A. (1986) Toward renewed economic growth in Latin America; summary, overview, and recommendations.

Washington D.C.: Institute for International Economics.Berg, A., Presbitero, A., Zanna, L-F. (2016) ‘Public investment efficiency and growth’, VoxEU.org, 5 January. Available at:

http://voxeu.org/article/public-investment-efficiency-and-growth Booth, D. and Golooba-Mutebi, F. (2015) ‘Reforming the roads sector in Uganda: a six-year retrospective’. ODI Working

Paper 424. London: Overseas Development Institute.Booth, D. and Golooba-Mutebi, F. (2009) ‘The political economy of roads reform in Uganda.’ Working Paper 307.

London: Overseas Development Institute.Brumby, J., Kaiser, K., Kim, J-H. (2013) ‘Public Investment Management and Public–Private Partnerships’ in Allen, R.,

Hemming, R. and Potter, B. (eds). (2013) The international handbook of public financial management. Palgrave Macmillan.

Burgess, R., Jedwab, R., Miguel, E., Morjaria, A., Padró i Miquel, G. (2015) ‘The Value of Democracy: Evidence from Road Building in Kenya’. American Economic Review, 105(6): 1817-51.

Collier, P. (2009) ‘Post-conflict recovery: How should strategies be distinctive?’ Journal of African Economies 18, suppl 1 i99–i131.

CoST (2011) Report on Baseline Studies: International Comparison. Summary findings of CoST pilot baseline studies. CoST, London: Construction Sector Transparency Initiative. http://gateway.transparency.org/tools/detail/4

Dabla-Norris, E., Brumby, J., Kyobe, A., Mills, Z., Papageorgiou, C. (2012) ‘Investing in public investment: an index of public investment efficiency’. Journal of Economic Growth, 17(3), pp.235-266.

Davis, K.E., Kingsbury, B. and Merry, S.E., (2012) Indicators as a technology of global governance. Law & Society Review, 46(1), pp.71-104.

Easterly, W. (1997) ‘The Ghost of Financing Gap: How the Harrod-Domar Growth Model Still Haunts Development Economics’. Working Paper, 1807, Washington D.C.: World Bank.

Estache, A. and Fay, M. (2009) ‘Current debates on infrastructure policy’. Working Paper 49 on Commission for Growth and Development. Washington D.C.: World Bank.

Evans, P. B. (1995) Embedded autonomy: states and industrial transformation. Princeton, NJ: Princeton University Press.Fainboim, I., D. Last, and E. Tandberg (2013) ‘Managing Public Investment’ in Cangiano, M., Curristine, T., Lazare, M.

(eds) Public Financial Management and Its Emerging Architecture. Washington D.C.: International Monetary Fund.Flyvbjerg, B. (2008) ‘Curbing optimism bias and strategic misrepresentation in planning: Reference class forecasting in

practice’. European planning studies, 16(1), 3-21.Flyvbjerg, B., Holm, M.S., Buhl, S. (2002) ‘Underestimating costs in public works projects: Error or lie?’. Journal of the

American planning association, 68(3), pp.279-295Foster, V. and Briceño-Garmendia, C. (eds) (2010) Africa’s Infrastructure: A Time for Transformation. Washington, D.C.:

World Bank.Grigoli, F. and Mills, Z. (2014) ‘Institutions and public investment: an empirical analysis’. Economics of Governance,

15(2), 131-153.Hadley, S. and Miller, M. (2016) PEFA: What is it good for? The role of PEFA assessments in public financial

management reform. London: Overseas Development Institute.Helm, D. and Mayer, C. (2016) ‘Infrastructure: why it is under provided and badly managed’. Oxford Review of

Economic Policy, 32(3), 343-359.Hemming, R. and Ter-Minassian, T. (2004) ‘Making room for public investment’. Finance and Development, 41(4),

30-33.HM Treasury, (2013) The Green Book: Appraisal and Evaluation in Central Government. London: The Stationery Office.

32 ODI Report


Høyland, B., Moene, K., Willumsen, F. (2012) ‘The tyranny of international index rankings’. Journal of Development Economics 97(1): 1-14.

IMF (2015a) Making Public Investment More Efficient. Washington, D.C.: International Monetary Fund.IMF (2015b) Financing for Development: Revisiting the Monterrey Consensus. Washington, D.C.: International

Monetary Fund.IMF (2014) World Economic Outlook: Legacies, Clouds, Uncertainties. Washington, D.C.: International Monetary Fund.Kahneman, D. (2011) Thinking, Fast and Slow. New York: Farrar, Straus and Giroux.Keefer, P. and Knack, S. (2007) ‘Boondoggles, rent-seeking, and political checks and balances: public investment under

unaccountable governments’. The Review of Economics and Statistics, 89(3), 566-572.Killick, T. (2004) ‘Politics, evidence, and the new aid agenda’. Development Policy Review, 22, pp. 5–29.Krause, P. (2013) ‘Of institutions and butterflies: is isomorphism in developing countries necessarily a bad thing?’.

Background Note, London: Overseas Development Institute.Krueger, A. O. (1990) ‘Government failures in development’. Working Paper No. 3340. Cambridge MA: National Bureau

of Economic Research.Langbein, L. and Knack, S. (2010) ‘The worldwide governance indicators: six, one or none?’. The Journal of Development

Studies 46(2): 350-370.Lewis-Faupel, S., Neggers, Y., Olken, B. A., Pande, R. (2014) ‘Can electronic procurement improve infrastructure

provision? Evidence from public works in India and Indonesia’. Working Paper No. 20344. Cambridge MA: National Bureau of Economic Research.

Little, I. M. D. and Mirrlees, J. A. (1991) ‘Project Appraisal and Planning Twenty Years On’. Proceedings of the World Bank Annual Conference on Development Economics 1990. Washington, D.C.: World Bank.

OECD (2015) Effective Public Investment Across Levels of Government: Principles for Action. Paris: Organisation for Economic Co-operation and Development.

Paternostro, S., Rajaram, A., Tiongson, E. R. (2007) ‘How does the composition of public spending matter?’ Oxford Development Studies, 35(1), 47-82.

Pritchett, L. (2000) ‘The tyranny of concepts: CUDIE (cumulated, depreciated, investment effort) is not capital’. Journal of Economic Growth, 5(4), 361-384.

Rajaram, A., Kaiser, K., Le, T.M., Kim, J.H., Frank, J. (2014) The Power of Public Investment Management: Transforming Resources into Assets for Growth. Washington, D.C.: World Bank.

Rajaram, A., Le, T. M., Biletska, N., Brumby, J. (2010) ‘A diagnostic framework for assessing public investment management’. World Bank Policy Research Working Paper 5397. Washington D.C.: World Bank.

Robinson, J. A. and Torvik, R. (2005) ‘White elephants’. Journal of Public Economics, 89(2), 197-210.Rodrik, D. (2010) ‘Diagnostics before Prescription’. Journal of Economic Perspectives 24(3): 33-44.Rodrik, D. (1995) ‘Getting interventions right: how South Korea and Taiwan grew rich’. Economic Policy, 10(20), 53-107.Rostow, W. (1962) The Stages of Economic Growth. London: Cambridge University Press.Ruiz Nunez, F. and Wei, Z. (2015) ‘Infrastructure investment demands in emerging markets and developing economies.’

Policy Research Working Paper 7414. Washington D.C.: World Bank.Schiavo-Campo, S. and Tommasi, D. (1999) Managing government expenditure. Manila: Asian Development Bank.Schick, A. (1998) A contemporary approach to public expenditure management. Washington D.C.: World Bank Institute.Simson, R. (2012) ‘Following the money: Examining the evidence on ‘pro-poor’ budgeting’. ODI Background Note.

London: Overseas Development Institute.Thomas, M.A. (2010) What do the worldwide governance indicators measure? The European Journal of Development

Research, 22(1), pp.31-54.Warner, A. (2014) ‘Public Investment as an Engine of Growth’. IMF Working Paper 14/148. Washington D.C.:

International Monetary Fund.Williams, M. (2017) The political economy of unfinished development projects: corruption, clientelism, or collective

choice? American Political Science Review, forthcoming.Williamson, J. (ed.) (1990) Latin American Adjustment: How Much Has Happened? Institute for International

Economics, Conference Volume. Washington D.C.: Institute for International Economics.Williamson, T. (2015) Change in challenging contexts: how does it happen? ODI Report. London: Overseas Development

Institute. World Bank (2012) Identifying the constraints to budget execution in the infrastructure sector: DIPA tracking study.

Washington D.C.: World Bank.World Bank (2007) ‘Fiscal Policy for Growth and Development: Further Analysis and Lessons from Country Case

Studies’. Background Paper (DC2007-0004) for April 2007 meeting of the Development Committee. Washington D.C.: World Bank Group and International Monetary Fund.

World Bank (2006) ‘Fiscal Policy for Growth and Development: An Interim Report.’ Background Paper (DC2006-0003) for April 2006 meeting of the Development Committee. Washington D.C.: World Bank Group and International Monetary Fund.

World Bank (2005) The Poverty Reduction Strategy Initiative: Findings from 10 Country Case Studies of World Bank and IMF Support, World Bank Operations Evaluations Department and IMF Internal Evaluations Office, Washington D.C.: World Bank.

World Bank (1998) Public expenditure management handbook. Washington D.C.: The World Bank.World Bank (1994) World Development Report 1994: Infrastructure for development. Washington D.C.: World Bank.

34 ODI Report

ODI is the UK’s leading independent think tank on international development and humanitarian issues.

Readers are encouraged to reproduce material from ODI Reports for their own publications, as long as they are not being sold commercially. As copyright holder, ODI requests due acknowledgement and a copy of the publication. For online use, we ask readers to link to the original resource on the ODI website. The views presented in this paper are those of the author(s) and do not necessarily represent the views of ODI.© Overseas Development Institute 2017. This work is licensed under a Creative Commons Attribution-NonCommercial Licence (CC BY-NC 4.0).

All ODI Reports are available from www.odi.org

Cover photo: Construction workers on site. Arne Hoel / World Bank

Overseas Development Institute203 Blackfriars Road London SE1 8NJTel +44 (0) 20 7922 0300 Fax +44 (0) 20 7922 0399

odi.org

Reviewing the role of external actors · International development actors are, increasingly,...

Documents

Transcript of Reviewing the role of external actors · International development actors are, increasingly,...