Evaluation Approach Paper Sector-wide Evaluation: ADB’s 2009 Energy … · 2019. 8. 14. · of...
28
Asian Development Bank. 6 ADB Avenue, Mandaluyong City, 1550 Metro Manila, Philippines Tel +63 2 632 4444; Fax +63 2 636 2163; [email protected]; www.adb.org/evaluation Evaluation Approach Paper Sector-wide Evaluation: ADB’s 2009 Energy Policy and Program, 2009–2018 July 2019 Team Leaders: Alfredo Baño Leal, Evaluation Specialist ([email protected]) Kapil Thukral, Principal Evaluation Specialist ([email protected]) Sherine Ibrahim, Principal Evaluation Specialist ([email protected]) Contact: [email protected] A. Introduction 1. This paper sets out the rationale, approach, and methodology for an independent evaluation of the Asian Development Bank’s (ADB) current 2009 Energy Policy, 1 and the support of ADB for the development of the energy sector in developing member countries (DMCs), as well as energy-related interventions in other sectors during the study period, 2009–2018. Consultations with ADB’s Board of Directors and management have informed the scope and outline of the evaluation. 1. Rationale 2. The current ADB Energy Policy was enacted 10 years ago in June 2009, just 1 year after ADB’s long-term strategic framework, 2008–2020 (Strategy 2020) was approved in 2008. 2 In July 2018, ADB adopted a new long-term corporate strategy to 2030 (Strategy 2030) that sets the direction for ADB’s efforts to respond effectively to the region’s changing needs, focused on eradicating extreme poverty and expanding its vision to achieve a prosperous, inclusive, resilient, and sustainable Asia and the Pacific. 3 Further, in the past decade, there has been fundamental changes in the energy sector that require different approaches to benefit from new opportunities and tackle new challenges. 3. Therefore, considering the new corporate Strategy 2030 and the current transition of global energy sectors, ADB’s Board of Directors has requested ADB’s Independent Evaluation Department (IED) to review the current 2009 Energy Policy and its implementation, in light of the ongoing global energy transformation, rising climate change concerns and deteriorating air quality (in many urban centers) to inform a new ADB’s energy policy, which would need to be aligned with Strategy 2030. In particular, the evaluation will consider coal use from various perspectives (such as impact on climate change, air pollution, support to base load generation and energy security), and take note of the extent other multilateral development banks (MDBs) support coal for power generation or other applications. Furthermore, recognizing that energy choices vary considerably among DMCs, it is important that the evaluation recognizes and considers such differences among DMCs (for example small island developing states, fragile states, land-lock states, and resource-rich countries) in suggesting options in the energy policy. The evaluation will review the policy role and design, its implementation through the energy program, and will 1 ADB. 2009. Energy Policy. Manila. 2 ADB. 2008. Strategy 2020: The Long-Term Strategic Framework of the Asian Development Bank, 2008–2020. Manila. 3 ADB. 2018. Strategy 2030: Achieving a Prosperous, Inclusive, Resilient, and Sustainable Asia and the Pacific. Manila.
Transcript of Evaluation Approach Paper Sector-wide Evaluation: ADB’s 2009 Energy … · 2019. 8. 14. · of...
Asian Development Bank. 6 ADB Avenue, Mandaluyong City, 1550 Metro Manila, Philippines Tel +63 2 632 4444; Fax +63 2 636 2163; [email protected]; www.adb.org/evaluation
Evaluation Approach Paper Sector-wide Evaluation: ADB’s 2009 Energy Policy and Program, 2009–2018 July 2019
Team Leaders: Alfredo Baño Leal, Evaluation Specialist ([email protected]) Kapil Thukral, Principal Evaluation Specialist ([email protected]) Sherine Ibrahim, Principal Evaluation Specialist ([email protected]) Contact: [email protected]
A. Introduction
1. This paper sets out the rationale, approach, and methodology for an independent evaluation of the Asian Development Bank’s (ADB) current 2009 Energy Policy,1 and the support of ADB for the development of the energy sector in developing member countries (DMCs), as well as energy-related interventions in other sectors during the study period, 2009–2018. Consultations with ADB’s Board of Directors and management have informed the scope and outline of the evaluation.
1. Rationale
2. The current ADB Energy Policy was enacted 10 years ago in June 2009, just 1 year after ADB’s long-term strategic framework, 2008–2020 (Strategy 2020) was approved in 2008.2 In July 2018, ADB adopted a new long-term corporate strategy to 2030 (Strategy 2030) that sets the direction for ADB’s efforts to respond effectively to the region’s changing needs, focused on eradicating extreme poverty and expanding its vision to achieve a prosperous, inclusive, resilient, and sustainable Asia and the Pacific.3 Further, in the past decade, there has been fundamental changes in the energy sector that require different approaches to benefit from new opportunities and tackle new challenges.
3. Therefore, considering the new corporate Strategy 2030 and the current transition of global energy sectors, ADB’s Board of Directors has requested ADB’s Independent Evaluation Department (IED) to review the current 2009 Energy Policy and its implementation, in light of the ongoing global energy transformation, rising climate change concerns and deteriorating air quality (in many urban centers) to inform a new ADB’s energy policy, which would need to be aligned with Strategy 2030. In particular, the evaluation will consider coal use from various perspectives (such as impact on climate change, air pollution, support to base load generation and energy security), and take note of the extent other multilateral development banks (MDBs) support coal for power generation or other applications. Furthermore, recognizing that energy choices vary considerably among DMCs, it is important that the evaluation recognizes and considers such differences among DMCs (for example small island developing states, fragile states, land-lock states, and resource-rich countries) in suggesting options in the energy policy. The evaluation will review the policy role and design, its implementation through the energy program, and will
1 ADB. 2009. Energy Policy. Manila. 2 ADB. 2008. Strategy 2020: The Long-Term Strategic Framework of the Asian Development Bank, 2008–2020.
Manila. 3 ADB. 2018. Strategy 2030: Achieving a Prosperous, Inclusive, Resilient, and Sustainable Asia and the Pacific.
Manila.
2
ascertain ADB’s institutional structure and human resources to deliver the policy and program outcomes.
2. Sector Context
4. Recent technological advances in clean energies and grid control systems, together with strong impetus to mitigate climate change across the world, are significantly transforming the energy sector. Traditional utility business models, customer-supplier relationships, and established economic structures are starting to adapt and transform in response to the new environment.
5. Such structural transformation in the energy sector is also reflected in how countries respond to their own energy needs and challenges. Asia Pacific, the region with the highest economic growth, is in the middle of such transformation. Several countries in the region are aiming at addressing the energy trilemma: (i) energy security, by balancing domestic and imported energy resources; (ii) energy equity, to provide not only access but also affordable supply; and (iii) environmental sustainability, with transition to low-carbon technologies, as well as through sensitive project design, siting, construction, and operation of energy infrastructures.4 The way these countries tackle the energy trilemma will define in the following decades their energy sector structure and sustainability. Although not stated explicitly, financial sustainability of energy institutions and utilities—which is an important aspect of the ADB’s Energy Policy and energy program—is embedded in the energy trilemma.
6. The evaluation covers a critical period when renewable energy markets for certain technologies have evolved considerably with shifts in market structure (e.g., solar photovoltaic and wind power), where the global community has placed considerable resources to help tackle climate change5 and achieve the Sustainable Development Goals (SDGs) set by the United Nations General Assembly in 2015.6 This includes taking actions towards meeting climate change mitigation targets established at the Paris Agreement7 (i.e., nationally determined contributions [NDCs] to mitigate global warming) of which all but one of ADB’s DMCs signed the agreement and ratified its NDCs.8 The evaluation also recognizes that ADB will set its climate finance commitment of $80 billion from own resources, cumulatively from 2019 to 2030, and to have 75% of committed operations (on a 3-year rolling average) supporting mitigation and adaptation as part of its Strategy 2030.
3. Overview of other Multilateral Development Banks’ Energy Policies and Strategies
7. Table 1 shows the different energy strategic documents from various MDBs.
4 The World Energy Council defines the “energy trilemma” as the three core dimensions of energy sustainability,
namely energy security, energy equity, and environmental sustainability. 5 Since 1992 when the United Nations Framework Convention on Climate Change (UNFCCC) came into effect, the
Kyoto Protocol in 1997, and more recently through the Paris Agreement in 2015, as well as the support from Climate Investment Funds and Green Climate Fund.
6 United Nations General Assembly. 2015. Resolution A/RES/70/1 of 25 September 2015. Transforming Our World: The 2030 Agenda for Sustainable Development. Geneva.
7 UNFCCC. 2016. Decision 1/CP.21 Adoption of the Paris Agreement by the Conference of the Parties on its twenty-first session, held in Paris from 30 November to 13 December 2015.
8 Kyrgyz Republic signed the agreement in September 2016 but has not ratified its greenhouse gas reduction contributions.
3
Table 1: Other Multilateral Development Banks’ Strategic Energy Sector Documents
IFI Year Title Type Timeframe
ADB 2009 Energy Policy Policy not specified AfDB 2012 Energy Sector Policy of the AfDB Group Policy 10 years AIIB 2017 Energy Sector Strategy: Sustainable Energy for Asia Strategy not specified EBRD 2018 Energy Sector Strategy, 2019–2023 Strategy 5 years EIB 2013 Energy Lending Criteria Other not specified IDB 2018 Energy Sector Framework Document Other 3 years WBG 2013 Directions for the WBG’s Energy Sector Other not specified
ADB = Asian Development Bank, AfDB = African Development Bank, AIIB = Asian Infrastructure Investment Bank, EBRD = European Bank for Reconstruction and Development, EIB = European Investment Bank, IDB = Inter-American Development Bank, IFI = international financial institution, WBG = World Bank Group. Source: Adapted from EBRD. 2018. Review of the EBRD Energy Sector Strategy. London. 8. Only ADB and the African Development Bank have an energy sector policy, whereas the European Bank for Reconstruction and Development (EBRD) and the Asian Infrastructure Investment Bank (AIIB) have an energy sector strategy. The World Bank Group (WBG) had an energy sector strategy until 2009; however, the revised energy sector strategy submitted to the WBG’s Committee of Development Effectiveness in 2011 was stalled and eventually discarded in 2012.9 The current energy strategic document is referred to as the Directions for the WBG’s Energy Sector, published in 2013.10
9. The Directions for the WBG’s Energy Sector document of 2013 articulates the central objective of supporting client countries in securing the affordable, reliable, and sustainable energy supply needed to end poverty and promote shared prosperity. Its guiding principles include: (i) catalyzing the transformation of the energy sector in the context of long-term system-wide planning; (ii) emphasizing improvements in the financial, operational, and institutional environment of the sector; (iii) seeking market solutions and helping governments foster private sector participation and investment; and (iv) embracing a multi-stakeholder, inclusive approach to energy development, tailored to individual country and regional circumstances.
10. The EBRD published in December 2018 its 5-year energy sector strategy, 2019–2023, which aims at promoting secure, affordable, and sustainable energy in its client countries through a transitioning to a market-oriented low carbon energy sector.11 EBRD will focus on the following four strategic directions: (i) decarbonizing power generation and promoting electrification, (ii) fostering the development of energy markets and regional integration, (iii) improving environmental standards and energy efficiency in the oil and gas supply chains, and (iv) achieving high operating standards through energy efficiency and inclusive economies.
11. The Inter-American Development Bank published in October 2018 its Energy Sector Framework Document,12 to be revised every 3 years, which shows the following key principles of investments in energy sector: (i) developing economic and sustainable energy access, both on-grid and off-grid, as well as modern and efficient technologies for cooking; (ii) promoting energy efficiency, renewable energies, and cleaner fuels for sustainable energy; (iii) stimulating energy
9 R.K. Lattanzio. 2013. The World Bank Group Energy Sector Strategy. Washington, DC: Congressional Research
Service. 10 The World Bank. 2013. Toward a Sustainable Energy Future for All: Directions for the World Bank Group’s Energy
Sector. Washington, DC. 11 European Bank for Reconstruction and Development. 2018. Energy Sector Strategy 2019-2023. London. 12 Interamerican Development Bank. 2018. Energy Sector Framework Document. Washington, DC.
4
security; and (iv) promoting good governance and increasing financial sustainable and strong institutions.
12. The AIIB published in June 2017 its energy sector strategy,13 which is consistent with AIIB’s three thematic priorities of sustainable infrastructure, cross-country connectivity and private capital mobilization. The strategy defines six principles that guide AIIB’s energy portfolio, namely (i) promoting energy access and security, (ii) realizing energy efficiency potential, (iii) reducing the carbon intensity of energy supply, (iv) managing local and regional pollution, (v) catalyzing private capital, and (vi) promoting regional cooperation and connectivity.
13. African Development Bank published its 10-year Strategy for the New Deal on Energy for Africa, 2016–2025, which aims at achieving universal access to electricity by 2025 (5 years earlier than SDG 7 on universal access by 2030).14 The New Deal addresses seven strategic themes: (i) setting up the right enabling policy environment, (ii) enabling utility companies for success, (iii) dramatically increasing the number of bankable energy projects, (iv) increasing the funding pool to deliver new projects, (v) supporting “bottom of the pyramid” energy access programs, (vi) accelerating major regional projects and driving integration, and (vii) rolling out waves of country-wide energy ”transformations.”
14. The European Investment bank (EIB) published in July 2013 its energy lending criteria, which mainly focused on delivering economic growth, security and sustainability. EIB activities are primarily guided by European Union (EU) policies in energy, climate change, and external affairs and development. Outside the EU, EIB will prioritize energy network projects, which are in line with its mandates, notably: (i) support the EU’s external energy policies, in particular the diversification of energy supply through physical interconnection in both electricity and gas networks; (ii) increase the rational use of energy and economic development through regional integration; (iii) enhance the reliability and security of energy supply; and (iv) increase access to affordable energy.
B. ADB’s 2009 Energy Policy and Energy Program, 2009–2018
15. The energy sector at ADB, together with the water sector, are the only infrastructure sectors with a policy document.15 The information and communication technology sector has a strategy document from 2003, the knowledge management and governance themes have action plans, the climate change theme has an operational framework, the environmental services theme has an operational directions document, whereas all the other sectors and themes have operational plans.16 It is unclear on the differentiation of the various sectors and themes regarding strategic policy documents, and why energy has a policy but not the other sectors.
16. The 2009 Energy Policy was prepared after a detailed assessment of key policy issues was conducted in 2008, taking into account the 1995 Energy Policy, its 2000 review, and the findings and recommendations of the 2007 special evaluation of the 2000 Review, and priorities established in Strategy 2020. The 2009 Energy Policy is aligned with Strategy 2020, enabling 13 Asia Infrastructure Investment Bank. 2017. Energy Sector Strategy: Sustainable Energy for Asia. Beijing. 14 African Development Bank. 2016. The Bank Group’s Strategy for The New Deal on Energy for Africa 2016–2025.
Abidjan. 15 The Water Policy was approved in 2001 and published in 2003. While the policy has not been updated since then,
the water sector is currently guided by its operational plan 2011-2020. 16 Disaster risk management is covered by the 2004 Disaster and Emergency Assistance Policy (currently under review)
as well as the Operational Plan for Integrated Disaster Risk Management 2014-2020. Most of these documents are date-limited to 2020 and might be updated with operational plans or frameworks under Strategy 2030; one of the exceptions is the Climate Change Operational Framework which runs to 2030.
5
energy operations to build on ADB's overall strategy emphasizing energy security, facilitating a transition to a low-carbon economy, providing universal access to energy, and for achieving ADB’s vision of a region free of poverty.
1. Key Elements of the 2009 Energy Policy
17. This section provides an overview of the key aspects of the current ADB Energy Policy.
(i) The objective of the 2009 Energy Policy is to help DMCs provide reliable, adequate, and affordable energy for inclusive growth in a sustainable way. This objective aligns with Strategy 2020 core area of operations on infrastructures that aims at meeting growing energy demands in a sustainable manner.
(ii) The Energy Policy outcomes as defined in its design and monitoring framework (DMF) are (a) improved energy efficiency and greater use of renewable energies in DMCs, (b) expanded access to energy in DMCs, and (c) efficient and viable energy sector in DMCs.
(iii) Among others, the following principles guide implementation of the Energy Policy, such as effective regional cooperation in the energy sector; governance, sector reform, and capacity building; private sector participation and public–private partnerships; compliance with ADB safeguards policies; and promotion of superior knowledge management.
(iv) The policy sets ADB’s target for clean energy investments as $2 billion per year from 2013, including renewable energies as well as investments in natural gas transport, distribution, liquefaction or regasification, and gas-fired power plants.17
(v) The policy restricts direct financing of coal mine development except for captive power plants, although allows support for coal mines safety, efficient use of coal for power generation, carbon capture and storage, coal bed methane extraction and use, coal gasification and scrubbers, waste coal utilization, and efficient coal transportation.
(vi) The policy prohibits oil field exploration, but allows support for refining, transportation and distribution of petroleum products, as well as small oil-based power plants for island or remote communities.
(vii) Nuclear power generation is not allowed for ADB financing.
2. Energy Sector Program, 2009–2018
18. The energy sector was the second largest sector in ADB, after transport, during the period 2009–2018. ADB approved a total of $40.0 billion (equivalent to one-fourth of the total approvals) and committed $34.7 billion during the period. Nonetheless, since 2015, the energy sector has surpassed the transport sector in volume of approved operations per year and is currently the largest sector at ADB in terms of net committed financing volume of active operations.
19. Most of the approved financing in the energy sector has been through sovereign operations, with 75% of the total. ADB energy approved portfolio during 2009–2018 was comprised of a mix of sovereign loans ($27.7 billion, 69%), sovereign grants ($2.7 billion, 7%), technical assistance ($437.0 million, 1%), and nonsovereign operations ($9.2 billion, 23%). While sovereign loans have been the major vehicle of support, nonsovereign energy sector operations
17 For purposes of this evaluation, IED will align the definition of clean energy with that adopted by ADB from time to
time. Since 2015, cleaner fuels such as supercritical coal and gas are no longer classified as Clean Energy, as per a decision of the ADB’s Energy Sector Group. Not all of the clean energy investments as defined by ADB are eligible for climate finance.
6
account for more than one-fifth of energy sector approvals during the study period and have increased significantly since 2015. Further details on ADB’s approved energy portfolio is presented in Attachment 1 and the list of sovereign, nonsovereign, and TA projects is presented in the Supplementary Attachment.
20. Per subsector. Electricity transmission and distribution continue to dominate the approved portfolio, but the share of other subsectors has increased since 2015. Most of the approved energy operations during 2009–2018 supported electricity transmission and distribution projects ($17.0 billion, 43%), renewable energy generation ($5.3 billion, 13%),18 energy efficiency and conservation ($4.1 billion, 11%), conventional energy generation ($6.0 billion, 15%), energy sector development and institutional reform ($2.9 billion, 7%), and large hydropower generation ($2.1 billion, 5%) among other subsectors (Figure 1). Renewable energy generation is comprised of biomass and waste ($0.4 billion), geothermal ($1.1 billion), small hydro ($0.7 billion), solar ($1.8 billion), wind ($1.1 billion), and other renewable energy initiatives (i.e., supporting multiple renewable energy sources, $0.2 billion). Conventional energy generation is further broken down into coal-fired ($2.1 billion), gas-fired ($3.2 billion), oil-fired ($0.1 billion), and others (i.e., supporting multiple fuel types, $0.7 billion).
21. Geographically. Over a third of the total energy project approvals during the study period was approved for the South Asia region, mainly driven by Bangladesh, India, and Sri Lanka ($13.5 billion, 34%). Central and West Asia accounted for $12.2 billion (31%) notably, Afghanistan, Pakistan, and Uzbekistan. Energy approvals in Southeast Asia amounted to $9.3 billion (23%) led by Indonesia and Viet Nam, while the remainder is divided among East Asia ($3.9 billion, 10%), inter-regional projects ($0.6 billion, 2%), and the Pacific ($0.4 billion, 1.0%). At the country level, energy approvals in India ranked highest at $7.4 billion.
18 Including renewable energy from different sources—geothermal, small hydro, solar, wind and biomass and waste.
Note that municipal waste may contain a mix of renewable biomass (e.g., organic waste) as well as non-renewable waste (e.g., plastics, metals, etc.) that has not been recycled.
Figure 1: Approved ADB Energy Operations by Subsector, 2009–2018
ADB = Asian Development Bank. Sources: ADB Listing of loans, grants, and equity approvals database; Loans, grants and technical assistance details database from ADB Controller Department as of 31 December 2018; ADB e-operations database; project documents.
Conventional energy generation
15%
Electricity transmission and distribution
43%
Oil and gas transmission and distribution
3%
Energy utility services3%
Energy efficiency and conservation
11%
Energy sector development and institutional reform
7%
Large hydropower generation5%
Renewable energy generation13%
7
22. In addition to the projects classified under the energy sector, there are also projects with energy components implemented by other sector divisions (e.g., irrigation project using water pump powered with solar panels). During 2009–2018, a total of 29 sovereign projects were identified to have energy components amounting to a total of $583 million, or 1.6% of the whole energy portfolio (Attachment 1).
3. Available Evaluation Evidence
23. This report will use existing evaluation evidence from previous evaluation assessments carried out at various levels of detail.
24. Evaluation of the Energy Policy 2000 Review. In 2007, the former Operations Evaluation Department (now IED) evaluated the Energy Policy 2000 Review.19 The evaluation assessed the relevance and effectiveness of the 2000 Review of the 1995 Energy Policy, with a successful overall rating, and informed the formulation of the current 2009 Energy Policy.20
25. There were two key lessons: (i) sector reform and independent regulation are providing tangible impacts but efficiency gains during implementation can lead to further improvements; and (ii) energy efficiency, renewable energy, and clean energy programs are in their infancy, with a substantial need and potential to increase. Key recommendations focused on managing the energy and environment nexus, improving sector governance and developing an implementation plan for the new energy policy.
26. Sector evaluations for specific countries. One energy sector assistance program evaluation (SAPE) was produced for an evaluation period post 2009, the Pakistan SAPE (2005–2017).21 Also, the energy section of the last five country assistance program evaluations (CAPEs) will inform this report, including the CAPEs of Azerbaijan (2011–2017); India (2007–2015); Sri Lanka (2006–2015); People’s Republic of China (2006–2014); and Papua New Guinea (2001–2014).22 The energy section of the Indonesia CAPE (2005–2018) currently under preparation will also inform this evaluation.
27. Cross-sectoral, thematic, and corporate evaluations. Several cross-cutting issues at a corporate and thematic level are related with the design and implementation of the ADB’s energy program. As such, this study will use available evaluation evidence among others on climate change and greenhouse gas implications;23 energy efficiency in industry and buildings;24
19 This framework policy and strategy document reviewed and updated ADB’s 1995 Energy Policy. ADB. 2003. Energy
2000: Review of the Energy Policy. Manila (adopted in 2000). 20 Operations Evaluation Department. 2007. Special Evaluation Study on Energy Policy 2000 Review: Energy Efficiency
for a Better Future. Manila. 21 IED. 2019. Sector Assistance Performance Evaluation: ADB’s Support to the Pakistan Energy Sector (2005–2017).
Manila: ADB. 22 IED. 2019. Country Assistance Program Evaluation: Azerbaijan, 2011–2017. Manila: ADB; IED. 2017. Country
Assistance Program Evaluation: India, 2007–2015. Manila: ADB; IED. 2016. Country Assistance Program Evaluation: Sri Lanka. Manila: ADB; IED. 2015. Country Assistance Program Evaluation: People's Republic of China. Manila: ADB; and IED. 2016. Country Assistance Program Evaluation: Papua New Guinea. Manila: ADB.
23 IED. 2009. Evaluation Knowledge Brief on Greenhouse Gas Implications of ADB's Energy Sector Operations. Manila: ADB.
24 IED. 2011. Evaluation Knowledge Brief on Energy Efficiency Interventions. Manila: ADB.
8
access to climate finance;25 safeguards;26 gender;27 environment;28 regional cooperation;29 governance;30 specific financial modalities;31 engagement with middle-income countries;32 and partnerships.33
28. Project-level evaluations. At the project level, IED validated a total of 46 project completion reports of sovereign energy projects and 21 extended annual review reports on the nonsovereign side since 2009 (Table 2). In addition to validation reports, IED also produced 18 project performance evaluation reports, 9 for sovereign and 9 for nonsovereign operations. There was also a total of 201 technical assistance completion reports specific to the energy sector that were circulated from 2009 onwards. Moreover, IED carried out an impact evaluation study in 2010 covering rural electrification in Bhutan.34 On top of the validation reports, this evaluation will look at projects that have been recently completed (with project completion report or not yet self-evaluated), and those under advanced stages of implementation (roughly 5 years since loan effectiveness) but have made sufficient headway in their implementation.
Table 2: Evaluation and Validation Ratings of Energy Projects, 2009–2018
HS = highly successful, LS = less than successful, PPER = project performance evaluation report, PVR = project completion report validation report, S = successful, U = unsuccessful, XVR = extended annual review report validation report. Source: Asian Development Bank (Independent Evaluation Department).
25 IED. 2014. Thematic Evaluation: Real-Time Evaluation of ADB’s Initiatives to Support Access to Climate Finance.
Manila: ADB. 26 Ongoing evaluation on Safeguards to be published in 2020, and two completed studies: IED. 2016. Thematic
Evaluation: Real-Time Evaluation of ADB’s Safeguard Implementation Experience Based on Selected Case Studies. Manila: ADB, and IED. 2014. Corporate Evaluation: Safeguards Operational Review: ADB Processes, Portfolio, Country Systems, and Financial Intermediaries. Manila: ADB.
27 IED. 2017. Thematic Evaluation: Asian Development Bank Support for Gender and Development (2005–2015). Manila: ADB.
28 IED. 2016. Environmentally Sustainable Growth: A Strategic Review. Topical Paper. Manila: ADB. 29 IED. 2015. Thematic Evaluation: Asian Development Bank Support for Regional Cooperation and Integration. Manila:
ADB. 30 IED. 2014. Thematic Evaluation: ADB Support for Enhancing Governance in its Public Sector Operations. Manila:
ADB. 31 IED. 2018. Corporate Evaluation: Policy-Based Lending, 2008–2017: Performance, Results, and Issues of Design.
Manila: ADB; IED. 2017. Corporate Evaluation: Results-Based Lending at the Asian Development Bank: An Early Assessment. Manila: ADB.
32 IED. 2016. Corporate Evaluation: The Asian Development Bank’s Engagement with Middle-Income Countries. Manila: ADB.
33 IED. 2016. Thematic Evaluation: Effectiveness of Asian Development Bank Partnerships. Manila: ADB. 34 IED. 2010. ADB's Assistance for Rural Electrification in Bhutan—Does Electrification Improve the Quality of Rural
Life? Impact Evaluation Study. Manila: ADB.
Source HS S LS U Total Success Rate Sovereign PVR 5 34 6 1 46 85% PPER 2 5 2 0 9 78% Nonsovereign XVR 4 14 2 1 21 86% PPER 3 2 4 0 9 56%
9
C. Evaluation Scope and Methodology
1. Scope and Objectives of the Evaluation
29. The purpose of this evaluation is to obtain evidence-based findings as to how effectively ADB helped its clients address energy needs in a sustainable and integrated manner towards achieving inclusive economic growth which is environmentally sustainable. The evaluation will assess the ADB’s 2009 energy policy and energy-related activities approved and/or implemented during the 2009–2018 study period, addressing its ability to guide investments in a rapidly changing environment; and learning from this experience to better calibrate future assistance for achieving successful development outcomes
30. The general objectives of this study are to (i) provide the Board of Directors and Management with an independent evidence-based review of the ADB 2009 Energy Policy and evaluation of the energy programs that ADB has supported under its policy from 2009 to 2018; and (ii) identify lessons and make suggestions to Management and the Board, as inputs to the next energy policy.
31. This evaluation follows standard international evaluation practice and principles. The review process follows a systematic approach, by which the report will answer the evaluation questions formulated below and provide conclusions and recommendations. The supplementary attachment provides the list of loans, grants, and TA for both sovereign and nonsovereign projects that will be the object of the evaluation. In addition, the projects approved prior to 2009 but were closed and evaluated during the period 2009–2018 will also be included in the object of this evaluation.
2. Theory of Change
32. A preliminary theory of change (TOC) aims at identifying cause–effect links on how ADB contributes to energy sector development of DMCs, towards achieving economic growth, improving welfare, protecting the local and global environment, and supporting climate change adaptation and mitigation (Figure 2). IED has prepared this TOC based on key elements of the 2009 Energy Policy, as approved by the Board, and after consultation with operations departments, the energy sector group and other relevant internal ADB stakeholders. The TOC shows the cause-effect relations assumed by the Energy Policy that lead to achieve the expected results, through the implementation of the energy program.
10
Program Inputs
Financing instruments and investments
Technical assistance
Policy dialogue
Transaction advisory and support
Cofinancing mobilization
Knowledge products
Regional coordination
Program Outputs
Energy infrastructures (CE, T&D, EE, RE, off-grid)
Sector and regulatory reform
Institutional and market development
Knowledge transfer
Technical and operational capacity
Efficient and financially sustainable energy institutions
Improved policies and standards Enhanced energy markets Improved cost recovery Enabling environment for private sector
participation
ADB Context Staff
resources
Organizational structure
Strategies, policies, and initiativesb
Knowledge
Partnerships
External Context
DMCs challenges and
needs
Ongoing energy
transformation
Global agreementsc
Regional agreementsd
Global energy markets
ADB Energy Policy
Impacts
Increased socioeconomic growth in an inclusive and sustainable manner
Increased
energy security
Reduced
impact on climate change and the environment
Energy Program
Guide Deliver
Achieve
Congruent
Inform
Define
Define
Scope of Evaluation
Improved energy efficiency and use of renewable energies
Increased cross-border energy trade Increased connections to energy services
(cooking gas, grid-based electricity, heating, natural gas) for new consumersa
Reliable and affordable energy supply
Greater integration of renewable energy Improved demand-side and supply-side
energy efficiency
Intended Program Outcomes
Efficient and viable energy sector
Expanded access to energy
Stated Policy Outcomes
Con
trib
ute
to
Figure 2: Preliminary Theory of Change
ADB = Asian Development Bank, CAREC = Central Asia Regional Economic Cooperation, CE = conventional energy, DMC = developing member country, EE = energy efficiency, GMS = Greater Mekong Subregion, MDG = Millennium Development Goal, RE = renewable energy, SASEC = South Asia Sub-regional Economic Cooperation, T&D = transmission and distribution. a Energy services refer to grid-based and off-grid electricity supply, as well as modern heating and cooking fuels and technologies. b Includes the multipronged clean energy program that seeks to (i) increase regional energy efficiency in energy, transport, and urban sectors; (ii) adopt renewable energy sources,
and (iii) improve access to energy for the poor and remote regions. Also included are initiatives for Carbon Capture and Storage, Energy for All Initiatives, Asia Solar Energy Initiative, Quantum Leap in Wind, Small Wind Initiative, Low Carbon Technology Exchange, Asia Climate Change, and Clean Energy Venture Capital Initiative. Additionally, ADB has a Carbon Market Program, its three components being Carbon Funds, Technical Support Facility, and the Credit Marketing Facility.
c MDGs, Kyoto, and other various multilateral economic, social, and environmental agreements. d CAREC, GMS, SASEC, and other various regional economic, social, and environmental agreements. Source: Asian Development Bank (Independent Evaluation Department).
11
33. The preliminary TOC starts with ADB’s inputs in the form of sovereign and nonsovereign financing, technical assistance, mobilization of cofinancing, regional coordination, transaction advisory, knowledge products, and policy dialogue. These inputs are expected to deliver outputs that include energy infrastructures (i.e., electricity transmission and distribution, conventional energy generation [fossil fuel-based], large hydropower and other renewable energy generation, energy efficiency and conservation, oil and gas transmission and distribution, and off-grid technologies); sector development and institutional reform; institutional and market development; knowledge transfer; and improved technical and operational capacity. These outputs would help achieve the various intended program outcomes that include efficient and financially sustainable energy institutions; improved policies and standards; enhanced energy markets; improved cost recovery; enabling environment for private sector; increased cross-border energy trade; increased access to energy services for new consumers; reliable and affordable energy supply; expanded access to modern energy services; greater integration of renewable energy; and improved demand-side and supply-side energy efficiency. These intended outcomes would be consistent with the outcomes stated in the Energy Policy, which are (i) an efficient and viable energy sector, (ii) expanded access to energy, and (iii) improved energy efficiency and use of renewable energies.
34. The intended program outcomes and the stated energy policy outcomes, however, can be influenced by several factors:
(i) Contextual factors, which normally help define inputs and outputs, but not necessarily guarantee the intended outcomes or policy outcomes. Such factors include: (a) DMCs challenges and needs, for example, the sector reforms and restructuring intended to improve performance of energy institutions or to create competitive energy markets may not have the requisite political support which lets vested interests come in the way; (b) inability to adopt emerging energy technologies given weaknesses in existing energy networks and sector regulations; (c) volatility in global energy markets which can have adverse fiscal implications for the country, and which call for immediate attention from the government; and (d) global and regional agreements that can tend to change energy sector development pathways.
(ii) ADB attributes, which also contribute to defining inputs and outputs in a bid to achieve intended outcomes and energy policy outcomes. Such attributes include ADB’s corporate strategy, various sector and thematic directional documents and relevant energy initiatives,35 available skill sets within ADB, and internal organization mechanisms that enable ADB to respond effectively to DMCs needs and priorities, ADB’s knowledge management systems, and its partnership arrangements for finance and knowledge.
3. Evaluation Questions
35. The main evaluation questions of this evaluation are guided by the TOC. The overarching question relates to the role of the policy and achievement of the policy outcomes. Other underpinning questions address program output linkages, organization for delivery issues, and other topics. The detailed evaluation questions are presented in Attachment 2.
36. The evaluation will focus in answering the overarching question: To what extent has ADB’s 2009 Energy Policy and the subsequent program been relevant, effective, sustainable, and
35 These initiatives include (among others) the Energy Efficiency Initiative and Carbon Market Program.
12
efficient for meeting DMCs’ needs and requirements and will continue to be so in the foreseeable future?
37. The overarching question is underpinned by three supporting questions:
(i) To what extent ADB’s 2009 Energy Policy and the subsequent program have been relevant, effective, sustainable, and efficient for meeting DMCs’ needs and requirements? (a) To what extent ADB’s 2009 Energy Policy has been relevant for meeting
DMCs’ needs and requirements, other MDBs’ strategies, and the global energy context? (1) To what extent the ADB Energy Policy has been relevant to the
overall economic, social, environmental, and energy sector contexts of DMCs?
(2) To what extent the ADB Energy Policy has facilitated and supported the global energy sector transformation, in particular, the transition toward low-carbon technologies in the context of changing technologies and systems?
(3) To what extent the ADB Energy Policy has been consistent with ADB's corporate strategies, policies, and operational plans?
(4) To what extent the ADB Energy Policy has been consistent with directional documents of other MDBs and international financial institutions?
(5) To what extent the ADB Energy Policy has been sustainable in terms of enabling and/or guiding ADB support as the DMCs’ energy sector and institutions evolve, minimizing risk of having stranded assets in the energy system over the medium to long term, and supporting demonstration projects (that have been replicated)?
(6) To what extent the ADB Energy Policy has supported greater private sector participation in the energy sector?
(b) To what extent the ADB’s energy program in 2009–2018 has been relevant, effective, efficient, and sustainable for meeting DMCs’ needs and requirements? (1) To what extent the ADB’s energy program has effectively and
sustainably met the requirements of DMCs to date? (2) To what extent the ADB’s energy program has effectively improved
the financial viability of utilities to reduce fiscal burden on governments, promoted regional cooperation and electricity interconnection, reduced air pollution, and mitigated climate change?
(3) To what extent the completed and validated projects were relevant, effective, efficient, and sustainable ?
(4) What is the outlook for other completed projects (with or without validated completion reports) and ongoing (mature) projects regarding their relevance, effectiveness, efficiency, and sustainability ?
(5) What are the key findings, issues, and lessons from all studied projects regarding their relevance, effectiveness, efficiency, and sustainability ?
13
(ii) To what extent the ADB Energy Policy continues to be aligned with current and likely future DMCs’ needs and requirements, and responds to the evolving global energy context and Strategy 2030? (a) To what extent shall the ADB Energy Policy remain relevant to the overall
economic, social, environmental, and energy sector contexts of DMCs? (b) To what extent shall the ADB Energy Policy continue to inform the country
partnership strategies and effectively guide the ADB’s energy program to address the DMCs’ upcoming challenges?
(c) To what extent shall the ADB Energy Policy remain consistent with ADB’s corporate strategies, policies, and operational plans?
(d) To what extent shall the ADB Energy Policy continue to facilitate and support the global energy sector transformation, in particular, the transition toward low-carbon technologies in the context of changing technologies and systems?
(e) To what extent shall the ADB Energy Policy continue to support greater private sector participation in the energy sector?
(iii) To what extent has ADB’s institutional structure and its human resources
supported the implementation of the ADB Energy Policy and the subsequent program? (a) To what extent has ADB institutional structure contributed to improving the
efficiency in delivering the energy program? (b) To what extent have sector divisions, regional departments, private sector
department, knowledge departments, and other ADB units collaborated to deliver the energy program and engaged in policy dialogue?
(c) To what extent were staff skills complemented by external consultants and knowledge partnerships (with global and regional centers of excellence in energy) to design and deliver the energy program plus engage in policy dialogue?
(d) Regarding knowledge and innovation, what was the role of ADB and ADB Institute in supporting research, innovation, technology transfer, knowledge events and demonstrative projects?
(e) What was the quality at entry of projects and whether risks assessments and DMFs were appropriately defined at project appraisal and used during project implementation?
D. Evaluation Structure
38. To answer these questions, the evaluation will undertake the following analyses and assessments.
39. Policy relevance. The evaluation will look into the contextual factors at the time the policy was designed, as well as an understanding of the goals set in the Energy Policy that have guided the design of ADB’s energy programs and projects.
(i) Policy context. This is best understood by examining the global agendas at the time the policy was being formulated in 2008 (e.g., climate change and the Kyoto Protocol, Millennium Development Goals, and other global and regional agreements), issues specific to the Asia and Pacific region, ADB’s corporate Strategy 2020, non-energy sector policies and operational plans, ADB’s other directional documents (such as on safeguards, gender, and environment), and
14
relevant energy and other policies and strategies of other MDBs and international financial institutions. The evaluation will also focus on understanding the extent to which the energy policy continues to be aligned with current and future DMCs needs and requirements, and responds to the evolving global energy context and Strategy 2030. Further, the status of energy technologies at the time of the policy preparation and the evolution of the energy sector in the past decade will be analyzed in relation to climate change, environmental implications, socioeconomic development, energy security, and other factors. In particular, the use of coal and fossil fuels (including natural gas) for power generation and heating, as well as industrial uses will be reviewed. Also, echoing recent views of various stakeholders on the use of nuclear power generation to contribute to NDC targets;36 this report will synthesize recent studies and data. Also, current trends of other energy technologies, such as hydrogen or energy storage, will be included as part of the analysis.
(ii) Policy goals. These are best understood by examining how clearly the goals are articulated in the energy policy, whether they are consistent with the policy context, whether the goals are too broad or is something missing, as well as the underlying logic for how the policy was expected to produce intended changes (i.e., following the TOC). This section will also evaluate the policy’s DMF, including outcome and output definitions, indicators, and targets.
40. Energy program. This will require an evaluation of energy program subsequent to the 2009 Energy Policy, i.e., implementation of ADB supported sovereign, nonsovereign, technical assistance, advisory, knowledge dissemination, partnerships with other institutions, and other types of interventions under regional, multi-country and country programs. This will cover the following:
(i) Relevance and linkages with energy policy and other strategies by examining the consistency of country partnership strategy (CPS) and regional subprogram objectives with the energy policy, as well as examining the alignment of the energy portfolio with the energy policy and country strategies.
(ii) Effectiveness of the energy policy by assessing the extent to which the energy policy guided project screening and selection, as well as investigating changes in CPS, country operational plans, and portfolio approvals before and after 2009. This will include a review of the extent to which the ADB energy portfolio has supported the achievement of the intended outcomes of the 2009 Energy Policy. These outcomes include efficient and viable energy sector, expanded access to energy, and improved energy efficiency and use of renewable energy. Special attention will be given to climate change and environmental sustainability as well as compliance with social and environmental safeguards policies, and how the ADB’s energy program supports gender equity.
(iii) Efficiency of the energy policy and program, largely from estimated economic internal rates of return ex-ante and/or at completion, as well as estimating time and cost overruns from available project level information.
(iv) Sustainability, by assessing to what extent the program outcomes will be sustained over time and the results of policy reform and institutional actions have been implemented. The analysis will be performed on a consolidated manner by using data from evaluation documents at project, program and sector levels.
36 It is noted however, that in ADB’s Safeguards Policy Statement of 2009, nuclear is included on its prohibited
investment activities list.
15
41. Operational Priorities of Strategy 2030. Recent changes across the global energy sector, including disruptive technologies, new business models, the SDGs, and the Paris Agreement on climate change in 2015 (including the NDCs which are expected to become increasingly ambitious in the coming decades), will affect how ADB designs the new Energy Policy. The evaluation will analyze the extent to which the 2009 Energy Policy continues to be aligned with current and likely future DMCs’ needs and requirements, and responds to the evolving global energy context and Strategy 2030. Although the approved energy program was not guided by Strategy 2030, as an input to understanding the readiness of the bank with regards to the upcoming implementation of Strategy 2030, the evaluation will, through illustrated examples, provide insights on consistency of the existing energy program with Strategy 2030. Economic literature on the seven operational priorities, along with this portfolio analysis, will be useful in identifying possible changes in emphasis that may be required in the coming years.
42. Organization for delivery. The evaluation will examine the extent to which existing ADB staff, protocols, and reporting structures are geared toward performing efficiently and effectively in an environment that is characterized by rapid technological change, as well as changing government and client expectations and ADB’s competitive environment. The evaluation will examine the adequacy of ADB’s sectoral structure to deliver energy projects, including cross-sectoral interventions and private sector participation. The report will look in detail at the following:
(i) Quality at entry, and whether risk assessments and appropriate DMF are properly defined at project appraisal and used during project implementation.
(ii) Staff skills and reliance on external consultants to deliver the project portfolio, as well as to engage in a meaningful policy dialogue (outside CPS boundaries) with the clients.
(iii) Internal collaboration among sectoral divisions, regional departments, as well as knowledge and private sector groups.
(iv) Collaboration and partnerships with global and/or regional centers of excellence in the energy sector and use of innovative approaches.
(v) Knowledge and innovation, and the role of ADB (and ADB Institute) as a knowledge center, and the role it plays in supporting research, innovation, technology transfer, knowledge events, and demonstrative projects in the energy sector.
E. Data and Evaluation Methods
43. The study period has been defined as the time period that the existing Energy Policy has been in force, i.e., 2009–2018. The most salient features of evaluation components and methods are portfolio analysis, case assessments in selected countries, lessons from the 2007 evaluation of the previous energy policy, interviews with stakeholders in ADB and case assessment countries, literature review on emerging energy technologies and systems and their interface with operational priorities outlined in Strategy 2030, review of ADB documents that spell out directions and strategies at the country, review of regional and subregional levels as well as project documents, and an ADB staff survey at headquarter and resident missions to gauge staff perception on resources and skills adequacy, interdepartmental coordination, and One ADB approach, and strategy and operations alignment. Some of these evaluation components can be elaborated at this early stage.
44. Literature review. This will help understand the context that will influence the ADB’s energy sector operations related to (among others) coal, other fossil fuels, nuclear, renewable energy, and hydrogen technologies. Economic, technical, political, institutional, and social
16
perspectives will be considered. Information will be gathered from leading specialized energy institutions (e.g., Asia-Pacific Energy Research Center, International Energy Agency, International Renewable Energy Agency [IRENA], and World Energy Council) as well as other sources (e.g., Bloomberg New Energy Finance, McKinsey and Company) and academia.
45. Portfolio analysis. The portfolio will comprise sovereign and nonsovereign technical and financial assistance projects approved and under implementation during 2009–2018 that are classified under the energy sector. To assess whether or not there was a discernable change in the ADB’s energy portfolio after the 2009 Energy Policy came into effect, the portfolio analysis will also cover energy projects approved before 2009 but were under implementation when the energy policy became effective. To enable an understanding of the extent ADB has helped synergize energy development with other development priorities (e.g., tackling climate change, making cities more livable, improving food security, environmental conservation, and enhancing regional cooperation) the portfolio will attempt to include interventions classified under other sectors (as per ADB’s project classification system) but have some explicitly stated energy activity, output, or outcome.
46. Country case studies. Considering ADB’s support, the evolution of the energy sector in the past several years, and the future outlook, this review will include specific case studies of selected DMCs in order to provide detailed analysis of specific issues and lessons learned in each case. These country assessments include Mongolia (East Asia region), India (South Asia region), Uzbekistan (Central and West region), and Tonga (Pacific region). These will be complemented by recent sector and/or country assessments carried out by IED within the last 4 years, including Indonesia CAPE (under preparation, Southeast Asia region); Pakistan SAPE and Azerbaijan CAPE (2019, Central and West region); India CAPE (2017, South Asia region); Sri Lanka CAPE (2016, South Asia region); People's Republic of China CAPE (2015, East Asia region); and Papua New Guinea CAPE (2015, Pacific region).
47. Other evaluation evidence. In order to address thematic and corporate issues related directly or indirectly with the energy program, this study will build on IED’s evaluation evidence of applicable thematic and corporate issues, such as safeguards, gender, climate change, environmental sustainability, knowledge, regional cooperation, governance, specific modalities, engagement with middle-income countries, and partnerships.
48. Interviews with key stakeholders. The evaluation team will meet with key ADB staff at headquarters and the resident missions to gauge strategic and operational perspectives on ADB’s energy sector interventions. Key stakeholders in government agencies during country visits, client public sector entities and private sector participants will be interviewed to gauge their perspectives on energy sector development and needs, ADB strategies and interventions, as well as on ADB’s efficiency and institutional matters. Additionally, other MDBs and international organizations (e.g., WBG, IADB, IRENA), bilateral donors, think-tanks, academics, and civil society organizations will also be interviewed to gather information on issues of coordination and collaboration. The evaluation will also benefit from participating in international energy conferences during the drafting period (e.g., 24th World Energy Congress, September 2019).
49. Staff surveys. ADB staff working on the design and/or implementation of sovereign and nonsovereign financial and technical assistance energy sector programs and projects, will be surveyed anonymously to gauge among others, the following: (i) the extent of reliance on external consultants, (ii) skills existing inhouse in energy divisions and other relevant divisions of regional departments as well as skills sought from consultants; (iii) perspective on skills required going forward, (iv) skills available in the Energy Sector Group and other relevant sector groups and
17
thematic groups, (v) collaboration and information exchanges among departments (the One ADB approach), and (vi) whether partner organizations have provided value added services (e.g., guidance during energy program and project design stage).
F. Implementation Arrangements and Resource Requirements
50. The evaluation is expected to be completed for final circulation to the Board of Directors for the Development Effectiveness Committee meeting in June 2020. The proposed timeline is shown in Table 3.
Table: Proposed Evaluation Timeline
Activity Tentative Schedule Approval of evaluation approach paper III July 2019 Case-study country visits II August–I October 2019 IED management review meeting II December 2019 Interdepartmental circulation II January 2020 Heads of departments meeting II March 2020 Technical meeting on recommendations III March 2020 Editing III March–II April 2020 IED Director General Approval III April 2020 Board Circulation IV April 2020 DEC meeting I June 2020 DEC = Development Effectiveness Committee, IED = Independent Evaluation Department. Source: Asian Development Bank (Independent Evaluation Department).
51. The evaluation will be conducted by a team comprising Alfredo Baño Leal, Kapil Thukral (until September 2019), Sherine Ibrahim (from July 2019), Lawrence Nelson Guevara, and Elizabeth Li-Mancenido. The IED staff will be supported by external consultants, including an MDB policy and strategy expert, a country partnership strategy expert, an energy policy and regulatory expert, an energy technology expert, two national consultants at ADB headquarters to support research and data gathering, as well as national consultants to support country missions.
52. For the final report, external peer reviewers will represent both technical, operational, and evaluation perspectives from development organizations and energy think tanks. The external peer reviewers will be Mr. Lucio Monari, Director of Energy and Extractives Global Practice of the World Bank; Prof. David Newbery, Director of the Energy Policy Research Group of the University of Cambridge; and Dr. Kazumoto Irie, President of the Asia Pacific Energy Research Center (affiliated to the Institute of Energy Economics of Japan).
53. The evaluation findings will be disseminated within ADB and externally. After the Development Effectiveness Committee discussion of the final report, knowledge sharing and learning activities, including publication of learning lessons materials, dissemination seminars, and presentations in conferences within (e.g., in consultation with the thematic group), and outside ADB (e.g., the Asian Evaluation Week) will follow.
Attachments: 1. ADB’s Energy Program, 2009–2018 2. Detailed Evaluation Questions
Supplementary Attachment: 1. List of ADB’s Sovereign and Nonsovereign Projects for Energy Sector, 2009–2018
(available upon request)
18 Attachment 1
ADB’S ENERGY PROGRAM, 2009–2018
A. Energy Sector Portfolio
1. Energy sector operations from 2009 to 2018 represented one-fourth of total approvals. For the evaluation period 2009–2018, total Asian Development Bank (ADB) approved operations amounted to $159.7 billion, of which, more than half supported infrastructure, mainly transport and energy sectors. Overall approved operations in the energy sector during the period amounted to $40.0 billion or one-fourth of the total ADB portfolio, ranking second to the transport sector with approvals amounting to $42.5 billion (26.6%, Figure A1.1).
2. Since 2015, the volume of energy approved operations has overtaken that of the transport sector. In 2009, the energy sector ranked only third in terms of amount of approved operations, after transport and public sector management sectors. The volume of energy approvals has since ranked second to the transport sector from 2010 to 2014. Starting 2015, approved operations in the energy sector already dominates the ADB portfolio. In fact, the level of energy sector approvals more than doubled from $2.9 billion in 2014 to $6.2 billion in 2015. The spike in the approvals in 2015 was led by the approval of the $1 billion Green Energy Corridor and Grid Strengthening Project in India (Loans 3365 and 3375) and the $575 million results-based loan for the Electricity Grid Strengthening for Sumatra Program in Indonesia (Loan 3339), among others. Although the level of approvals had declined to $3.6 billion in 2018, energy sector approvals remained the highest among ADB’s 11 sectors (Figure A1.2).
Figure A1.1: Overall Approved ADB Projects by Sector, 2009–2018
ADB = Asian Development Bank; ANRRD = agriculture, natural resources, and rural development; ICT = information and communication technology; PSM = public sector management; WOUIS = water and other urban infrastructure and services. Sources: ADB Listing of loans, grants, and equity approvals database; Loans, grants and technical assistance details database from ADB Controller Department as of 31 Dec 2018; ADB e-operations database; project documents.
ANRRD7.6%
Education4.7%Energy
25.0%
Finance9.7%
Health1.4%
Industry & trade1.7%
ICT0.3%
Multisector0.8%
PSM12.3%
Transport26.6%
WOUIS9.9%
Attachment 1 19
3. Support to energy sector was diversified. Figure A1.3 shows that the ADB energy approved portfolio during 2009–2018 comprised of a mix of sovereign loans ($27.7 billion, 69%), grants ($2.7 billion, 6.7%), and technical assistance ($0.4 billion, 1%). Sovereign support had been delivered in three results-based loans (two for Indonesia and one for Pakistan),1 four program loans,2 eight sector loans, and 20 multitranche financing facility projects. While sovereign loans had been the major vehicle of support, the nonsovereign operations for the energy sector had also been increasing. Since 2015, energy approvals in the nonsovereign operations has hovered above the $1 billion mark taking off from an average of less than $600 million during 2009–2014. Total nonsovereign operations in the energy sector amounted to $9.2 billion (23%) of the total ADB energy portfolio during the period.
1 These include the (1) Electricity Grid Strengthening Program for Sumatra (Loan 3339-INO funded under Ordinary
capital resources and Loan 8297 under AIF), (2) Sustainable Energy Access in Eastern Indonesia (Electricity grid Development Program, Loan 3560-INO), and the Access to Clean Energy Investment Program (Loan 3476-PAK).
2 Includes the (i) Indonesia Sustainable Energy Sector Reform Program (Loan 2805-INO), (ii) Pakistan Sustainable Energy Sector Reform Program (Loans 3126-PAK, 3321-PAK, 3322-PAK and 3537-PAK), (iii) Sustainable and Inclusive Energy Program (Loans 3303-INO, 3561-INO, 8293-INO, 8330-INO), and (iv) the Beijing-Tianjin-Hebei Air Quality Improvement Program (Loan 3356-PRC).
Figure A1.2: Annual Approved ADB Projects Values by Sector, 2009–2018 ($ million)
ADB = Asian Development Bank; ANRRD = agriculture, natural resources, and rural development; PSM = public sector management; WOUIS = water and other urban infrastructure and services. Note: Others category is formed by health, education, industry and trade, information and communication technology, and multisector projects. Sources: ADB Listing of loans, grants, and equity approvals database; Loans, grants, and technical assistance details database from ADB Controller Department as of 31 Dec 2018; ADB e-operations database; project documents.
-
1,000
2,000
3,000
4,000
5,000
6,000
7,000
2009 2010 2011 2012 2013 2014 2015 2016 2017 2018
Energy Transport ANRRD PSM Finance WOUIS Others
Figure A1.3: ADB Approved Energy Operations by Instrument, 2009–2018
ADB = Asian Development Bank, TA = technical assistance. Sources: ADB Listing of loans, grants, and equity approvals database; Loans, grants and TA details database from ADB Controller Department as of 31 December 2018; ADB e-operations database; project documents.
Sovereign Loans69.3%
Nonsovereign22.9%
Grants6.7%
TA1.1%
20 Attachment 1
4. On average, ADB approved $3.9 billion worth of loans, grants, technical assistance projects, and nonsovereign operations for the energy sector annually, reaching more than $6 billion in 2015 (Figure A1.4).
5. Electricity transmission and distribution continues to dominate the portfolio, but the share of other subsectors has increased in the last 4 years. In terms of subsector focus, majority of the approved energy operations during 2009–2018 supported the electricity transmission and distribution subsector ($17.0 billion, 43%), renewable energy generation ($5.3 billion, 13%),3 conventional energy generation ($6.0 billion, 15%), energy efficiency and conservation ($4.1 billion, 11%), energy sector development and institutional reform ($2.9 billion, 7%), and large hydropower generation ($2.1 billion, 5%) among other subsectors (Table A1.1). Renewable energy generation is comprised of biomass and waste ($0.4 billion), geothermal ($1.1 billion), small hydro ($0.7 billion), solar ($1.8 billion), wind ($1.1 billion), and other renewable energy initiatives ($0.2 billion).4 Conventional energy on the other hand includes coal ($2.1 billion), gas ($3.2 billion), oil ($0.1 billion) and other sources ($0.7 billion).5
3 Includes renewable energy from different sources—biomass and waste, geothermal, small hydro, solar, and wind. 4 Includes interventions that support not just one renewable energy source but a combination of two or more sources
(biomass and waste, geothermal, small hydro, solar, and wind) and also interventions that support renewable energy in general, i.e., without any specific source at the time of approval, which is more common for technical assistance projects.
5 Includes interventions that support a combination of two or more conventional energy source (coal, gas, and oil) as well as those that were supporting conventional energy generation in general, i.e., unspecified at the time of approval as in the case of some technical assistance projects.
Figure A1.4: ADB Annual Approved Energy Operations by Instrument, 2009–2018 ($ million)
ADB = Asian Development Bank, TA = technical assistance. Sources: ADB Listing of loans, grants, and equity approvals database; Loans, grants and TA details database from ADB Controller Department as of 31 Dec 2018; ADB e-operations database; project documents.
-
1,000.0
2,000.0
3,000.0
4,000.0
5,000.0
6,000.0
7,000.0
2009 2010 2011 2012 2013 2014 2015 2016 2017 2018
Sovereign Loans Nonsovereign Sovereign Grants TA
Attachment 1 21
Table A1: Approved ADB Energy Operations by Subsector, 2009–2018 ($ million)
Subsector Sovereign
Loans Sovereign
Grants TA Nonsovereign Total Percentage Conventional energy
generation 2,965 63 17 2,922 5,967 15% Coal 1,938 5 1 120 2,064 5% Gas 860 2 6 2,312 3,179 8% Oil - 53 - - 52 0% Othera 167 5 11 490 673 2%
Electricity transmission and distribution
14,172 1,839 81 870 16,961 42%
Oil and gas transmission and distribution
493 - 5 753 1,251 3%
Energy utility services 680 32 12 503 1,227 3% Energy efficiency and
conservation 3,778 30 78 280 4,166 10%
Energy sector development and institutional reform
2,498 12 137 300 2,947 7%
Large hydropower generation
1,387 277 15 406 2,084 5%
Renewable energy generation
1,726 417 93 3,128 5,364 13%
Biomass and waste 66 12 6 344 428 1% Geothermal 599 - 3 498 1,099 3% Small hydro 391 242 10 84 726 2% Solar 461 156 39 1,141 1,798 4% Wind 209 - 10 887 1,106 3% Otherb - 6 25 175 206 1%
Total 27,699 2,671 437 9,161 39,967 ADB = Asian Development Bank, TA = technical assistance. a Other conventional energy subsector category includes interventions that support a combination of two or more
conventional energy source (coal, gas, oil) as well as those that were supporting conventional energy generation in general, i.e. unspecified at the time of approval as in the case of some technical assistance projects.
b Other renewable energy generation subsector category. Source: Asian Development Bank. 6. The 3-year moving average of the annual share of energy approvals by subsector shows that while electricity transmission and distribution dominate the ADB energy portfolio, the share has been relatively flat especially during the last 4 years (Figure A1.5). By contrast, the share of conventional energy generation (fueled mostly by gas-fired power plants) has been increasing during the last 3 years. The share of renewable energies is relatively flat, as large hydropower has decreased in the past few years.
22 Attachment 1
7. Geographically, $13.5 billion or more than one-third of the total energy portfolio during the period were approved for South Asia at the regional level, mainly driven by Bangladesh, India, and Sri Lanka (Figure A1.6). Central and West Asia accounted for $12.2 billion (30%) notably, Afghanistan, Pakistan, and Uzbekistan. Energy approvals in Southeast Asia amounted to $9.4 billion (23%), while the remainder portfolio is divided among East Asia ($3.9 billion, 10%), inter-regional projects ($0.6 billion, 2%), and the Pacific ($0.4 billion, 1%). At the country level, energy approvals in India ranked highest at $7.4 billion (Figure A1.7).
Figure A1.5: ADB Approved Energy Portfolio, 2007–2018 (3-Year Moving Average)
ADB = Asian Development Bank. Source: Asian Development Bank (Independent Evaluation Department).
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
2008-2010 2009-2011 2010-2012 2011-2013 2012-2014 2013-2015 2014-2016 2015-2017 2016-2018
Conventional energy generation Electricity transmission and distributionOil and gas transmission and distribution Energy efficiency and utility servicesSector development and institutional reform Renewable energies and large hydropower
Figure A1.6: ADB Approved Energy Portfolio by Region, 2009–2018
EA = East Asia, CW = Central and West Asia, REG = regional, SA = South Asia, SEA = Southeast Asia. Source: Asian Development Bank.
CW30%
EA10%
Pacific1%
REG2%
SA34%
SEA23%
Attachment 1 23
4. Other Sector’s Portfolio with Energy Related Interventions
8. In addition to the core energy projects or those that are classified mainly under the energy sector, there are also other projects which have energy components or classified not exclusively as energy but with other sectors. During the period 2009–2018, a total of 29 sovereign loans were identified to have energy components amounting to a total of $583 million. Half of the non-core energy projects were derived from public sector management projects (Figure A1.8). The energy components from these projects which are considered here as non-core energy projects range from 4% to as much as 51% or an average of 24.6% of the total approved ADB amount. 9. More than half of the non-core energy project components ($298 million) supported the energy sector development and institutional reform subsector while one-fourth focused on electricity transmission and distribution ($144 million, Figure A1.9). The remainder also supported energy utility services ($121 million) and energy efficiency and conservation ($20 million).
Figure A1.7: ADB Energy Approvals by Country, 2009–2018 ($ million)
ADB = Asian Development Bank, AFG = Afghanistan, BAN = Bangladesh, IND = India, INO = Indonesia, MON = Mongolia, PAK = Pakistan, PNG = Papua New Guinea, PRC = People’s Republic of China, SRI = Sri Lanka, THA = Thailand, TON = Tonga, UZB = Uzbekistan, VIE = Viet Nam. Notes: 1. Figure includes top 10 recipient countries and top 2 countries from each region if not included in the top 10. 2. Blue = South Asia; Red = Central and West Asia; Yellow = East Asia; Green = Southeast Asia; Brown = Pacific. Sources: ADB Listing of loans, grants, and equity approvals database; Loans, grants and technical assistance details database from ADB (Controller Department) as of 31 December 2018; ADB e-operations database; project documents.
7,430.56
5,211.23
3,920.94
3,812.77
4,133.33
2,578.82
2,237.68
1,580.23
1,436.62
1,126.75
200.07
89.68
42.85
- 1,000.00 2,000.00 3,000.00 4,000.00 5,000.00 6,000.00 7,000.00 8,000.00
IND
PAK
BAN
PRC
INO
UZB
VIE
AFG
THA
SRI
PNG
MON
TON
24 Attachment 1
Figure A1.8: ADB Approvals for Non-Core Energy by Main Sector, 2009–2018
ADB = Asian Development Bank; ANRRD = agriculture, natural resources, and rural development; PSM = public sector management; WOUIS = water and other urban infrastructure and services. Sources: ADB Listing of loans, grants, and equity approvals database; Loans, grants and technical assistance details database from ADB Controller Department as of 31 Dec 2018. ADB e-operations database; project documents.
ANRRD 3.1%
Industry & trade 19.1%
Multisector 0.3%
PSM 50.8%
WOUIS 19.3%
Figure A1.9: ADB Approvals for Non-Core Energy by Energy Subsector, 2009–2018
ADB = Asian Development Bank; ANRRD = agriculture, natural resources, and rural development; PSM = public sector management; WOUIS = water and other urban infrastructure and services. Sources: ADB Listing of loans, grants, and equity approvals database; Loans, grants, and technical assistance details database from ADB Controller Department as of 31 Dec 2018. ADB e-operations database; project documents.
Electricity transmission and
distribution25%
Energy efficiency and conservation
3%
Energy sector development and institutional reform
51%
Energy utility
services21%
Attachment 2 25
DETAILED EVALUATION QUESTIONS
Overarching Question
To what extent has the 2009 Energy Policy of Asian Development Bank (ADB) and the subsequent program been relevant, effective, sustainable, and efficient for meeting developing member countries’ (DMCs) needs and requirements and will continue to be so in the foreseeable future?
Supporting Questions
1. To what extent ADB’s 2009 Energy Policy and the subsequent program have been relevant, effective, sustainable, and efficient for meeting DMCs’ needs and requirements? (i) To what extent ADB’s 2009 Energy Policy has been relevant for meeting DMCs’
needs and requirements, other multilateral development banks’ (MDBs) strategies, and the global energy context? (a) To what extent the ADB Energy Policy has been relevant to the overall
economic, social, environmental, and energy sector contexts of DMCs? (1) To what extent the ADB Energy Policy has been relevant to DMCs’
socioeconomic structures (e.g., trends in gross domestic product level and composition, rural vs. urban populations and trends)?
(2) To what extent the ADB Energy Policy has led DMCs to develop their energy sectors (energy resources, meet energy demand, reliability of energy supply, institutional and regulatory frameworks, extent of private participation, and relevant international commitments)?
(3) To what extent the ADB Energy Policy has guided DMCs to define their commitments and obligations regarding improving the local environment, mitigating global climate change, and having disaster resilient energy infrastructure?
(4) To what extent the ADB Energy Policy has influenced DMCs’ medium- to long-term plans (if any) for developing and strengthening their energy sectors, and changing their energy priorities since 2009?
(b) To what extent the ADB Energy Policy has facilitated and supported the global energy sector transformation, in particular, the transition toward low-carbon technologies in the context of changing technologies and systems? (1) To what extent the ADB Energy Policy has influenced the increase
of clean and renewable energy share in ADB energy portfolio since 2009?
(2) To what extent the ADB Energy Policy has supported the adoption of emerging technologies and systems (smart grid technology, electric vehicles)?
(3) To what extent the ADB Energy Policy has supported the emerging market mechanisms and business models, and emerging regulatory frameworks?
(4) To what extent the ADB Energy Policy has led to improving the supply and demand-side energy efficiency, improving access to energy, and improving the operational efficiency of energy utilities?
(5) To what extent the ADB Energy Policy has led to improving the financial viability of energy enterprises?
26 Attachment 2
(6) To what extent the ADB Energy Policy has led to increasing the regional cooperation and electricity interconnection?
(7) To what extent did the ADB Energy Policy seek to provide better energy services in DMCs’ rural and urban areas?
(c) To what extent the ADB Energy Policy has been consistent with ADB’s corporate strategies, policies, and operational plans? (1) To what extent the ADB Energy Policy has been consistent with
various sector and thematic policies and operational plans in force in mid-2009 and articulated since then, including those that remain in effect to date while the corporate strategy has changed?
(2) To what extent did the ADB Energy Policy seek to focus ADB energy program to the operational priorities of Strategy 2020?
(d) To what extent the ADB Energy Policy has been consistent with directional documents of other MDBs and international financial institutions? (1) To what extent the ADB Energy Policy has been consistent with the
directional documents and policies of other MDBs and international financial institutions, including those in force in mid-2009 and those in force now?
(e) To what extent the ADB Energy Policy has been sustainable in terms of enabling and/or guiding ADB support as the DMCs’ energy sector and institutions evolve, minimizing risk of having stranded assets in the energy system over the medium to long term, and supporting demonstration projects (that have been replicated)? (1) To what extent the ADB Energy Policy has been sustainable in
terms of demonstrating new technologies, business models, and other measures that have been replicated for improving indoor and ambient air quality and increase regional cooperation?
(2) To what extent the ADB Energy Policy has been sustainable in terms of minimizing risk of having stranded assets in the energy system in the medium to long term (whether or not these were created with ADB support)?
(3) To what extent the ADB Energy Policy has been sustainable in terms of enabling and/or guiding support to DMCs’ energy sector and energy institutions as they evolve (for example, from transmission system expansion to strengthening and/or upgrading and better operation and maintenance practices)?
(4) To what extent the ADB Energy Policy has enabled country partnership strategies (CPSs) and energy program to align with changing government energy priorities since 2009?
(5) To what extent do CPSs with a particular DMC aim to improve energy efficiency and increase penetration of renewable energy?
(6) To what extent do CPSs with a particular DMC aim to increase access to affordable and quality of energy supply?
(7) To what extent do CPSs with a particular DMC aim to improve the operational efficiency and financial viability of energy sector enterprises?
(8) To what extent do energy portfolios in a particular DMC support energy policy outcomes and/or other outcomes beyond those articulated in the energy policy (e.g., improving ambient air quality)?
(f) To what extent the ADB Energy Policy has supported greater private sector participation in the energy sector?
Attachment 2 27
(1) To what extent has the ADB Energy Policy supported the creation of an enabling environment for private sector participation into the development of DMCs sector?
(2) To what extent has the ADB Energy Policy catalyzed the private sector clientele to replicate energy projects and mainstream its participation in energy sector?
(ii) To what extent the ADB’s energy program in 2009–2018 has been relevant, effective, efficient, and sustainable for meeting DMCs’ needs and requirements? (a) To what extent the ADB’s energy program has effectively and sustainably
met the requirements of DMCs to date? (1) To what extent have completed and validated sovereign,
nonsovereign, and technical assistance (TA) projects effectively and sustainably met DMC requirements to date?
(2) To what extent have other completed and ongoing (mature) sovereign, nonsovereign, and TA projects effectively and sustainably met DMC requirements to date?
(b) To what extent the ADB’s energy program has effectively improved the financial viability of utilities to reduce fiscal burden on governments, promoted regional cooperation and electricity interconnection, reduced air pollution, and mitigated climate change? (1) To what extent have completed and validated sovereign, non-
sovereign, and TA projects effectively improved operational efficiency and financial viability of energy sector enterprises, promoted regional cooperation and electricity interconnection, reduced air pollution, and mitigated climate change?
(2) To what extent have other completed projects (with or without completion reports) and ongoing (mature) projects effectively improved operational efficiency and financial viability of energy sector enterprises, promoted regional cooperation and electricity interconnection, reduced air pollution, and mitigated climate change?
(c) To what extent the completed and validated projects were relevant, effective, efficient, and sustainable in achieving the following policy outcomes?
(d) What is the outlook for other completed projects (with or without validated completion reports) and ongoing (mature) projects regarding their relevance, effectiveness, efficiency, and sustainability?
(e) What are the key findings, issues, and lessons from all studied projects regarding their relevance, effectiveness, efficiency, and sustainability?
2. To what extent the ADB Energy Policy continues to be aligned with current and likely future DMCs’ needs and requirements, and responds to the evolving global energy context and Strategy 2030?
(i) To what extent shall the ADB Energy Policy remain relevant to the overall economic, social, environmental, and energy sector contexts of DMCs?
(ii) To what extent shall the ADB Energy Policy continue to inform the CPSs and effectively guide the ADB’s energy program to address the DMCs’ upcoming challenges?
28 Attachment 2
(iii) To what extent shall the ADB Energy Policy remain consistent with ADB’s corporate strategies, policies, and operational plans? (a) To what extent does the ADB Energy Policy seek to focus ADB energy
program to the seven operational priorities of Strategy 2030? (b) To what extent does the ADB Energy Policy seek to align the energy
program with ADB’s thematic operational plans for (among others) climate change and disaster risk management, environment services, governance, and gender and development?
(iv) To what extent shall the ADB Energy Policy continue to facilitate and support the global energy sector transformation, in particular, the transition toward low-carbon technologies in the context of changing technologies and systems?
(v) To what extent shall the ADB Energy Policy continue to support greater private sector participation in the energy sector?
3. To what extent has ADB’s institutional structure and its human resources supported the implementation of the ADB Energy Policy and the subsequent program?
(i) To what extent has ADB institutional structure contributed to improving the efficiency in delivering the energy program?
(ii) To what extent have sector divisions, regional departments, private sector department, knowledge departments, and other ADB units collaborated to deliver the energy program and engaged in policy dialogue? (a) How has the One ADB approach been applied in the delivery of the energy
program and in engaging in policy dialogue? (iii) To what extent were staff skills complemented by external consultants and
knowledge partnerships (with global and regional centers of excellence in energy) to design and deliver the energy program plus engage in policy dialogue? (a) To what extent have knowledge partnerships and collaboration with
centers of excellence promoted an effective energy program? (b) To what extent has ADB relied on external consultants to deliver the energy
program? (iv) Regarding knowledge and innovation, what was the role of ADB and ADB Institute
in supporting research, innovation, technology transfer, knowledge events, and demonstrative projects? (a) What is the relevance of ADB and ADB Institute as a knowledge center in
energy sector policy? (b) To what extent has ADB supported energy technology transfer to DMCs? (c) To what extent has ADB contributed to the promotion of innovative energy
technologies and demonstrative projects in DMCs? (d) What is the reach of knowledge events hosted or promoted by ADB about
the energy sector? (v) What was the quality at entry of projects and whether risks assessments and
design and monitoring frameworks were appropriately defined at project appraisal and used during project implementation? (a) To what extent did project design at appraisal affect the implementation of
projects? (b) To what extent were risk assessments through and realistic, and addressed
the main issues and problems that later affected the projects? (c) To what extent were design and monitoring frameworks appropriately
designed and structured?
