Post on 24-Oct-2020
APPROACH PAPER
EBRD Energy Sector Strategy (2014-2018) - EvD Interim Review
September 2017
EBRD EVALUATION DEPARTMENT
EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 1
The Evaluation department (EvD) at the EBRD reports directly to the Board of Directors, and is independent from the Bank’s Management. This independence ensures that EvD can perform two critical functions, reinforcing institutional accountability for the achievement of results; and, providing objective analysis and relevant findings to inform operational choices and to improve performance over time. EvD evaluates the performance of the Bank’s completed projects and programmes relative to objectives.
EvD’s Special Studies review and evaluate Bank activities at a thematic or sectorial level. They seek to provide an objective assessment of performance, often over time and across multiple operations, and to extract insights from experience that can contribute to improved operational outcomes and institutional performance.
This approach paper has been prepared by EvD and is circulated under the authority of the Chief Evaluator. The views expressed herein do not necessarily reflect those of EBRD Management or its Board of Directors. Responsible members of the relevant Operations teams were invited to comment on it prior to internal publication. Any comments received will have been considered and incorporated at the discretion of EvD. Whilst EvD considers Management’s views in preparing its evaluations, it makes the final decisions about the content of its approach papers.
Nothing in this document shall be construed as a waiver, renunciation or modification by the EBRD of any immunities, privileges and exemptions of the EBRD accorded under the Agreement Establishing the European Bank for Reconstruction for Development, international convention or any applicable law.
Under the supervision of the Chief Evaluator, Joe Eichenberger, this Approach Paper was prepared by Chiara Bocci, Senior Evaluation Manager, supported by consultancy services provided by Climate & Energy Solutions. Inputs have been provided by Evaluation Department staff: Barry Kolodkin, Regina Husakova, Saeed Ibrahim, Alejandra Palma and Stephanie Crossley.
The Approach Paper was prepared in coordination and collaboration with EBRD Management focal points in the EBRD Energy Business Group.
© European Bank for Reconstruction and Development, 2014
One Exchange Square
London EC2A 2JN
United Kingdom
Web site: www.ebrd.com
All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, including photocopying and recording, without the written permission of the copyright holder. Such written permission must also be obtained before any part of this publication is stored in a retrieval system of any nature.
EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 1
Contents
1. Introduction ...................................................................................................................... 4
1.1 Objectives 4
1.2 Rationale for inclusion in the work programme 4
1.3 Scope 4
2. EBRD Energy Sector Strategy (2014-2018).................................................................... 5
2.1 Key elements 5
2.2 Theory of change 7
2.3 Portfolio analysis 9
3. Approach and process .................................................................................................. 12
4. Evaluation questions ..................................................................................................... 14
5. Data collection tools and sources ................................................................................ 15
6. Challenges and limitations ........................................................................................... 17
7. Administrative arrangements ....................................................................................... 18
7.1 EvD team 18
7.2 Peer review 18
7.3 Management review 18
7.4 Timetable 19
7.5 Budget 19
Annex 1 Evaluation matrix ................................................................................................ 20
Annex 2 Portfolio analysis ................................................................................................ 24
Natural Resources 29
Power & Energy 33
Technical cooperation (TC) 39
Annex 3 Documents consulted ........................................................................................ 43
Annex 4 People met .......................................................................................................... 45
EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 2
List of figures
Figure 1: EBRD’s Energy Sector Strategy (2014-2018) Theory of change – simplified version ................... 8
Figure 2: EBG NCBI per country (Dec 2013 – Apr 2017) ............................................................................. 9
Figure 3: Review approach ......................................................................................................................... 12
Figure 4: EBG Disbursements (Dec 2013 – Apr 2017) in € million ............................................................. 26
Figure 5: Net Cumulative Bank Investment EBG (Dec 2013 – Apr 2017) .................................................. 27
Figure 6: EBG NCBI per country (Dec 2013 – Apr 2017) in million € ......................................................... 28
Figure 7: Natural Resources – industry distribution by number of operations (Dec 2013 – Apr 2017)
.................................................................................................................................................................... 30
Figure 8: Natural Resources – industry distribution by volume (Dec 2013 – Apr 2017) .................... 30
Figure 9: Natural Resources - Net Cumulative Bank Investment and number of projects (Dec 2013 – Apr
2017) ........................................................................................................................................................... 31
Figure 10: Power & Energy – industry distribution by number of operations (Dec 2013 – Apr 2017)
.................................................................................................................................................................... 35
Figure 11: Power & Energy – industry distribution by volume (Dec 2013 – Apr 2017) ...................... 35
Figure 12: Power & Energy - Net Cumulative Bank Investment and number of projects (Dec 2013 – Apr
2017) ........................................................................................................................................................... 37
Figure 13: Donors for energy related technical cooperation activities marked as non-transactional (2013-
2017) ........................................................................................................................................................... 42
List of tables
Table 1: EBG Financial Frameworks approved by the EBRD Board (Dec 2013-Apr 2017) in chronological
order............................................................................................................................................................ 24
Table 2: EBG NCBI and operations (Dec 2013-Apr 2017) ......................................................................... 25
Table 3: EBG portfolio class (Dec 2013-Apr 2017) ..................................................................................... 25
Table 4: EBG disbursements (Dec 2013-Apr 2017) ................................................................................... 26
Table 5: Natural Resources – country distribution per industry (Dec 2013-Apr 2017) ................................ 29
Table 6: Natural Resources – geographic distribution (Dec 2013-Apr 2017) ............................................. 30
Table 7: Natural Resources – Transition Impact at approval (Dec 2013-Apr 2017) ................................... 31
Table 8: Natural Resources – SEI /GET and Environmental Category (Dec 2013-Apr 2017) .................... 32
Table 9: Power & Energy – country distribution per industry (Dec 2013-Apr 2017) ................................... 33
Table 10: Power & Energy – geographic distribution (Dec 2013-Apr 2017) ............................................... 36
Table 11: Power & Energy – Transition Impact at approval (Dec 2013-Apr 2017) ..................................... 38
Table 12: Power & Energy – SEI and Environmental Category (Dec 2013-Apr 2017) ............................... 38
Table 13: Natural Resources operations – transactional TCs per country ................................................. 39
Table 14: Natural Resources operations – donor distribution of transactional TCs .................................... 39
Table 15: Power & Energy operations – transactional TCs per country ..................................................... 39
Table 16: Power & Energy operations – donor distribution of transactional TCs........................................ 40
Table 17: Energy related technical cooperation activities marked as non-transactional (2013-2017) ........ 41
EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 3
Abbreviations
DFI Development Finance Institution
EBRD European Bank for Reconstruction and Development
EBG Energy Business Group
E2C2 Energy Efficiency and Climate Change (team)
EPG Economics, Policy and Governance
EQ Evaluation Question
ERCCA Energy, Russia, Caucasus & Central Asia
ESD Environment and Sustainability Department
ESS Energy Sector Strategy
EvD Evaluation Department
GET Green Economy Transition
LTT Legal Transition Team
NCBI Net Cumulative Bank Investment
NR Natural Resources
OECD DAC Organisation for Economic Co-operation and Development - Development Co-operation Directorate
OPI Operation Performance Indicator
RO Resident Office
SEI Sustainable Energy Initiative
SEMED South Eastern MEDiterranean
SRI Sustainable Resources Initiative
TC Technical Cooperation
ToC Theory of Change
EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 4
1. Introduction
1.1 Objectives
This Approach Paper sets out the focus, scope and methodological approach proposed by the EBRD’s
Evaluation Department (EvD) for the Interim Review of the EBRD Energy Sector Strategy (ESS)1.
Consistent with the Operations Manual Management is invited to comment and provide guidance.
The general objectives of the Review are:
To provide the EBRD Board and Management with an independent, evidence-based
assessment of the Bank’s current Energy Sector Strategy;
To identify findings and where appropriate make recommendations useful to Management and
the Board in drafting the next Energy Sector Strategy due by the end of 2018.
1.2 Rationale for inclusion in the work programme
One of EvD’s core strategic objectives indicated in the Evaluation Department Work Programme for 2017-
19 is to embark in sector strategy evaluations timed to feed analysis and evidence into
Board/Management reviews or updates. “The current Energy Sector Strategy dates from late-2013 and is
scheduled to be updated and approved by the EBRD Board by late-2018.” EvD’s review “will provide an
evidence-based strategic and operational assessment directly relevant to multiple initiatives (e.g., Green
Economy Transition) and country strategies, and to Board and Management review. The evaluation will
among other things encompass the operationalisation of strategic priorities, assess evidence of results
relative to objectives, and review issues related to execution and delivery of the strategy.”2
1.3 Scope
The ESS covers the time-frame 2014-2018 and sets EBRD’s general direction and identifies how it will
prioritise its activities in the sector. The ESS covers all the Bank’s activities in electricity generation,
transmission, distribution and supply and hydrocarbon extraction, processing, transportation, distribution
and supply. Hydrocarbons for this purpose include oil, gas and thermal coal.3 Thus this Review focuses
primarily on energy supply-side activities. End-use activities have been and/or are covered by the
Sustainable Energy Initiative (SEI), the Sustainable Resources Initiative (SRI), the Green Economy
Transition (GET), and other EBRD’s Strategies and Policies.4 Thus energy efficiency activities are only
covered if they relate to energy sector facilities, such as power plants, networks or refineries.
This Interim Review follows the standard OECD DAC evaluation principles and criteria. The evaluation will
focus on the following three questions:
i. Did the ESS set appropriate priorities for the EBRD?
ii. Did the EBRD adequately and consistently operationalise the ESS?
iii. What results have emerged from the ESS implementation so far?
1 As approved by the EBRD Board of Directors on 10 December 2013 2 As approved by the EBRD Board of Directors on 13 December 2016 3 page 6 4 Among others: the Mining Operations Policy; the Transport Strategy; the Municipal and Environmental Infrastructure Strategy.
http://www.ebrd.com/cs/Satellite?c=Content&cid=1395238499084&d=&pagename=EBRD%2FContent%2FDownloadDocumenthttp://www.ebrd.com/cs/Satellite?c=Content&cid=1395254811504&d=&pagename=EBRD%2FContent%2FDownloadDocumenthttp://www.ebrd.com/cs/Satellite?c=Content&cid=1395254811504&d=&pagename=EBRD%2FContent%2FDownloadDocument
EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 5
In order to provide meaningful answer to those questions, the evaluation team has identified a number of
criteria and indicators – specified in Annex 1.
For the purpose of this review, EvD will cover the period running from approval of the ESS in December
2013 until April 2017.
In terms of activities implemented, EvD will consider projects (investments and technical assistance) and
other activities (such as policy dialogue) as implemented by the Energy Business Group (EBG) and its
three teams with support of other departments in EBRD HQ and resident offices (including among others:
Energy Efficiency and Climate Change – E2C2; Legal Transition Team – LTT; Environment and
Sustainability Department – ESD; Economics, Policy and Governance – EPG; etc.).
2. EBRD Energy Sector Strategy (2014-2018)
This section provides an overview of the key aspects of the EBRD Energy Sector Strategy. This section
also provides key figures of ESS implementation through a preliminary portfolio analysis.
2.1 Key elements
The 2013 EBRD Energy Sector Strategy replaced the EBRD Energy Operations Policy dated 2006. This
was in response to major developments occurring in the sector, and the new strategic directions set by the
Capital Resource Review 4 (2010-2015) which called for a “shift towards an energy efficient low carbon
economy supporting energy security and economic competitiveness”5 as well as the need to align energy
sector operations with the 2008 EBRD’s Environmental and Social Policy.6 The ESS was also drafted at a
time when the EBRD was starting reformulating its approach to results and shifting from a sector focus to
a country focus.7
EvD extracted the key elements of the ESS as presented in the document itself and approved by the
EBRD Board.
5 Capital Resource Review 4: 2011-2015 as approved by the Board of Directors at its Meeting on 23/24 March 2010, p. 38 6 as approved by the Board of Directors at its meeting on 12 May 2008 7 In July 2013 a document about Nomenclature was circulated to clarify the terminology used in Bank documentation including the definition of the terms strategy, policy, initiative and framework. Eventually in September 2014 the Board Financial and Operations Policies Committee discussed a document on The Architecture of Transition Impact Results Frameworks in the Bank.
EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 6
Box: Key Elements of EBRD Energy Sector Strategy8
The ESS is built around energy efficiency and rooted in EBRD’s transition mandate and its
direction is set by the persistent transition challenges in its countries of operations.
The ESS emphasises the need to take a “holistic approach to energy systems”, and
“complements the Bank’s SEI which defines the Bank’s approach to the wider energy
systems” including energy use, albeit strictly from a climate perspective.
The ESS defines the Bank's role in the energy sector as “promoting the transition to the
policies, assets, institutions, actors and regulations that comprise a market oriented energy
sector, which will in turn deliver sustainable, secure and affordable energy services.”
The ESS “sets EBRD’s general direction and identifies how it will prioritise its activities in
the sector.” (…) “The detailed focus for each country and region is set, within the parameters
of the ESS, in the Country Strategy prepared for each country of operations.”
The ESS’s operational approach is organised around seven themes, rather than sub-sectors
(as in the previous Energy Operations Policy), namely:
o Energy Efficiency and Demand Side Measures;
o Building deep and liquid energy markets;
o Rethinking energy systems;
o Low carbon transition;
o Cleaner energy production and supply; fossil-fuelled generation;
o Setting standards and best practice;
o The wider role of the energy sector.
The ESS states that “the Bank will not finance any greenfield coal-fired power plant
except in rare circumstances, where there are no economically feasible alternative energy
sources.” In September 2014 the Bank circulated its methodology for the assessment of coal
fired generation projects.
The ESS specifies integrated approach as instrument enabling the Bank in some cases to
“draw together additional resources, technical assistance and policy dialogue coordinated with
a series of targeted investments to address specific transition gaps.”
As a monitoring framework, the ESS defines five Operation Performance Indicators (OPIs):
o OPI1: Private participation;
o OPI2: Cost reflective pricing;
o OPI3: Energy efficiency;
o OPI4: Carbon intensity;
o OPI5: Interconnections/energy trade.
8 All elements indicated in the box are quotes from the ESS itself.
EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 7
2.2 Theory of change
Based on the key elements of the ESS as approved by the EBRD Board of Directors, EvD has elaborated
a theory of change (ToC) that was not included in the ESS documentation, but implicit in the paper. The
ToC reflects purely the view of the chain of results as indicated in the ESS itself – and it does not reflect
any EvD’s interpretation of it. In essence, it depicts the causal pathway that, according to the information
provided in the ESS, leads to achieving the expected short, medium and long term results. The ToC
elaborated by EvD has been shared with and validated by the Energy Business Group and will be used by
EvD as element for the analytical framework of this Review.
The ToC in its simplified version is presented in Figure 1 in the form of an expected impact diagram, which
differentiates between the levels of intended hierarchy of results (outputs – outcomes – impacts) and the
anticipated causal links between them.9 Correspondingly, the ToC has been used for the development of
the evaluation question and related judgement criteria focused at ESS effectiveness (achievement of
results) as specified in section 3.
The ToC also indicates at outcomes level where the ESS has strong links with SEI and its targets. Also, it
contains reference to the five OPIs identified in the ESS.
The ToC identifies five main outcomes for the ESS that are closely inter-related, namely:
A - Improved energy and resource efficiency / lower energy intensity on the supply and demand
sides
B - Improved environmental performance
C - Enhanced quality and security of supply
D - More open and better functioning markets
E - Improved transparency, governance, skills, and standards/practices.
9 An expanded version of the ToC has been developed by EvD. It identifies also risks and assumptions, and specifies in details all level of results.
EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 8
Figure 1: EBRD’s Energy Sector Strategy (2014-2018) Theory of change – simplified version
Physical outputs
(i) In the power
sub-sector
(ii) In the oil and
gas sub-sectors
/ extractive
industries
(iii) In energy end-
use sectors
(outside the
energy sector)
(iv) In nuclear
safety
Policy and regulatory
changes completed
ImpactsOutcomesInputs
EBRD’s financing
resources (debt, equity,
etc.)
Donor funding for EBRD
technical assistance,
capacity building and
policy dialogue
EBRD’s staff expertise
Financing projects:
• Direct financing
(medium and large
operations)
• Indirect financing
via PFIs or direct
lending to MSMEs
(SBI) and
investment funds
• Coal-fired power
plant only in rare
circumstances
OutputsActivities
EBRD’s attributes i.e.
ability to foster and
support reforms
Policy dialogue and
cooperation to promote
energy market reforms,
cost-reflective tariffs,
setting standards and
best practice, etc.
Higher standards
and best practices
adopted
Secure, affordable
and sustainable
energy
Improved energy and resource
efficiency / lower energy intensity
on the supply and demand sides
T (SEI3) (OPI3)
More and better functioning
markets
(OPI1, OPI2, OPI5)
Improved environmental
performance
T (SEI3) (OPI4)
Equity and debt co-
financing resources of
private and public
sponsors and co-financiers
(including IFIs)
Mobilisation of co-
financing (syndication,
CIF, EU, etc.)
Transactional technical
assistance
Integrated approach
Enhanced quality and security of
supply
T (SEI3) (OPI5)
Improved transparency,
governance, skills, and
standards/practices
EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 9
2.3 Portfolio analysis
The population taken into consideration for the purpose of this Interim Review is the portfolio of the
Energy Business Group (EBG), composed by three teams: Natural Resources, Power & Energy Utilities,
and Energy Russia Caucasus & Central Asia (ERCCA).
The timeframe under consideration is December 2013, when the ESS was approved, until April 2017.
For the purpose of the analysis of the population of investments, the evaluation team did consider all
operations approved by the Board and eventually signed in the timeframe under consideration. The
evaluation team did not consider investments classified as ore mining, as falling under the umbrella of the
EBRD Mining Operations Policy.
All information is specified in Annex 1 as directly derived from EBRD official databases. Key elements are
summarised below:
The Board has approved seven financial frameworks in different countries and regions
(Poland, SEMED, Turkey, Georgia, Kazakhstan, and Greece) for different purposes, mainly to
develop renewable energy projects. Of these, one was an Integrated Approach approved in
Poland about renewables. See Table 1.
A total of 85 operations have been approved by the EBRD Board of Directors and eventually
signed. One was cancelled after signing.
The total amount of Bank investment over the 84 operations is €4.5 billion, divided as €1.7
billion for the Natural Resources team (23 projects) and €2.8 billion for the Power & Energy
team (61 projects). See Table 2 and Figure 5.
This represents 8.9% in terms of number of projects, and 18.9% in terms of volume of the EBRD
investments in the same timeframe.
Only two investments have been structured as equity; the other 82 as loans (Table 2).
60 projects (71%) of the projects are classified as private, the remaining 24 (29%) as state
(Table 3).
Across the two sub-sectors of natural resources and power & energy, the countries that
received more investments are Turkey, Kazakhstan, Ukraine, Azerbaijan, Egypt, and
Jordan. However, as indicated in Figure 6, the composition of the portfolio is very different in
those countries and further specified in the description below.
Figure 2: EBG NCBI per country (Dec 2013 – Apr 2017)
Source: EvD elaboration on Tableau based on DWH data
EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 10
Natural Resources (excluding ore mining projects)
23 projects for a total amount of €1.7 billion, accounting for 37.8% of the population.
The vast majority of projects are classified as private (19), only four as state - see Table 3.
In terms of industries (by volume) – see Table 5 Figure 7 and Figure 8:
o 36% to oil and gas extraction projects, mainly in Azerbaijan, Egypt and Kazakhstan.
o 27% to support activities for oil and gas operations mainly in Ukraine and Egypt.
o 17% to pipeline transformation of natural gas, mainly in Kazakhstan.
o 9% to petroleum refineries mainly in Turkey.
o 9% to pipeline transportation in Ukraine.
In terms of geographic distribution – see Table 6 and Figure 9:
o Eastern Europe and Caucasus account for 52% because of the investments in
Azerbaijan (two projects with one client) and Ukraine (two projects).
o Central Asia accounts for 19% because of the investments in Kazakhstan (two
projects).
o SEMED accounts for 13% because of the investments in Egypt (five projects).
o Turkey, counting also as a region, accounts for around 8% (two projects).
o Investments in Greece accounts for 5% (two projects).
In terms of transition impact, the projects were rated with good or strong good, with high risk
– see Table 7
12 projects out of 23 contribute to the SEI target - cleaner energy production (Table 8).
The majority of the projects were classified B, three only A, as environmental category (Table 8).
A total of €2.2 million have been committed with transactional grants (project preparation and
implementation) mainly devoted to support operations in Greece and Ukraine (see Table 13
and Table 14).
Power & Energy
61 projects for the total amount of €2.8 billion, accounting for 62.2% of the population.
33% of projects are classified as state (20), and 77% (41) as private - see Table 3.
In terms of industries – see Table 9, Figure 10 and Figure 11:10
o 49% of volume to alternative energy across 62% of the total number of operations
(38 projects).
o 27% of volume to electric power generation across 13% of number of operations
(eight projects).
o 14% of volume to electric power distribution in a same number of eight projects,
i.e. 13% of the number of operations.
o 8% of volume to electric power transmission, control and distribution across six
projects, i.e. 10% of the number of operations.
o 2% of volume to natural gas distribution in one project, i.e. 2% of the number of
operations.
In terms of geographic distribution – see Table 10 and Figure 12:
o South-Eastern Europe accounts for 29% with investments mainly in Albania,
Bulgaria, Romania and Serbia.
10 In September 2017 the Energy Business Group re-classified the industry of nine operations from Electric Power Generation to Alternative Energy. The implications of this change will be taken into consideration, if any, in the Review itself.
EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 11
o SEMED accounts for 20% because of the investments in Egypt (one project) and
Jordan (eight projects).
o Turkey, counting also as a region, accounts for around 20% (seven projects).
o Central Asia accounts for 15% because of the investments in Kazakhstan (six
projects).
o Investments in Central Europe (6%) are concentrated in Poland (four projects) where
the Integrated Approach for renewables has been implemented.
o Eastern Europe and Caucasus accounts for 6% with investments in Georgia (five
projects) and Ukraine (four projects).
In terms of transition impact, four have been rated excellent, 17 strong good and 30 good,
with high risk (Table 11).
58 projects out of 61 contribute to the SEI targets – corporate energy efficiency, cleaner
energy production, or renewable energy (Table 12).
48 projects were classified B, 13 A as environmental category (Table 12).
A total of €22.3 million have been committed with transactional grants (project preparation
and implementation) mainly devoted to support operations in Tajikistan, Kazakhstan, Albania,
Egypt, Kosovo, Morocco, Tunisia and Kyrgyz Republic (see Table 15). The main donor is
the EBRD Shareholder Special Fund, followed by Austria, Italy, SEMED Multi-Donor, Climate
Investment Fund - CIF, Japan (see Table 16)
A total of €48 million of investment co-financing (via CIF and Global Environment Facility -
GEF) was provided to support seven operations in Georgia, Kazakhstan, Morocco, Mongolia,
and Ukraine
Policy dialogue
From the information gathered by the evaluation team, donor funded activities aimed at the broader
category of policy dialogue (including: capacity building, institution building, legal and regulatory
reforms, research) have been initiated and implemented on broad energy related topics by EBG, E2C2,
LTT, ESD, etc. Key data are provided below and will be further analysed by the evaluation team during
the ESS Review – see Table 17 and Figure 13:
A total of €17 million for 77 assignments (see Table 17)
Policy dialogue activities took place mainly in Egypt, Kazakhstan, Turkey (see Figure 13)
Activities included support activities to the Regulators, assistance to relevant Ministries on
energy related legislation and action plans, transparency in extractive industries, and broader
research activities.
Additional policy dialogue activities carried out by the Bank in the sector and not funded by donors will
complement the data outlined above and in Table 17 and Figure 13. Information about those activities will
be collected during the Review as provided by the teams involved – as not captured systematically in any
EBRD system.
EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 12
3. Approach and process
This Interim Review follows the standard OECD DAC evaluation principles, and addresses its subject
through selected aspects of the standard evaluation criteria (in particular relevance, effectiveness,
efficiency).
The review process adopts a systematic approach, through which different tools and data collection
methods are used to gradually construct answers to evaluation questions and to formulate conclusions
and recommendations. The process is structured along three main phases as outlined in Figure 3.
Figure 3: Review approach
A. Development of analytical framework
In this phase, the outcome of which is represented by this Approach Paper, the analytical framework
(evaluation matrix) for the Review is developed. The key building blocks underpinning the selection of key
issues to be included in the evaluation matrix are the understanding of the strategic objectives of the ESS,
its actual implementation, and context.
The objectives and key elements of the ESS as approved in December 2013 are described in section 2
and schematically represented via the theory of change (Figure 1).
EvD has preliminary reviewed the actual implementation of the ESS through the analysis of the portfolio
data of the EBG and the TCs as well as policy dialogue activities – as harvested during this Approach
Paper phase. The summary of the portfolio analysis is presented in section 2.3 and detailed in Annex 1.
The portfolio analysis facilitated among others the preliminary selection for countries to be visited and
areas of interest (see section 5).
The evaluation team explored the key context of the ESS, both internally within EBRD and externally in its
interaction with stakeholders, to be able to identify the main issues of interest for the Review. This was
pursued through preliminary document analysis (see overview in Annex 3), and through first exploratory
Theory of
change
Context
overview
Portfolio analysis
Evaluation matrix= Evaluation Questions,
Judgment criteria, Indicators
Conceptualisation of the evaluation framework
Structuring of evaluation questions
B. Data collection and analysis
Strategy-
level
documents
Country and
programme
level
documents
Context
documents
Stakeholders
’ views
Facts,
Findings
Analysis,
Evaluative
judgments
C. Synthesis (output = ESS
Review Report)
Answers to EQs
Conclusions
Recommendations
A. Development of analytical framework (output = Approach Paper)
Portfolio
analysis
TC MIS data
EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 13
interviews with key stakeholders (see overview in Annex 4). This allowed further development of the
evaluation questions.
Based on these inputs the evaluation matrix of evaluation questions and sub-questions (judgment criteria
and indicators) was developed (see section 4 and Annex 1). It is intended to serve as the analytical
framework to guide and structure data collection through the various tools and data collection methods
(see below). While no significant revisions of the matrix are foreseen, the evaluation matrix might be
slightly amended during the subsequent phases of the Review to reflect new inputs substantive for
achieving the objective of the evaluation.
B. Data collection and analysis
Once the analytical framework is completed, data is collected by the evaluation team via the various tools
and methods for data collection (see section 5). Data collection through all tools is structured along the
evaluation matrix, to allow for the analysis of the data across the different tools and, to the extent possible,
for verification and cross-checking (triangulation) of data from different sources. The evaluation will
combine secondary (desk) sources with fresh data collection through interviews of relevant stakeholders
in the countries of operations, EBRD HQ, and the main donors’ HQs.
C. Synthesis
The synthesis phase is then devoted to constructing answers to the evaluation questions and formulating
conclusions and recommendations on the basis of the data collected throughout the process. The final
output of the synthesis will be a comprehensive report to be circulated to the audience of the evaluation.
The timing is specified in section 7.4.
EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 14
4. Evaluation questions
The main evaluation questions and their underlying rationale are briefly outlined below and are to be read
together with challenges and limitations outlined in section 6. The full evaluation matrix, which will be used
to guide the collection and structuring of data, is presented in Annex 1.
The evaluation team has identified three evaluation questions to guide this Interim Review of the ESS
which broadly correspond to the OECD/DAC criteria of relevance, efficiency and effectiveness.
EQ1: Did the ESS set appropriate priorities for the EBRD?
In order to answer to this evaluation question, the evaluation team has identified a number of judgment
criteria associated with the theme of ‘relevance’. In particular, the evaluation team will assess whether at
the time of drafting the ESS the Bank: (i) correctly identified the challenges and needs of the sector in its
countries of operations; (ii) derived adequate priorities for its interventions from this analysis of challenges
and needs; (iii) adequate considered for the EBRD specific business model and unique value added
(“institutional additionality”) in the identification of priorities in the ESS; (iv) took proper account potential
for complementarity and cooperation with other players; (v) whether the Bank in doing all of this
incorporated the inputs from relevant stakeholders, including but not limited to host governments, industry
players, other DFIs and international organisations, and CSOs; and (vi) whether elements of internal
learning were derived by the Bank from the 2006 Energy Operations Policy and previous relevant sector
evaluations.
EQ2: Did the EBRD adequately and consistently operationalise the ESS?
The evaluation team will be focusing on whether the EBRD set up an adequate operational organisation
(broadly defined) and allocated sufficient and adequate resources in the ESS to enable its successful
implementation (eventually assessed in terms of emerging results via EQ3). Answering this evaluation
question could include an assessment of: (i) an adequate result framework for the ESS; (ii) adequate
operational approaches, including the use of grants; (iii) adequate alignment of subsequent Country
Strategies with the ESS (as the detailed focus for each country and region is indicated in the ESS to be
set in the Country Strategy prepared for each country of operations); (iv) the adoption of an internal
organisation, and allocation of resources (including human resources), and (v) identification of internal
processes enabling the Bank to achieve the results identified in the ESS.
EQ3: What results have emerged from the ESS implementation so far?
This evaluation question is directly related to actual EBRD operations approved, signed and implemented
since the ESS came into force at the end of 2013. Bearing in mind this is an Interim Review of an on-
going Strategy, the evaluation team will seek through this question to carry out a preliminary assessment
of the progress(es) that the Bank has made in reaching its intended results in the energy sector through
its investment operations, technical cooperation and policy dialogue activities. As explicitly indicated in the
Theory of Change, the ESS medium-term results could be conveniently summarised by five expected
outcomes – which are closely interlinked among them. The evaluation team will collect data to gauge
progress toward the emerging results related to the five outcomes. The evaluation team will also update
the five OPIs identified in the ESS – bearing in mind limitations in terms of attribution. Some measures
could only be partial as it will be based on a selection of countries and only elements of EBRD’s
contribution could be harvested. In light of the harvested findings, the evaluation team will seek to identify
the major factors that could influence the achievement of the expected results.
EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 15
5. Data collection tools and sources
A series of tools will be used for collecting, structuring, processing and analysing data. Some have already
been used in the preparation of this Approach Paper (such as preliminary review of documents, initial
interviews, and portfolio analysis). This section presents the tools and collection methods foreseen for the
use in all the phases of the evaluation as outlined in section 3.
EvD will mix the collection methods as fits the purpose of the specific judgement criteria identified to
respond to the three evaluation questions (see the evaluation matrix in Annex 1).
Interviews
First exploratory interviews with key stakeholders have already been conducted in the course of the
preparation of this Approach paper (see Annex 4). Further interviews, structured in line with the evaluation
matrix, will be conducted both in EBRD HQ and in the countries selected for field visits. Some interviews
may be conducted via phone or videoconferencing as necessary. The team will seek to gather the views
in particular of the following key stakeholders (not exhaustive):
EBG in HQ and ROs
Heads of ROs
Representatives of other relevant EBRD teams, including: E2C2, EPG/OCE, ESD, Vice
Presidency Policy, Donor Co-financing, Civil Society Engagement Unit, Legal Transition Team,
Internal Audit, Corporate Strategy, Budget, Operational Strategy and Planning, Operational
Effectiveness and Efficiency, etc.
Representatives of EBRD shareholders and donors
EBG clients
Relevant counterparts/stakeholders at country level (including local and national authorities,
regulatory agencies, sector associations, civil society organisations, etc.)
Representatives of DFIs, bilateral donors, etc.
Country visits
Operationally, the ESS was meant to provide the parameters based on which the EBRD would set
priorities at country level. The evaluation team intends to visit selected countries of operations to
complement the desk work. The objective is to gather information to feed into the answers to evaluation
questions at the country level, and to explore the different contextual determinants of ESS implementation
and success. The rationale is that the Bank is predominantly seeking to address transition challenges at
country level. It is important to emphasise that the country visits will not be aimed to be stand-alone
evaluations of EBRD country strategies.
The evaluation team will agree with Management on two countries to visit from a short-list of four
selected on the basis of a number of criteria, namely: Georgia, Jordan, Kazakhstan, and Turkey. Selection
criteria included:
The number of projects (and volume) signed: critical mass is essential to achieve the
outcomes pursued by the Bank through the ESS and the projects and other activities that deliver
these;
Diversification of the energy sector portfolio in the country in terms of: private and public
sectors; Power & Energy and Natural Resources projects, range of sub-sectors spanned by the
projects, etc.;
EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 16
Contribution to ESS expected outcomes, based on a preliminary assessment by the
evaluation team for the purpose of this Approach Paper of the likelihood that each signed
project will contribute to each of the five expected outcomes;
Transition Impact ratings at approval: a higher proportion of projects rated good or higher
denotes a high level of potential impact in the country and higher likelihood to achieve one or
more of the expected outcomes of the ESS;
The existence of substantial technical cooperation and policy dialogue-related
interventions;
Countries belonging to different regions characterised by different transition and generally
early and intermediate transition stages.
Logistical arrangements will be discussed and coordinated with RO teams in advance, and success of the
missions will depend to a large degree on the cooperation of the EBG colleagues in HQ and EBRD
Resident Offices staff, especially in terms of provision of contact details for local clients and other
stakeholders.
Specific themes of interest
To complement data collection and structuring along strict geographical lines, other aspects will be
investigated. Given the importance given by the ESS to themes (see section 1) the evaluation team also
consider the Bank interventions in the sector through cross-cutting or thematic lens. For instance, a
preliminary topic of interest will be the use of the so-called ‘Integrated Approach”, and the specific case
that is most advanced in implementation is the Integrated Approach to Polish Renewables.
Depending on resources implications, this Interim Review will consider possible implications on the
interpretation of the ESS in light of the on-going EBRD’s revision of the measurement of transition impact.
Other topics for thematic case studies could be added to the evaluation where they would help to answer
the evaluation questions and resources and time will allow.
Portfolio analysis
Preliminary analysis of the EBG portfolio (excluding ore mining projects) over the period December 2013-
April 2017 has already contributed to the preparation of the methodology (see Annex 1) and for the
preliminary selection of countries for field visit. During the Review, it might be further developed as
needed to contribute to further data collection. The same approach applies to the population of grants
available in the period under evaluation. Data analysis will also be used for selecting stakeholders to be
interviewed, in collaboration with the EBG and country teams.
Documentary analysis
Similarly, preliminary reading of key ESS-related and contextual documents has already been carried out
for the purpose of preparation of this Approach paper (see Annex 3). Further documents will be collected
and consulted to complement information relevant to the issues outlined in the evaluation matrix. This will
include but not be limited to the following documents:
EBRD Board-approved documents relating to activities in the energy sector, particularly but not
limited to the EBG and SEI/GET, including strategies, policies, programme documents, technical
cooperation and non-technical cooperation grants; minutes of EBRD Board of Directors meeting
and its committees discussions on energy related matters; Board information sessions; etc.
EBRD Country Strategies approved after December 2013, etc.
EBG projects-level documents, including TC documents and available reports, etc.
Strategic EBRD documents such as: CRRs, SCF, SIPs, etc.
EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 17
Contextual documents and statistics relevant to EBRD countries of operations, including EBRD
Assessment of Transition Challenges as included in the annual Transition Reports, national
development strategies, relevant documents of OECD, DFIs, IEA and other international
organisations, etc.
Previous relevant evaluation work, including:
o 2011 Special Study on Power and Energy Sector Review,11never discussed at the EBRD
Audit Committee. Management comments were provided in the framework of the 2012
Follow-up of Evaluation Recommendations by Management12
o 2014 Special Study on EBRD’s Experience with Policy Dialogue in Ukraine, Case study on
Power & Energy and Case study on Energy Efficiency13
o 2015 Evaluation of the EBRD’s Sustainable Energy Finance Facilities (SEFFs)14
o 2016 Evaluation of Four Wind Energy Projects15
o 2016: Evaluation of EBRD’s Use of Subsidies16
o Sector related validations and reviews of Operations Performance Assessments.
Documents from other DFIs and relevant energy sector strategies and policies.
6. Challenges and limitations
A number of challenges and potential limitations for the study are acknowledged, which could affect to
various degrees the outcome of the evaluation.
Ownership of the ESS
Roles and responsibilities are not defined in the ESS. EvD has made the assumption that the EBG is the
main key responsible department although the ESS affects the entire EBRD business, as the energy
sector itself touches upon many projects directly or indirectly.
EBRD strategic planning context
Since the ESS was approved in December 2013, the EBRD internal context has developed. Changes
introduced are affecting the ESS and its implementation to an unknown extent which is going to be
verified by the evaluation team during this Interim Review.
Results contribution
The evaluation team will seek to identify and assess the contribution of EBRD to observed results, mainly
through qualitative analysis. The source of information may come from the EBRD itself, and from direct
interviews carried out as part of the evaluation with relevant stakeholders. In addition, views and opinions
will be collected as a part of direct interviews in the countries visited.
Availability and quality of available information
Experience from past EvD Special Studies shows that the necessary documentation may not always be
fully available to the evaluation team. In addition, Special Studies of large programmes rely to some extent
on available monitoring and evaluation data and their quality and comprehensiveness. Lastly, the
availability of EBRD current clients for interviews and their willingness to share information with the
evaluation team may affect the data collection, as may the availability of other counterparts and
stakeholders.
11 Circulated in November 2011 12 As approved by the EBRD Board of Directors on 27 November 2012 13 Discussed at the Audit Committee on 6 May 2014 14 Discussed at the Audit Committee on 25 January 2016 15 Discussed at the Audit Committee at its meeting on 25 February 2016 16 Discussed at the Audit Committee on 7 November 2016
EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 18
Staff turnover
Related to the previous issue is the challenge of staff turnover and institutional memory. It is a common
occurrence that key staff changes both internally in EBRD, and externally within local stakeholders and
counterparts. This may hinder the collection of data of older date, as institutional memory may not be
preserved.
7. Administrative arrangements
7.1 EvD team
The evaluation team is composed of Ms Chiara Bocci, EvD Senior Evaluation Manager. As appropriate,
specific contributions may be made by other EvD colleagues, in particular for specific data collection in
countries of operations if efficient. The evaluation team will benefit from the services provided by a
consultancy team – to be contracted based on the needs of this Review on specific aspects.
7.2 Peer review
The evaluation team will select one external peer reviewer. The expert will provide an independent review
of the final draft of the study with respect to its quality, in particular soundness of the analysis, appropriate
evidence base and presentation of findings, links between findings, conclusions and recommendations,
etc.
7.3 Management review
The draft Approach Paper has been circulated to the designated counterparts in Management as per the
Operations Manual 8.5, namely to the two Focal Points (Director Strategy & Policy Coordination in VP
Policy, and Managing Director Portfolio Business Group) who co-ordinated with the relevant
departments/teams for circulation of the draft Approach Paper and shared consolidated comments with
EvD. Management Focal Points and Reviewers for the study include:
Focal Points:
Managing Director, Banking Portfolio
Director, Country Strategy Coordination and Results Management
Management Reviewers:
Managing Director, Energy & Natural Resources
Director, Head of Energy, Russia, Caucasus & Central Asia
Director, Head of Natural Resources
Director, Head of Power&Energy Utilities
Director, Strategy, Energy & Natural Resources
Managing Director, OSP, Energy Efficiency & Climate Change
Director, Head of Energy Efficiency Climate Change
Managing Director, Environment & Sustainability
Managing Director, Economics Policy & Governance
Director, Sector Economics & Policy
Director, Legal Transition
This list may be added to during the course of the evaluation.
EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 19
7.4 Timetable
Milestone Date
Study starts April 2017
Draft Approach paper circulated to Management July 2017
Approach paper approved August 2017
Field visits September/October 2017
Draft circulated to internal peer reviewers November 2017
Draft cleared by Chief Evaluator for circulation to external peer
reviewers
December 2017
Draft cleared by Chief Evaluator for Management Comments December 2017 / January 2018
Final approved by Chief Evaluator January 2018
Final distribution to Board February 2018
7.5 Budget
This study will require resources for EvD staff, consultancy team, travel costs and external consultant for
peer review, all within EvD approved budget for 2017-18.
EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 20
Annex 1 Evaluation matrix
Evaluation questions Judgement criteria Indicators Sources of Data
1. Did the ESS set appropriate priorities for the EBRD?
1.1 Clarity of ESS’s identification of sector challenges (and broader context)
Existence and use of methodology to assess sector challenges aimed at drafting ESS
Peer-review process
ESS Preparatory analysis to ESS OCE/EPG documents External literature
1.2 Clarity of priorities/themes of intervention in the ESS derived from sector challenges and aligned with EBRD strategic planning documents
Coherence between sector/needs analysis and priorities
Prioritisation based on CRR4, SCF, SEI 3 and SRI and other strategic planning documents
Identification of specific methodologies to address specific challenges (e.g. related to coal)
ESS EBRD CRR4, SCF SEI 3, SRI Other strategic planning documents
1.3 Adequate consideration for EBRD specific business model and unique value added (“institutional additionality”) in the identification of priorities in the ESS
EBRD institutional attributes reviewed
ESS includes an ex-ante assessment on how EBRD’s interventions are likely to be additional
Risks
ESS Preparatory analysis to ESS
1.4 The ESS adequately identifies the potential for and implications of complementarity with other key players
Existence of mapping of other IFI/donors on-going or planned activities in the sector and indication of possible complementarity actions and activities
ESS Preparatory analysis to ESS
1.5 ESS incorporates adequately inputs of all relevant stakeholders
Existence of processes (such as specific meetings/hearings) for expression and/or contribution by stakeholders of their views
Actual incorporation in the ESS of points raised/inputs made by stakeholders during the consultation or otherwise
ESS and Report on the Invitation to Public to Comment Minutes and other documents from consultation meetings
1.6 ESS incorporates elements of internal learning
Findings and lessons from the previous Energy Operations Policy
Findings and lessons from previous relevant sector evaluations
ESS Preparatory analysis to ESS
2. Did the EBRD adequately and consistently operationalise the ESS?
2.1 Results of the ESS are captured in an adequate results framework to ensure inter alia the ESS is on track
Existence of explicit or implicit architecture of results linked to priorities
Existence of adequate OPIs and targets
ESS and underlying documents
2.2 The ESS identifies resources to implement the adequate operational approaches (e.g. integrated approach) vis a vis priorities
References to operational approaches per priority/theme of intervention
Identified integrated approach
ESS and underlying documents
EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 21
Evaluation questions Judgement criteria Indicators Sources of Data
Use of technical assistance, subsidies and concessional resources
Identification of resources in the ESS
2.3 EBRD Country Strategies objectives and targets are aligned with the ESS
EBRD Country Strategies incorporate sections about the energy sector with results and indicators aligned with ESS priorities and OPIs
28 Country Strategies approved after December 2013
2.4 Adequate resources and capacities (human, financial) have been allocated to ensure implementation of the ESS
Δ Bank’s headroom
Δ headcount in Energy Business Group and other departments implementing the ESS
Δ consultancy budget
Δ technical cooperation budget
Existence of action plans
Energy Group Scorecard Other internal EBRD documents Other IFI energy sector strategies/policies
2.5 An adequate internal organisation (team roles and responsibilities, processes, etc.) has been put in place in order to best implement the ESS
Management tools for cross-team collaboration processes and procedures
Processes and procedures between HQ and ROs
Accountability mechanisms towards the Board
Structured mechanism of learning and knowledge management
Internal EBRD available documents Other IFI energy sector strategies/policies
3. What results have emerged from the ESS implementation so far?
3.1 Progresses towards improved energy and resource efficiency / lower energy intensity on the supply and demand sides
Contribution of projects to ESS objectives and SEI as reflected by:
Δ in absolute energy consumption per capita (OPI 3)
Energy savings (Toe)
Addition of RE generation capacity
Δ of share of renewable energy in the energy mix
Δ of energy intensity (Toe/GDP)(OPI 3)
Δ of CO2 emissions per capita (OPI 4)
Δ of carbon intensity (MtCO2/ kwh)(OPI 4)
Progresses made by policy dialogue activities (donor funded, EBRD funded, and part of daily EBRD activities in HQ and countries of operations)
Sources utilised by the Bank for the initial calculation and compilation of the OPIs
External literature
International databases (World Bank, IRENA, IEA)
Project documentation (investments, TCs and policy dialogue)
Interviews and documents from EBRD colleagues
3.2 Progresses towards improved environmental performance
Contribution of projects to ESS objectives and SEI as reflected by:
OPI 4 indicators above
GHG emissions reduction (Mt CO2-equ.)
Δ of operators adopting best practices and standards for nuclear electricity production and waste disposal (e.g. IAEA safety standards) (see 3.5)
Project documentation provided by EBG and E2C2 (investments, TCs and policy dialogue)
EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 22
Evaluation questions Judgement criteria Indicators Sources of Data
Progresses made by policy dialogue activities
3.3 Progresses towards enhanced quality and security of supply
Contribution of projects to ESS objectives and SEI as reflected by:
Indicator of quality of electric power grid: o Availability (continuity of supply), o Technical properties (voltage quality) o Commercial quality (timeliness of services to customers)
Indicator of quality of gas power grid: o Supply (technical operational quality) o Composition (natural gas quality) o Commercial quality (timeliness of services to customers) o Loss reductions, collections rates improvements
Addition of RE generation capacity (see 3.1)
Δ of share of renewable energy in the energy mix (see 3.1)
Δ of renewable energy consumption (% of total final energy consumption)
Δ of cross-border energy trade: o Proportion of energy exports over total energy production (OPI 5) o Proportion of energy imports over total energy consumption (OPI
5) o Aggregate interconnection capacity (MW) (OPI 5, see 3.4)
Progresses made by policy dialogue activities
International databases: CEER, IRENA, World Bank
Project documentation (investments, TCs and policy dialogue)
Monitoring reports
3.4 Progresses towards more open and better functioning markets
Investment in energy with private participation (current US$)
% of domestic electricity generation under private sector control (OPI 1)
% of domestic gas production under private sector control (OPI 1)
% of domestic oil production under private sector control (OPI 1)
% of refining capacity under private sector control (OPI 1)
% of the national electricity network under private sector control (OPI 1)
% of the national gas transmission network under private sector control (OPI 1)
% of electricity consumers switching suppliers
Change in number of countries with operating exchanges
% of total electricity consumption served by private companies (OPI 1)
% of total gas consumption served by private companies (OPI 1)
International databases: World Bank
Project documentation
Policy dialogue documentation
EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 23
Evaluation questions Judgement criteria Indicators Sources of Data
Total gas pipeline interconnection capacity (in bcm/y) (OPI 5)
Number of electricity interconnectors (OPI 5)
Total oil pipeline interconnection capacity (Mtoe)
Δ of proportion of energy prices, weighted by consumption, that is either liberalised or, if regulated, at levels that do not imply any pre-tax subsidies (OPI 2)
Δ Number of countries with unbundled power and gas industries
Δ Number of countries with (deemed) independent power and gas regulators
Δ Number of countries providing transparent and non-discriminatory third party access to gas and power grids
Progresses made by policy dialogue activities
3.5 Progresses towards improved transparency, governance, skills, and standards/practices
Contribution of projects to ESS objectives as reflected by:
Δ Number of operators (esp. in the oil and gas sector) having adopted transparency practices (e.g. EITI principles) in a country
Δ Number of companies having adopted CSR policies (re. social, environmental and governance practices) in a country
Δ Safer NPPs/ total number of NPPs in a country
Progresses made by policy dialogue activities
International databases
Project documentation (investments, TCs and policy dialogue)
EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 24
Annex 2 Portfolio analysis
The population taken into consideration for the purpose of this Review is the portfolio of the Energy
Business Group (EBG), composed by three teams: Natural Resources, Power & Energy Utilities, and
Energy Russia Caucasus & Central Asia (ERCCA).
The timeframe under consideration is December 2013, when the ESS was approved, until April 2017.
For the purpose of the analysis of the population the evaluation team did consider all operations approved
by the Board and eventually signed in the timeframe under consideration. The evaluation team did not
consider investments classified as ore mining, as falling under the umbrella of the EBRD Mining
Operations Policy.
From December 2013 until April 2017, the EBRD Board of Directors has approved seven financial
frameworks, including one Integrated Approach, as listed below.
Table 1: EBG Financial Frameworks approved by the EBRD Board (Dec 2013-Apr 2017) in chronological order
Framework name
Board approval
Banking team
Short Description Country (Sub-) operations* approved
Integrated Approach to Polish Renewables
15/10/2014 Power & Energy
To address key transition challenges of the Polish renewable energy sector through policy dialogue and targeted outcome-
driven engagements and associated investments
into private sector renewable energy projects
and electricity grid investments
Poland 4
Genesis Framework
28/01/2015 Power & Energy
Bond Framework covering four vertically integrated
Polish energy groups
Poland 2 (cancelled)
SEMed Private Renewable Energy Framework
14/10/2015 Power & Energy
To provide finance to private developers of
renewable energy generation projects in
SEMed
Regional 1
PLUTO - Early stage geothermal support framework
09/12/2015 Power & Energy
Financing for early stage geothermal development
Turkey 2
Georgian Low Carbon Framework
16/12/2015 Power & Energy
Support the Government strategy to foster low
carbon generation, cover the country’s seasonal winter demand and to
support private ownership in the energy sector
Georgia 2
Kazakhstan 14/12/2016 Power & Finance the modernisation Kazakhsta 3
EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 25
Framework name
Board approval
Banking team
Short Description Country (Sub-) operations* approved
Renewables Framework
Energy and strengthening of the grid in order to enhance
the integration of RES into the grid system
n
Greek Renewable Energy Framework
08/03/2017 Power & Energy
Policy dialogue and financing for renewable
energy projects
Greece 1
Source: EvD elaboration on DTM data
*Operations approved as part of Integrated Approach financial frameworks are classified as stand-alone
For the purpose of this analysis, the evaluation team has considered all approved investments and
analysed data available for the signed ones. 85 operations have been signed in the period under
consideration, and one signed was cancelled. Therefore the total population under consideration is
composed by 84 operations.
Table 2: EBG NCBI and operations (Dec 2013-Apr 2017)
Sector Team Net Cumulative Bank Investment (€ million)
% Number of operations
Average size (€ million)
Natural Resources 1,721 37.8% 23 75
Debt 1,721 37.8% 23 75
Power & Energy 2,831 62.2% 61 46
Debt 2,650 58.2% 59 45
Equity 180 4.0% 2 90
TOTAL 4,551 100% 84 54
Source: EvD elaboration on DWH data
In the same timeframe the EBRD approved 949 investments for a total amount of €24,128,913,558, thus
making count the EBG (exclusive of the ore mining projects) its 8.9% of it in terms of number of projects,
and 18.9% in terms of volume.
Table 3: EBG portfolio class (Dec 2013-Apr 2017)
Class Natural Resources Power & Energy TOTAL
Private 19 41 60
State 4 20 24
TOTAL 23 61 84
Source: EvD elaboration on DWH data
EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 26
Table 4: EBG disbursements (Dec 2013-Apr 2017)
Class Disbursements
(€ million)
To be disbursed
(€ million)
NCBI
(€ million)
Signed 0 815 815
Disbursing 1,267 859 2,126
Repaying 1,494 104 1,598
Completed 12 0 12
TOTAL 2,774 1,778 4,551
Source: EvD elaboration on DWH data
Figure 4: EBG Disbursements (Dec 2013 – Apr 2017) in € million
Source: EvD elaboration based on DWH data
EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 27
Figure 5: Net Cumulative Bank Investment EBG (Dec 2013 – Apr 2017)
Source: EvD elaboration based on DWH data
EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 28
Figure 6: EBG NCBI per country (Dec 2013 – Apr 2017) in million €
Source: EvD elaboration based on DWH data
EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 29
Natural Resources
The description of the 23 projects signed by the Natural Resources sector team is available in the tables
and graphs below.
Table 5: Natural Resources – country distribution per industry (Dec 2013-Apr 2017)
Industry Op # NCBI (€) % total volume
Average size (€)
Oil and Gas Extraction 8 613,581,326 35.7% 76,697,666
Azerbaijan 2 411,390,959 23.9%
Egypt 3 108,685,774 6.3%
Greece 1 18,284,043 1.1%
Kazakhstan 1 64,908,351 3.8%
Romania 1 10,312,200 0.6%
Support Activities for Oil & Gas Operations
5 464,139,050 27.0% 92,827,810
Bulgaria 1 10,000,000 0.6%
Egypt 2 111,313,251 6.5%
Greece 1 68,565,160 4.0%
Ukraine 1 274,260,639 15.9%
Pipeline Transportation of Natural Gas (inc storage)
2 297,340,441 17.3% 148,670,220
Kazakhstan 1 256,340,441 14.9%
Moldova 1 41,000,000 2.4%
Petroleum Refineries 2 157,370,442 9.1% 78,685,221
Estonia 1 20,240,122 1.2%
Turkey 1 137,130,320 8.0%
Pipeline Transportation (Ukraine) 1 150,000,000 8.7% 150,000,000
Gasoline Stations 4 33,746,152 2.0% 8,436,538
Georgia 2 25,140,559 1.5%
Kyrgyz Republic 1 1,325,593 0.1%
Turkey 1 7,280,000 0.4%
Coal Mining (Mongolia) 1 4,571,011 0.3% 4,571,011
TOTAL 23 1,720,748,421 100% 74,815,149
Source: EvD elaboration based on DWH data
EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 30
Figure 7: Natural Resources – industry distribution
by number of operations (Dec 2013 – Apr 2017)
Figure 8: Natural Resources – industry distribution by
volume (Dec 2013 – Apr 2017)
Source: EvD elaboration based on DWH data
Table 6: Natural Resources – geographic distribution (Dec 2013-Apr 2017)
Region & Country Op # NCBI (€) % total volume
Eastern Europe and Caucasus 7 901,792,156 52.4%
Azerbaijan 2 411,390,959 23.9%
Georgia 2 25,140,559 1.5%
Moldova 1 41,000,000 2.4%
Ukraine 2 424,260,639 24.7%
Central Asia 4 327,145,396 19.0%
Kazakhstan 2 321,248,792 18.7%
Kyrgyz Republic 1 1,325,593 0.1%
Mongolia 1 4,571,011 0.3%
Southern and Eastern Mediterranean 5 219,999,025 12.8%
Egypt 5 219,999,025 12.8%
Turkey 2 144,410,320 8.4%
Turkey 2 144,410,320 8.4%
Cyprus and Greece 2 86,849,202 5.0%
Greece 2 86,849,202 5.0%
South-Eastern Europe 2 20,312,200 1.2%
Bulgaria 1 10,000,000 0.6%
Romania 1 10,312,200 0.6%
Central Europe and Baltics 1 20,240,122 1.2%
Estonia 1 20,240,122 1.2%
TOTAL 23 1,720,748,421 100.0%
Source: EvD elaboration based on DWH data
EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 31
Figure 9: Natural Resources - Net Cumulative Bank Investment and number of projects (Dec 2013 – Apr 2017)
Source: EvD elaboration based on DWH data
Table 7: Natural Resources – Transition Impact at approval (Dec 2013-Apr 2017)
TI TI Risk
High Medium TOTAL
Excellent - - -
Strong Good 5 - 5
Good 10 1 11
Moderate Good - 1 1
Satisfactory - - -
Marginal - - -
SUB-TOTAL 15 2 17
N/A* - - 6
TOTAL 23
Source: EvD elaboration based on DWH data
* 6 projects does not have TI rating because either are sub-operations in a Framework or because they are an extension of an existing operation
EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 32
Table 8: Natural Resources – SEI /GET and Environmental Category (Dec 2013-Apr 2017)
SEI Category / Env Category A B C None TOTAL
3 – Cleaner Energy Production 2 10 - - 12
No SEI/GET 1 7 1 2 11
TOTAL 3 17 1 2 23
Source: EvD elaboration based on DWH data
EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 33
Power & Energy
The description of the 61 projects signed by the Power and Energy sector team is available in the tables
and graphs below.
In September 2017 the Energy Business Group re-classified the industry of nine operations from Electric
Power Generation to Alternative Energy. The implications of this change will be taken into consideration, if
any, in the Review itself.
Table 9: Power & Energy – country distribution per industry (Dec 2013-Apr 2017)
Industry Op #
NCBI (€) % of total volume
Average size (€)
Alternative Energy (geothermal, ocean wave, solar, wind)
38 1,385,140,411 48.9% 36,451,063
Cyprus 2 10,350,000 0.4% 5,175,000
Estonia 1 42,000,000 1.5% 42,000,000
Georgia 5 133,563,640 4.7% 26,712,728
Greece 1 50,000,000 1.8% 50,000,000
Jordan 7 188,698,491 6.7% 26,956,927
Kazakhstan 2 81,545,272 2.9% 40,772,636
Mongolia 1 22,855,053 0.8% 22,855,053
Montenegro 2 52,700,000 1.9% 26,350,000
Morocco 2 86,824,364 3.1% 43,412,182
Poland 4 167,501,425 5.9% 41,875,356
Romania 1 73,426,313 2.6% 73,426,313
Serbia 1 3,100,000 0.1% 3,100,000
Tajikistan 1 45,710,107 1.6% 45,710,107
Turkey 4 405,905,746 14.3% 101,476,436
Ukraine 4 20,960,000 0.7% 5,240,000
Electric Power Generation 8 770,640,036 27.2% 96,330,004
Albania 1 118,000,000 4.2% 118,000,000
Bulgaria 1 80,000,000 2.8% 80,000,000
Egypt 1 173,698,405 6.1% 173,698,405
Jordan 1 67,469,926 2.4% 67,469,926
Kazakhstan 2 105,663,555 3.7% 52,831,777
Serbia 1 200,000,000 7.1% 200,000,000
Turkey 1 25,808,150 0.9% 25,808,150
Electric Power Distribution 8 385,237,547 13.6% 48,154,693
Bosnia and Herzegovina 1 7,500,000 0.3% 7,500,000
Bulgaria 1 69,600,000 2.5% 69,600,000
Kazakhstan 1 13,077,467 0.5% 13,077,467
EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 34
Industry Op #
NCBI (€) % of total volume
Average size (€)
Kyrgyz Republic 1 4,000,000 0.1% 4,000,000
Romania 2 137,180,203 4.8% 68,590,101
Turkey 2 153,879,877 5.4% 76,939,939
Electric Power Transmission, Control & Distribution
6 231,383,494 8.2% 38,563,916
Bosnia and Herzegovina 1 8,500,000 0.3% 8,500,000
FYR Macedonia 1 37,000,000 1.3% 37,000,000
Kosovo 1 30,000,000 1.1% 30,000,000
Romania 1 8,821,259 0.3% 8,821,259
Tajikistan 1 100,562,234 3.6% 100,562,234
Tunisia 1 46,500,000 1.6% 46,500,000
Natural Gas Distribution (Kazakhstan)
1 58,122,076 2.1% 58,122,076
TOTAL 61 2,830,523,563 100.0% 46,402,026
Source: EvD elaboration based on DWH data
EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 35
Figure 10: Power & Energy – industry distribution by
number of operations (Dec 2013 – Apr 2017)
Figure 11: Power & Energy – industry distribution by
volume (Dec 2013 – Apr 2017)
Source: EvD elaboration based on DWH data
EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 36
Table 10: Power & Energy – geographic distribution (Dec 2013-Apr 2017)
Region & Country Op # NCBI (€) %
South-Eastern Europe 15 825,827,775 29.2%
Albania 1 118,000,000 4.2%
Bosnia and Herzegovina 2 16,000,000 0.6%
Bulgaria 2 149,600,000 5.3%
FYR Macedonia 1 37,000,000 1.3%
Kosovo 1 30,000,000 1.1%
Montenegro 2 52,700,000 1.9%
Romania 4 219,427,775 7.8%
Serbia 2 203,100,000 7.2%
Southern and Eastern Mediterranean 12 563,191,186 19.9%
Egypt 1 173,698,405 6.1%
Jordan 8 256,168,417 9.1%
Morocco 2 86,824,364 3.1%
Tunisia 1 46,500,000 1.6%
Turkey 7 585,593,773 20.7%
Turkey 7 585,593,773 20.7%
Central Asia 10 431,535,764 15.2%
Kazakhstan 6 258,408,370 9.1%
Kyrgyz Republic 1 4,000,000 0.1%
Mongolia 1 22,855,053 0.8%
Tajikistan 2 146,272,341 5.2%
Central Europe and Baltics 5 209,501,425 7.4%
Estonia 1 42,000,000 1.5%
Poland 4 167,501,425 5.9%
Eastern Europe and Caucasus 9 154,523,640 5.5%
Georgia 5 133,563,640 4.7%
Ukraine 4 20,960,000 0.7%
Cyprus and Greece 3 60,350,000 2.1%
Cyprus 2 10,350,000 0.4%
Greece 1 50,000,000 1.8%
TOTAL 61 2,830,523,563 100.0%
Source: EvD elaboration based on DWH data
EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 37
Figure 12: Power & Energy - Net Cumulative Bank Investment and number of projects (Dec 2013 – Apr 2017)
Source: EvD elaboration based on DWH data
EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 38
Table 11: Power & Energy – Transition Impact at approval (Dec 2013-Apr 2017)
TI TI Risk
High Medium TOTAL
Excellent 4 1 5
Strong Good 17 - 17
Good 30 3 33
Moderate Good - - -
Satisfactory - - -
Marginal - - -
SUB-TOTAL 51 4 55
N/A* - - 6
TOTAL 61
Source: EvD elaboration based on DWH data
* 6 projects does not have TI rating because either are sub-operations in a Framework or because they are an extension of an existing operation
Table 12: Power & Energy – SEI and Environmental Category (Dec 2013-Apr 2017)
SEI Category / Env Category A B C None TOTAL
1 - Corporate Energy Efficiency - 1 - - 1
3 – Cleaner Energy Production 3 15 - - 18
4 - Renewable Energy 10 29 - - 39
No SEI - 3 - - 3
TOTAL 13 48 - - 61
Source: EvD elaboration based on DWH data
EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 39
Technical cooperation (TC)
Transactional TCs
For the purpose of this Approach Paper the evaluation team has analysed the Board approval document
for each investment to understand if transactional technical cooperation resources were expected to be
used for the purpose of the achievement of the projects’ results. Transactional TCs include resources
marked mainly for: project preparation, project implementation support, and capacity building.
The evaluation team has considered only the transactional TCs linked to investments part of the
population under consideration (investments approved from December 2013 until April 2017). The
information provided below does not include all other TCs implemented by the EBRD in the period under
review, but related to investments approved before December 2013 or not approved yet.
Table 13: Natural Resources operations – transactional TCs per country
Country €
Greece 1,352,465
Ukraine 583,218
Regional 82,465
Mongolia 72,306
Egypt 56,195
Kosovo 17,060
TOTAL 2,163,709
Source: EBRD databases
Table 14: Natural Resources operations – donor distribution of transactional TCs
Donor €
Not available 1,585,000
Ukraine Multi Donor 292,918
EBRD SSF 157,290
Japan 57,131
EU 56,195
ETC Fund 15,175
TOTAL 2,163,709
Source: EBRD databases
Table 15: Power & Energy operations – transactional TCs per country
Country €
Tajikistan 7,577,486
Kazakhstan 2,785,922
Albania 2,705,640
EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 40
Country €
Egypt 1,918,049
Kosovo 1,880,000
Morocco 1,595,000
Tunisia 1,274,000
Kyrgyz Republic 1,096,410
Turkey 397,598
Serbia 324,130
Georgia 276,790
Ukraine 212,999
Bosnia and Herzegovina 94,970
Jordan 66,320
Cyprus 47,500
Regional 11,400
TOTAL 22,264,214
Source: EBRD databases
Table 16: Power & Energy operations – donor distribution of transactional TCs
Donor €
EBRD SSF 8,188,118
Austria 5,476,212
Italy 2,197,640
SEMED Multi Donor 1,728,965
CIF 1,445,978
Japan 1,000,000
UK 998,920
EU 397,598
Norway 396,000
GEF 231,283
Not available 138,500
Germany 65,000
TOTAL 22,264,214
Source: EBRD databases
EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 41
Non-transactional – policy dialogue
For the purpose of tacking stock of policy dialogue activities carried out in the framework of the ESS, the
evaluation team has considered data provided by EBG, E2C2, LTT, ESD, and EBRD TC databases. As in
2013-2014 the EBRD has modified the way in which TC information are captured, the evaluation team has
done its best effort to consolidate those information and assumes that what has been provided covers the
almost the entirety of activities carried out. A snapshot of the policy dialogue activities is provided below.
The evaluation team will harvest more during the implementation phase of the Review.
Additional policy dialogue activities carried out by the Bank in the sector and not funded by donors will
complement the data outlined above and in Table 17 and Figure 13. Information about those activities will
be collected during the Review as provided by the teams involved – as not captured systematically in any
EBRD system
Table 17: Energy related technical cooperation activities marked as non-transactional (2013-2017)
Country € committed Number of TCs
Egypt 4,577,360 10
Regional 2,605,015 9
Kazakhstan 1,248,083 5
Turkey 1,043,479 6
Montenegro 877,212 4
Kyrgyz Republic 770,000 1
Mongolia 765,000 5
Bulgaria 750,000 1
Tajikistan 679,952 3
Georgia 667,999 4
Morocco 530,000 4
Jordan 471,450 2
Ukraine 398,840 3
SEMED 350,100 2
Moldova 349,900 2
Romania 300,220 3
Bosnia and Herzegovina 249,280 2
Albania 210,860 1
Poland 199,575 3
Tunisia 194,990 2
Belarus 180,000 1
Kosovo 157,205 1
FYR Macedonia 150,000 1
Serbia 139,990 1
Croatia 61,652 1
TOTAL 17,928,162 77
Source: EvD elaboration based on EBRD documents and databases
EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 42
Figure 13: Donors for energy related technical cooperation activities marked as non-transactional (2013-2017)
Source: EvD elaboration based on EBRD documents and databases
EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 43
Annex 3 Documents consulted
Documents related to ESS approval process and implementation
Year Date Title
2013 - Summary of comments received on the existing Energy Operations Policy (2006) and the forthcoming Draft Energy Strategy in the period from November 2012 to April 2013
14 March Information Session: Energy Strategy - key issues
27 June Energy Sector Strategy
12 September Minutes of the Meeting of the Financial and Operations Policies Committee of 11 July 2013
15 November Energy Sector Strategy
15 November Energy Sector Strategy: report on the invitation to the public to comment
3 December Minutes of the Meeting of the Financial and Operations Policies Committee of 25 November 2013
3 December Energy Sector Strategy
4 December Report by the Chairman of the Financial and Operations Policies Committee on Energy Sector Strategy
10 December Energy Sector Strategy
2014 29 January Minutes of the Board Meeting of 10 December 2013
25 September Methodology for the assessment of coal fired generation projects
- Energy Sector Strategy Operational Performance Indicators
2016 07 April Energy Sector Strategy Update
1 August Minutes of the Meeting of the Financial and Operations Policies Committee of 14 April 2016
Relevant EBRD Strategic documents
Year Date Title