INDEPENDENT QUALITY REVIEW (JASPERS IQR) · Issues noted in appraisal of Polish projects ... for...
Transcript of INDEPENDENT QUALITY REVIEW (JASPERS IQR) · Issues noted in appraisal of Polish projects ... for...
INDEPENDENT QUALITY REVIEW
(JASPERS IQR)
CUPT Conference on Traffic Modelling
Warsaw, 25 May 2017
2
I. Role of IQR
II. Scope of IQR assessment
III. Demand modelling and economic analysis
IV. Issues noted in appraisal of Polish projects
V. Conclusions
Contents
I. Role of Independent Quality Review
3
JASPERS in 2014-2020
European
Investment Bank
JASPERS
JASPERS
Advisory
JASPERS
Networking
Competence Centre
(NCC)
JASPERS
Independent Quality
Review
(IQR)
I. Role of Independent Quality Review (4)
4
In both routes:
Previous involvement of JASPERS Advisory is beneficial
for the quality of major project application
Identical scope of assessment for demand and economic
analysis for both routes (IQR and PSA)!
When is IQR used?
MANAGING
AUTHORITY
EUROPEAN
COMMISSION
Project Notification
JASPERS IQR
Post Submission
Appraisal (PSA)
I. Role of Independent Quality Review
How many projects have already been assessed by JASPERS IQR
(as of 03/05/2017)?
7 IQR reports issued so far, 6 of which were in transport sector, 2 of
which in turn were from Polish transport sector (1 road, 1 rail)
41 projects assessed in form of a post-submission appraisal,
including, 35 in transport sector, of which 30 from Polish transport
sector (26 roads, 4 rail/urban).
5
II. Scope of IQR assessment
Assessment by IQR team will cover virtually all aspects of the project,
including e.g.,
Technical, legal, financial and administrative capacity
Total eligible and non-eligible costs
Verification of state aid (for IQR projects only)
Option, demand, financial, economic and risk analyses
Climate change and environmental assessment (for IQR projects
only)
Appropriate measures to ensure optimal utilisation of infrastructure
Financing plan
Timetable
6
II. Scope of IQR assessment
With regard specifically to the assessment of the cost-benefit
analysis undertaken by IQR, this includes:
1. Verification of all the criteria for quality review of major projects
(Annex II to Regulation (EU) No 480/2014)
2. Verification of all the requirements of CBA Methodology
(Annex III to Regulation (EU) No 2015/207)
3. Compliance with EC Guide to Cost-Benefit Analysis of Investment
Projects
7
II. Scope of IQR assessment
1. Criteria for quality review of major projects outlined in Annex II to
Commission Delegated Regulation (EU) No 480/2014:
“…Reliability of the demand analysis … based on realistic estimates
and in line with main demographic trends and developments in the
respective sector, justifying the need for the project and the overall
capacity of the project facilities…
…Soundness of the conclusion that the project is economically and
financially viable and has positive socioeconomic effects justifying
the level of support to the extent envisaged under the ERDF or the
Cohesion Fund…”
8
II. Scope of IQR assessment
2. Methodology for carrying out the cost-benefit analysis outlined in
Annex III to Commission Implementing Regulation (EU) No 2015/207:
“…The objective of a cost benefit analysis (hereinafter referred to
as ‘CBA’) in the context of Cohesion Policy is to support the major
project assessment in order to: — assess whether the major project is
worth co-financing (from an economic point of view); — assess
whether the major project needs co-financing (from a financial point
of view)…
…A CBA for a major project shall include the elements set out
below: (1) Presentation of the context; (2) Definition of objectives; (3)
Identification of the project; (4) Results of feasibility studies with
demand and option analysis; (5) Financial analysis; (6) Economic
analysis; (7) Risk assessment…”
9
II. Scope of IQR assessment
2. Methodology for carrying out the cost-benefit analysis outlined in
Annex III to Commission Implementing Regulation (EU) No 2015/207
(cntd):
“…Demand analysis identifies and quantifies the social need for an
investment and considers as a minimum: — the current demand, by
the use of models and actual data; — the forecast demand, from
macroeconomic and sector forecasts and elasticity estimates of demand
to relevant prices, income, and other core determinants; — supply side
aspects including the analysis of existing supply and expected
(infrastructure) developments; — network effect (if any)…
Economic analysis is an analysis that is undertaken using economic
values, reflecting the social opportunity cost of goods and
services…The economic analysis should be carried out in constant
accounting (shadow) prices and shall be undertaken taking the financial
analysis cash flows as a starting point...”
10
II. Scope of IQR assessment
3. Guide to Cost –Benefit Analysis of Investment Projects:
“Demand Analysis. Demand analysis identifies the need for an investment by
assessing: current demand…; future demand (based on reliable demand
forecasting models that take into consideration macro- and socio-economic
forecasts, alternative sources of supply, elasticity of demand to relevant prices and
income, etc.) in both the scenarios with- and without-the-project.”
CBA Guide also includes:
(i) detailed principles underlying demand, economic, financial and risk analyses
(chapter 2)
(ii) discussion of sector specific issues (chapter 3 for transport)
(iii) worked case studies per sector (chapter 3, for road, railway and urban
transport)
11
Inputs Outputs
1. Costs (inv, O&M) -> Economic Analysis -> NPV
2. Parameter values -> Economic Analysis -> ERR
3. Demand -> Economic Analysis -> BCR
III. Demand modelling and economic analysis
12
III. Demand modelling and economic analysis
There are many different (valid) ways to calculate benefits for
transport projects, including:…
UK “COBA” approach – calculate link-by-link (for each link for each
year, time period, vehicle category…) benefits from time savings,
accidents, emissions, vehicle operating costs.
UK “TUBA” approach – calculate based on demand model skim
matrices (for each O-D pair for each year, time period…) benefits
from time savings and perceived vehicle operating costs. Need to
calculate external benefits (e.g., accidents) separately.
“Network indicators” approach – calculate all benefits based on
network indicators (e.g., passenger hours, vehicle kilometers)
derived from traffic model.
Polish approach seen so far more closely follows “network
indicators”. With each approach there are plusses and minuses….
13
IV. Issues noted in appraisal of Polish transport projects
Pros and cons of “Network indicators” approach…
Pros
Simple approach which is reasonably easy to implement
Re-use of standard spreadsheet (only need to check calculus
once)
Cons
Very reliant on traffic model outputs
Volatility of pattern of economic benefits if model outputs not
100% correct
How do you verify validity of calculated benefits?
14
IV. Issues noted in appraisal of Polish transport projects
Volatility: anonymised examples for recent road projects…
15
-100 000 000
-50 000 000
0
50 000 000
100 000 000
150 000 000
200 000 000
250 000 000
2019 2021 2023 2025 2027 2029 2031 2033 2035 2037 2039
Profile of undiscounted benefits
vehicle operating costs time savings accidents environment climate change noise
-150 000 000
-100 000 000
-50 000 000
0
50 000 000
100 000 000
150 000 000
200 000 000
250 000 000
300 000 000
20
18
20
19
20
20
20
21
20
22
20
23
20
24
20
25
20
26
20
27
20
28
20
29
20
30
20
31
20
32
20
33
20
34
20
35
20
36
20
37
20
38
20
39
Profile of undiscounted benefits
vehicle operating costs time savings
accidents environment
climate noise
-50000
0
50000
100000
150000
200000
250000
300000
Change in passenger Hours
Cars Light vans Heavy (w/out trailer) Heavy (with trailer) Bus
-500 000
-400 000
-300 000
-200 000
-100 000
0
100 000
200 000
300 000
400 000
500 000
20
15
20
17
20
19
20
21
20
23
20
25
20
27
20
29
20
31
20
33
20
35
20
37
20
39
20
41
20
43
20
45
Change in PKM per day
Cars Light vans Heavy (w/o trailer) Heavy (with trailer) Bus
IV. Issues noted in appraisal of Polish transport projects
Checking CBA outputs when using “Network indicators”
approach…
Q1. Are calculated benefits stable? Are there any years for which
there is a sharp increase / fall in benefits? If so, why? Do passenger
hours and kms follow a sensible stable trend?
Q2. Does the overall level of benefits in year one make sense? If
you calculated e.g., savings in passenger hours per vehicle, does
the level make sense considering the length of the project and
speeds involved?
Q3. Are estimated speeds (WP, WOP) reasonable?
Q4. Does the overall magnitude of each calculated benefits
makes sense given the nature of the project? E.G., if we were
replacing one single carriageway (1x1 road) with another we would
probably not expect to see much accident benefits.
16
IV. Issues noted in appraisal of Polish transport projects
Some issues noted on recent projects…
• High ERRs. For several road projects, very high ERRs (>>20/30%)
noted. Unusual for a transport project and ‘flags up’ potential issues
with CBA. IQR has had to recalculate ERRs. Better to be
conservative? […no prizes awarded for highest ERR on a project!]
• Low speeds on existing road (to be replaced) in without project
scenario. In some cases speeds of 50-55km/hour on relatively
uncongested rural roads (in seemingly good condition)
• High growth rates or transfer of traffic predicted by model (again,
better to be conservative?).
• Additional detail on model build, calibration and validation.
17
IV. Issues noted in appraisal of Polish transport projects
Other infrequent issues…
Roads sector… justification of cross section (i.e., carriageway
type). Roads where modelled traffic volume either too low or too high
for provided capacity.
Benefits being dependent on other projects on corridor which
may / may not happen.
Incompleteness of project (“half a bypass”) with remaining
sections to be built at a much later date (and therefore with
overcapacity in the intervening period).
Multimodal issues. Difference in approach to competing modes (for
roads, rail excluded; for rail, road included and transfer traffic high).
Need for consistency between modes along same corridor.
18
V. Conclusions
Polish transport sector and roads in particular viewed as the success of the
current programming period.
For road projects some issues noted with stability of demand model
outputs but we understand that following interaction with JASPERS
Advisory colleagues that these are being sorted and for future projects they
should be addressed.
Speed surveys to verify assumed speed on existing roads? [journey time
surveys for rail / PT].
(For roads sector) consider cross section in options analyses?
Some basic checking of demand model outputs / CBA results might be of
use (e.g. hours saved per passenger, stability of profile over time)?....
To minimise delays JASPERS Advisory support could be sought for
difficult projects.
19
Thank you!
For info or further questions on this presentation, please contact:
Fergal Trace
JASPERS IQR
Tel.: +32 2 712 41 84
www.jaspersnetwork.org
20
20