NETLIPSE Infrastructure Project Assessment Tool

Stuart Baker,

Deputy Director of National Rail Projects

Department for Transport, UK

Zagreb, November 10, 2009

Motivation (1)

The European Commission and member states: address the need for a Trans European Transport

Network (TEN-T) have already invested billions in the construction of

several Large Infrastructure Projects (LIPs) to create the TEN-T and to deliver infrastructure for economic gain.

But, these organisations: have limited possibilities for forecasting and

monitoring the effectiveness of these projects; face large delays and cost overruns on the supported

projects and experience local opposition; are aware that knowledge exchange between Large

Infrastructure Projects (LIPs) is scarce.

Motivation (2)

These organisations have a need for: Improvement of the current management and

organisation of LIPs. Insight in ‘the vitality’ of projects on certain

moments, e.g. financing (gate review): to have a reliable insight in risks and opportunities

before deciding; and if decision is ‘go’: as a basis to manage risks and opportunities

As well as the allocation of budgets to the most vital projects to also take into account which projects are most likely to deliver results and stick to programme.

Better insight in the progress of LIPs (risks, opportunities).

Benchmark projects.

The objective of the IPAT

“ The IPAT is a tool that can assess, monitor, benchmark, and evaluate project organisations of large infrastructure projects before, during, and after implementation in a competent and uniform way. In this way it also gives guidelines to a project organisation on the crucial factors in managing large infrastructure projects. ”

IPATInfrastructure Project Assessment Tool

IPAT-Assessment – Assessing when?

1. Ex ante evaluation, ‘go/no go-decision’

To indicate strengths and weaknesses of the project delivery organisation in advance.

2. Monitoring, during implementation:

To indicate the abilities of the project delivery organisation during implementation.

3. Ex post evaluation

To expand the knowledge on the project delivery organisation’s approaches of project planning and implementation (generates a comparative perspective on weaknesses and strengths on different implementation strategies).

4. Benchmark

To create the opportunity to benchmark large infrastructure projects in different stages.

IPAT-Assessment – Who’s involved?

IPAT - Criteria

The scientific criteria for the IPAT:1. Practical - Is the data available?

2. Reliable - Is the assessment carried out in a consistent manner?

3. Valid - Are all aspects measured that are supposed to be measured? Are the Assessors qualified?

A practical criteria:4. Applicability - Is the outcome understandable for

clients and projects in a way that they can benefit from the conclusions and recommendations?

2. IPAT – Results

The IPAT will allow: Project delivery organisations to increase the certainty of

successful execution of projects, resulting in particular in reduced cost overruns and time delays, and;

Clients and funders to understand the deliverability of projects by the project delivery organisations,and;

EU, local governments and financial institutes such as The EIB and The World Bank to monitor and evaluate projects (ex ante and ex post) in a systematic way, and;

the collection of information on research forecasts and future research demands.

IPAT- Building block (1)

IPAT - Building block (2)

Interaction

Control

IPAT - Building Block (3)

IPAT - Building Block (4)

IPAT - Assessment process

IPAT-Questionnaire - 12 Themes

T 1 Political Context

T 2 Objectives, Purpose and Business Case

T 3 Functional Specifications

T 4 Interfaces

T 5 Stakeholder Management

T 6 Finances

T 7 Legal procedures

T 8 Technology

T 9 Knowledge

T10 Organisation and Management

T11 Contracting

T12 Risks (Threats and Opportunities)

IPAT-Questionnaire – Project Phases

Phases or Milestones:

M 1 Initiation of the project

M 2 Funding assembly

M 3 Official approval official planning authority

M 4 Start of execution

M 5 Completion

M 6 Start operation

M 7 5 years after start of operation

M 1 M 2 M 3 M 4 M 5 M 6 M 7

T 1

T 2

T 3

T 4

T 5

T 6

T 7

T 8

T 9

T10

T11

T12

Levels of Importance byMilestone and by Theme

Prioritisation:Level 1: Minimal importanceLevel 2: Little importance

Level 3: Medium importanceLevel 4: ImportantLevel 5: Crucial

IPAT-Questionnaire – Theme 1

Theme 1 Political context M1

1Is there a political consensus about the need and general purpose of the project?

2Is there a political consensus on the defined outcomes from the project?

3Is there a political consensus on the type of infrastructure solutions proposed by the project?

4Is there a political consensus on the defined route of new infrastructure?

5Are there clear interfaces between the politicians and the clienting of the project?

6 Is there political consensus on the funding solutions?

7

Are there arrangements to handle any political interfaces, including differences of view between the various levels of (inter)national, regional and local, city level?

8How is the relation between the Sponsor/Client and the politics and are differences dealt with in proper ways?

Scoring Methodology - Criteria

The objective of scoring the criteria is to identify the main strengths and weaknesses of the project and its organisation within each theme.

Scoring table reflects weakness of the project (low score) and strength of the project (high score):

Score Qualification This reflects

Score 1 Very negative effect on the project being successful

a vital need to improve immediately

Score 2 Negative contribution to a successful project

an need to improve a weak area

Score 3 Positive contribution to a successful project

Further progress can be made to achieve a better result

Score 4 Very positive contribution to a successful project

a clear strong area for the project (probably close to best practice)

Scoring Methodology - Themes

The assessors will score the themes as follows: ΣImportance * Score criterion

The ‘pass score’ represents the score which the panel estimates an adequate but not brilliant project would get.

The score for a theme should pass the pass score (ΣImportance * Score > pass score)

IPAT – Testing the IPAT

Three pilot projects, tests in different project phases

The objective of the pilot projects is to:

1 – validate and fine-tune the questionnaire

2 – further develop the scoring methodology

3 – further develop the final analysis of the IPAT and co consider the appropriate ‘pass scores’

Development of the IPAT – Delivery

Completed – 2 ‘pilot pilots’ to test the questionnaire

Further steps: Q1, 2010:

Three pilot projects Further development of the IPAT

Q2, 2010: Evaluation of Pilot projects Further develop of the IPAT

Q3, 2010: Finalize the IPAT Complete and deliver the IPAT Assessors manual. IPAT-Brochure IPAT-assessors Course.

IPAT – Your input!

The IPAT needs to be widely understood and validated and improved

Projects are needed to pilot the IPAT The IPAT needs ‘buy in’ and acceptance by member states

to achieve broad acceptance so that it may be used as an effective tool by the EC and other users

The IPAT needs…

...your experiences and input to make it a tool

that will be used!

Time for your input now please...

Can you really do a

reliable and comparable

assessment of Projects?