Project Due Diligence Authors/affiliations Gaye Francis and Richard Robinson

R2A Due Diligence Engineers, Melbourne, Australia ABSTRACT Critical to Australia’s continued growth is the delivery of large infrastructure projects and the procurement of fit for purpose plant and equipment in a timely and cost effective manner. But many fail to deliver. Interestingly, one of the most common difficulties across projects seems to rest around an exclusive reliance on the Risk Management Standard (ISO 31000) as the basis to manage project risk. For high consequence, low likelihood events (the potential ‘project show stoppers’) the Standard of itself fails the test. The reason is simple; it is risk (likelihood and consequence) based not criticality (consequence) based. Like our courts, criticality based assessments ignore the likelihood side of the risk equation initially. If the event is considered credible and could prove fatal to the project, it is a potential project showstopper and must be dealt with up-front. The purchase of large rolling stock fleets in Melbourne in recent years has seen PTV adopt such a criticality driven project due diligence (rather than a project risk management) procurement process. This required an adaptation of a quite standard military intelligence technique, which has proved very successful. It involves keeping everyone’s eyes focused on the prize and ensuring no one loses sight of the overall objectives of the project. A series of independently facilitated functional threat and vulnerability reviews at strategic points in the procurement process helped to achieve this. A case study of the Class E tram procurement process is presented together with representative outputs of the process. Specifically, the approach does not use the risk based approach of the risk management standard as this is logically unable to positively demonstrate due diligence for the critical, rare potential project show stoppers, the issues of greatest concern and difficulty for senior decision makers. 1. INTRODUCTION Given the transport and population growth challenges facing many Australian and International cities, there continues to be large investment in various transport solutions including road, rail, aviation and maritime sectors. Investment in these large projects is essential to the success of our communities and the lives of all Australians. Appropriate infrastructure and the procurement of fit for purpose plant and equipment in a timely and cost effective manner is critical to Australia’s continuing growth. However, a number of these projects fail to deliver. There are many reasons cited including:

• Lack of clear links to strategic outcomes • Lack of effective stakeholder engagement • Lack of senior management support • Specialist groups within an organisation operating in silos

 

 

2. PROJECT GOVERNANCE OBLIGATIONS The attention on large projects, especially government projects, which rely on public funding, is increasing. The public is demanding transparency and accountability by those responsible to ensure projects deliver promised outcomes. Accountabilities and responsibilities associated with an organisation’s business as usual activities are laid down in their organisational governance arrangements. However, seldom does an equivalent framework exist to govern the development of projects. Project governance is one of the key factors often cited as to why projects fail. This is because the project framework is not contextually sound and aligned with the project’s critical success factors (the desired and expected outcomes). Project activities that don’t align with the project’s strategic objective mean there is a misdirection of resources to the more likely risk issues rather than potentially catastrophic project show stoppers. For projects to be successful, in terms of both project performance and project delivery, they require a contextually sound, robust and transparent project governance structure and effective risk management processes. Project governance responsibilities include:

• Community • Contractual • Legal • Professional • Reputational • Safety • Stakeholders

Such governance obligations are being increasingly met using the concept of due diligence. Due diligence is a legal concept. It represents an aspect of moral philosophy, that is, how the world ought to be and how humanity should behave in order to bring this about. It is forensically tested with the advantage of hindsight. Legally due diligence seems to manifest itself in at least two ways:

1. As a defence against negligence in common (case) law, and 2. In statute law, especially for fiduciary concerns, environmental issues, and more recently,

health and safety matters. Due diligence has been a primary defence against the tort (or wrong) of negligence in the common law. In this context, what constitutes due diligence in Australian case law has been established by the High Court of Australia. It basically means that a reasonable person in the same position (reasonable in the eyes of the court which has the advantage of 20:20 hindsight), would have undertaken certain procedures and processes to ensure that on the balance of probabilities, the undesirable occurrence shouldn’t have happened. Conceptually, the overall situation is perhaps best summarised by Chief Justice Gibbs of the High Court of Australia:

Where it is possible to guard against a foreseeable risk, which, though perhaps not great, nevertheless cannot be called remote or fanciful, by adopting a means, which involves little difficulty or expense, the failure to adopt such means will in general be negligent. Turner v. The State of South Australia (1982) (High Court of Australia before Gibbs CJ, Murphy, Brennan, Deane and Dawson JJ).

With regards to legislation, there are a number of examples of due diligence including:

• Commonwealth Corporations Act (2001) • Model Work Health and Safety Act/s (2011) • Rail Safety National Law Act/s (2012) • NSW Protection of the Environment Act (1997)

 

 

3. WHY PROJECT RISK MANAGEMENT ISN’T EFFECTIVE Projects conceptually have an interesting risk profile as shown in Figure 1. At the time the project is commissioned, someone, often a government minister has made promises of what the project will achieve. The task of the project manager is then to deliver that project on time, within budget and to specification. Therefore, project managers complete downside risk assessments from a promised upside risk position. However, this is often outside the context of the project’s overall performance objectives.

Figure 1: Project risk profile

One of the most common difficulties across projects seems to rest around an exclusive reliance on the Risk Management Standard (ISO 31000) as the basis to manage project risk. For high consequence, low likelihood events (the potential ‘project show stoppers’) the standard of itself fails the test. The reason is simple; it is risk (likelihood and consequence) based not criticality (consequence) based. Like our courts, criticality based assessments ignore the likelihood side of the risk equation initially. If the event is considered credible and could prove fatal to the project then it is a potential project showstopper and must be dealt with up-front. It should take into consideration the whole life of the project from concept to operation and maintenance. The concept is shown in Figure 2 below.

Figure 2: Project due diligence vs project risk management

 

 

Another common cause of ineffective project risk management for large projects is that each of the disciplines / project groups often undertake risk assessment activities in their own silos without regard for the high level project performance objectives. Each specialist group comes to an internalised understanding of what is important. Senior management therefore receives pieces of the puzzle but not within a unified framework. Common mode and common cause issues can also be overlooked using this bottom up silo approach as shown in Figure 3.

Figure 3: Bottom up silos miss common mode failures

4. PROJECT DUE DILIGENCE Project due diligence provides a transparent argument as to why the critical success factors for the project will be achieved. It ensures there are no foreseeable project show stoppers within a project’s scope and that all reasonable practicable precautions are in place for any issues of concern that remain. Project due diligence ensures projects succeed in both performance and delivery by encouraging the project team to look back from the desired project end-state (achievement of the critical success factors of the end users) to focus on those critical threats that will affect project outcomes.

Figure 4: Project Due Diligence Sequence

 

 

This top-down approach of identifying all project end state critical success factors allows clients’, partners’ and other stakeholders’ expectations to align. It is particularly valuable in identifying enhancements during planning stages to ensure that no foreseeable project show stoppers are encapsulated in the project’s scope and that tenderers are specifically advised of significant identified residual risk issues that need to be addressed in their submissions. The primary high-level process is the project functional vulnerability analysis. A vulnerability assessment is a top down technique that identifies the assets or critical success factors for the project and the credible threats. A vulnerability arises when a threat affects a critical success factor. Vulnerabilities are characterised using a criticality scale (minor, moderate or critical) typically using the following criteria:

xxx xx x - v/a

Potential project show stopper. Project not viable until addressed. Moderate potential vulnerability that needs to be specifically addressed by tenderers. Minor potential vulnerability. To be addressed by successful contactor project manager. No detectable change. Possible value adding.

Criticality refers to the assessment of the credible worst-case outcome for a known credible threat. It does not consider the likelihood component and is therefore not a direct measure of risk in the usual way risk is defined. It simply asks the question, if it did happen, how bad can the outcomes to the project be? A vulnerability assessment provides a completeness check to ensure that no significant threats and vulnerabilities have been overlooked. This approach also has the advantage of permitting the identification of superior risk control practices that may exist including elements of best practice. A vulnerability matrix developed in a workshop is one of the most successful means of obtaining consensus on the relative importance of vulnerabilities. However this can only be a successful approach if the workshop membership present in the room has the best available knowledge, that is, a group of persons with the greatest understanding of the various threat scenarios. 5. CASE STUDY – PROCUREMENT OF CLASS E TRAMS FOR MELBOURNE In December 2008, the Victorian Government released its $38b Victorian Transport Plan. As part of the Government’s $4.2b investment in rolling stock, there was a commitment to purchase 50 new low floor trams for the Melbourne tram system. This was primarily to address Melbourne’s rapidly growing needs. The new trams are high capacity, capable of carrying at least 200 passengers, energy and environmentally friendly and fully compliant to Disability Discrimination Act (DDA) standards. Prior to the award of the contract to Bombardier, R2A was engaged as the independent risk advisors to the Victorian government for this project. Public Transport Victoria (PTV) adopted a criticality driven project due diligence (rather than a project risk management) procurement process. The first task completed was a high level vulnerability assessment to ensure that all credible critical vulnerabilities had been identified and that all reasonable practicable precautions were in place. This provided a completeness check of issues for the entire project to ensure the context of all future risk work.

 

 

The critical success factors were defined as an element that was essential to the success of the project from the viewpoint of the Department of Transport (at the time, now PTV) both in terms of project outcomes (once delivered) and project delivery. These include:

• Enhanced service capability / flexibility • Maintainable asset over design life at target cost • Operationally safe • Compliant with all government policies • Project reputation • First tram operational on time • First 5 trams operational on time • First 5 trams proven to be maintainable and reliable on time • Sixth tram operational on time • Reliable fleet maintained during rollout • Network level of service maintained during rollout • Contractor safety sustained during delivery • Project delivered within budget

Credible threats were identified using a project lifecycle approach considering:

• Procurement • Contract • Design • Manufacturing and assembly • Transportation • Commissioning and testing • Operations and Maintenance

The completed vulnerability table is shown in Table 1 on the following page. Note that this is a criticality assessment not a risk assessment.

 

 

xxx xx x -

Potential project show stopper. Project not viable until addressed. Moderate potential vulnerability that needs to be specifically addressed by tenderers. Minor potential vulnerability. To be addressed by successful contactor project manager. No detectable change.

Table 1: Tram Procurement Project Vulnerability Table

 

 

The results of the above vulnerability assessment then fed into the project precautionary and risk register. A sample table for the threat identified as critical (xxx) is shown below.

Item Threat Critical success factor(s) impacted

Vulnerability / mechanism Criticality Proposed Controls / Treatment Plans

Procurement 1 Vehicle

specification issues

Enhanced service capability / flexibility, Maintainable asset over design life at target cost, Operationally safe, Compliant with Government Policies, Project reputation, First tram operational on time, First 5 trams operational on time, First 5 trams proven to be maintainable and reliable on time, Sixth tram operational on time, Reliable fleet maintained during rollout and Project delivered within budget

xxx 1. Tram wheelchair lift trials conducted 2. Review of tram axle load and vehicle kinematic

envelop 3. Franchisee submitted functional / operational

requirements 4. RTBU consultation re: new vehicles 5. Franchisee consultation, review, feedback and

acceptance 6. Facilitated discussions with industrial parties 7. Social Transit Unit and PTAC consultation and

acceptance 8. Consultation with PTSV 9. Consultation with Myki and AVL project teams 10. Functional specification developed internally 11. Internal critique / peer review 12. Review of potential supplier proposals 13. Technical consultation with short listed suppliers

2 Evaluation issues

First tram operational on time, First 5 trams operational on time and First 5 trams proven to maintainable and reliable on time

xxx 1. Experienced evaluation team 2. Robust agreed evaluation methodology and criteria 3. Procurement conduct plan and compliance 4. Probity auditor 5. Agreed approvals process (Governance process) 6. Strategic Procurement Plan 7. Two stage procurement process (EOI and Tender) 8. Legal and commercial advisors 9. Franchisee technical consultation 10. Suppliers experience in the market 11. EOI / RFT documentation peer reviewed 12. Project Gateway Review process

3 Contract model selection issues

Maintainable asset over design life at target cost, Reliable fleet maintained during rollout and Project delivered within budget

The contract model needs to deliver the best value for money over the whole life of the vehicle within the context of the new and any future franchise

xxx 1. Franchisee consultation 2. Project team experience in rolling stock supply,

maintenance and procurement 3. Legal and commercial advisors 4. Consultation with Treasury (Commercial) 5. Market feedback regarding commercial arrangements

in EOI 6. Risk review of procurement model

4 Refranchising transition issues (2009)

First tram operational on time, First 5 trams operational on time and First 5 trams proven to maintainable and reliable on time

Consultation with the preferred franchisee not possible until formally appointed - changes to franchisee requirements and possible delay to project

xxx 1. Project not to proceed to request for tender until consultation with the franchisee has occurred

2. Documentation prepared to final draft status 3. Franchisee representatives to join project team

Contract

Design 5 Failure to

ensure good practice design

Enhanced service capability / flexibility, Maintainable asset over design life at target cost, Operationally safe, Compliant with Government Policies, Project reputation, First tram operational on time, First 5 trams operational on time, First 5 trams proven to be maintainable and reliable on time, Sixth tram operational on time, Reliable fleet maintained during rollout and Project delivered within budget

xxx 1. Reputable contractor 2. Clear functional specification 3. Design based on existing product and components 4. Comprehensive agreed design review process (DoT,

Franchisee and Supplier) 5. Good communication links during design process 6. Appropriate project personnel for design process 7. Independent assessor to review design disputes 8. Mock up tram tested with stakeholders

6 Delay in the delivery of the approved design

First tram operational on time and First 5 trams operational on time

Mock up tram and associated consultation could delay delivery

xxx 1. Reputable contractor 2. Clear functional specification 3. Design based on existing product and components 4. Comprehensive agreed design review process (DoT,

Franchisee and Supplier) 5. Supplier progress payments for design completion 6. Good communication links during design process 7. Appropriate project personnel for design process

 

 

 

 

Item Threat Critical success factor(s) impacted

Vulnerability / mechanism Criticality Proposed Controls / Treatment Plans

Design continued 7 Incompatibility

with existing (& enhanced) infrastructure

Enhanced service capability / flexibility, Maintainable asset over design life at target cost, Operationally safe, Compliant with Government Policies, Project reputation, First tram operational on time, First 5 trams operational on time, First 5 trams proven to be maintainable and reliable on time, Sixth tram operational on time, Reliable fleet maintained during rollout and Project delivered within budget

xxx 1. Reputable contractor 2. Clear functional and interface specification 3. Design based on existing product and components 4. Comprehensive agreed design review process (DoT,

Franchisee and Supplier) 5. Good communication links during design process 6. Appropriate project personnel for design process 7. Independent assessor to review design disputes 8. Franchisee representative to join project team ASAP 9. Ensure infrastructure enhancements compatible with

new vehicles

Manufacture and Assembly 8 Failure to

ensure good practice manufacturing techniques

Enhanced service capability / flexibility, Maintainable asset over design life at target cost, Operationally safe, Compliant with Government Policies, Project reputation, First tram operational on time, First 5 trams operational on time, First 5 trams proven to be maintainable and reliable on time, Sixth tram operational on time and Project delivered within budget

xxx 1. Reputable contractor, sub-contractors and suppliers with DoT rights to refusal

2. Contractor to have demonstrated technology transfer experience

3. Clear functional and interface specification 4. Design based on existing product and components 5. Comprehensive agreed design review process (DoT,

Franchisee and Suppliers) 6. Third party QA advisor 7. Audits by project team 8. Mock up tram 9. Contractor to submit full manufacturing and QA plans 10. Initial robust test period for first 5 trams

Transportation

Commissioning and Testing 9 Franchisee

non-acceptance

First tram operational on time, first 5 trams operational on time and first 5 trams proven to be maintainable and reliable on time.

xxx 1. Franchisee representative part of the project team 2. Functional specification agreed by Franchisee prior to

RFT 3. Increased fleet size and operation advantageous to

Franchisee

10 Regulatory non-acceptance

First tram operational on time, first 5 trams operational on time and first 5 trams proven to be maintainable and reliable on time.

xxx 1. Safety Regulator consultation regarding vehicle concept, procurement model and procurement guidelines

2. Franchisee consultation with Safety Regulator prior to approval application

3. Risk based procurement model including reputable supplier, clear functional specification

4. Design based on existing product and components 5. Comprehensive agreed design review process (DoT,

Franchisee and Supplier) 6. Good communication links during design process 7. Appropriate project personnel for design process 8. Independent assessor to review design disputes 9. Mock up tram tested with stakeholders 10. RAMS processes to be contained in contact

11 Industrial non-acceptance

First tram operational on time, first 5 trams operational on time and first 5 trams proven to be maintainable and reliable on time.

xxx 1. Early industrial consultation with all relevant unions 2. RTBU requirements resulting from early consultation

process included in specification 3. Mock up tram tested for acceptance/sign-off with

stakeholders 4. Extensive initial trial of first 5 trams 5. Drivers on project team via YT (at appropriate time) 6. Stakeholder Communication and Management Plan 7. Mock-up Tram Evaluation Plan 8. Training Maintenance & Operations Manuals available 9. Maintenance secondee provided to Bombardier 10. Independent cab ergonomic assessment 11. YT cab review and feedback to RTBU from project

team 12. RTBU visited site on 26/4/13. No issues were raised. 13. Monthly meetings with RTBU

12 DoT non-acceptance

Project reputation and first 5 trams proven to be maintainable and reliable on time

Fleet reliability targets not achieved

xxx 1. Appropriate contract targets 2. Design control 3. Manufacturing controls 4. Comprehensive testing program 5. Franchisee encouraged to maximise vehicle km 6. Contractor to provide good technical and maintenance

support 7. Good information on vehicle performance collected

Operations and Maintenance

Table 2: Precautionary Review

 

 

5. CONCLUSION Critical to Australia’s continued growth is the delivery of large infrastructure projects and the procurement of fit for purpose plant and equipment in a timely and cost effective manner. For projects to be successful in terms of both project performance and project delivery, they require a contextually sound, robust and transparent project governance structure and an effective project risk management process. These two elements combined demonstrate project due diligence. The purchase of large rolling stock fleets in Melbourne in recent years has seen PTV adopt such a criticality driven project due diligence (rather than a project risk management) procurement process, which has proved very successful. It involves keeping everyone’s eyes focused on the prize and ensuring no one loses sight of the overall objectives of the project. Specifically, the approach does not use the risk based approach of the risk management standard as this is logically unable to positively demonstrate due diligence for the critical, rare potential project show stoppers, the issues of greatest concern and difficulty for senior decision makers. R2A’s process outlined in this paper ensures the organisation has confidence in the project risk management process and will ensure that the project is right the first time resulting in a successful outcome. References 1. Robinson Richard M, Gaye E Francis, Peter Hurley et al (2014). Risk and Reliability: Engineering

Due Diligence (9th Edition, Updated). R2A Pty Ltd.