Value-For-Money IT

Are You Operating Economically, Efficiently and Effectively?

Presenters: Ron Foster, CISA, CIA, PMP, CMA Auditor General – City of Oshawa Paul Wallis, CMA, CIA, CISA Director, Internal Audit – Region of Peel

Agenda

Value for Money Auditing - What is it?

IT Value - What is it? - Why is it Important? How

does it Link to Risk and Governance?

Value IT Example

Value for Money Audit Process Models and

Tools

Value-for-Money Auditing

Lets Clear The Air!

(It is More than Public

Sector Auditing)

Definitions??

Classic Definition

Value-for-Money Auditing is one of the Three

Elements of Comprehensive Auditing.

Comprehensive Auditing Embraces Three

Related but Separate Aspects of Public Sector

Accountability Including:

Financial Reporting

Compliance with Authorities

The Economical, Efficient and Effective

Management of Public Funds and Resources

Brief History

In 1977, the Parliament of Canada gave its

Auditor a Mandate to Report whether Money

was Spent with Due Regard to Economy

and Efficiency in the Acquisition and

Management of Goods and Services and

whether the Effectiveness of Programs is

being Measured and Reported

More Brief History

Comprehensive Auditing is now Practiced in

Virtually all Provincial Governments and in the

Federal Government

It is now Practiced by both External, Legislative

Auditors and Internal Auditors

Federal Crown Corporations are by Law

Required to Conduct Periodic “Special

Examinations” that Invoke all the Principal

Elements of Comprehensive Auditing

Economy

Refers to the Acquisition of the Appropriate Quality and Quantity of Financial, Human and Physical Resources at the Appropriate Times and at the Lowest Reasonable Cost

Right Amount

Right Place

Right Time

Right Kind

Right Cost

Efficiency

Maximizing outputs

for fixed level of

inputs

Minimizing inputs for

fixed level of outputs

Inputs include

physical and human

resources

Outputs include

services provided

Measures the Use of

Resources [Outputs]

Does not Measure

Quality or Relevance

[Outcomes]

[Efficient use of Resources

does not mean Business

Outcomes were met!!!]

Effectiveness

Refers to the achievement of objectives or other intended effects of operations, activities or programs

Highest Level of Accountability

Most Elusive

Measurement often not Reflected in Standard Time Period [Fiscal Year]

Program, Process, Service Relevance

Econom

y

Effic

iency

Results

(E

ffec

tive

nes

s)

Outcomes

Outputs

Production/

Delivery

Process

Inputs

Physical

Resources

Acquisition

Process

Money

Customer/Client Satisfaction Client Served Mission/Goal Achievement Financial Viability Profit Cost Benefit/Cost Effectiveness Quantity Quality Timeliness Price/Cost

Unit Cost Productivity

Quantity Quality Timing Price

Unit Cost Productivity Policies

Amount Timing

Value For Money Model

Source: Adapted from - Performance Auditing, A Measurement Approach 2nd Edition; Ronell Raaum and Stephen Morgan

Human Resources Materials

Policies Procedures Controls

Goods Services Programs

Goods Services

Input

Process

Output

Outcome

Private Sector Government

Example Outcome Measures

Customer Satisfaction

Market Share

Earnings

Profit

Return on Investment

Liquidity

Dividends per Share

Customer Satisfaction

Proportion - Target Population Served

Mission or Goal Achievement

Break Even; Cost Recovery

Cost-Benefit

Financial Viability

Cost-Effectiveness

Source: Performance Auditing, A Measurement Approach 2nd Edition; Ronell Raaum and Stephen Morgan

Value-For-Money Audit Standards

PS 5300 of the CICA Handbook [Standards for Assurance Engagements]

The IIA’s International Standards for the Professional Practice of Internal Auditing

INTOSAI [ISSAI 3000 – 3100, Performance Audit Guidelines]

• Value-for-Money Audit Manual [Auditor General of Canada]

What is Value??

Which do you Prefer……..Value??

Information Technology

How do we know IT

Is Enabling Positive -

if not Transformational -

Business Value??

Value?

2008 – 24% Fail, 32% Successful, 44% Challenged

[CHAOS SUMMARY 2009]

2002

20% Of All Expenditures - Wasted!!

Where is the Value??

IT Project Failure = Lost Value

• Oh…That’s What You Wanted!! [Requirements not Defined]

• Geez….I Thought Everything Was On Track!! [Weak Project Manager]

• It’s Those Technology Guys!! [Business Owners not Involved]

• It Worked When We Tested It!! [No Change Management]

• Only 1000 Days to Retirement, I’ll Wait It Out!! [No Commitment]

• ………..and More!!

Excuses and Face Saving!!

IT Project Costs

20%

80%

20% [Software Costs]

80% [Project Management, Process, Bureaucracy…etc]

Some Necessary…..some not!!

The Technology is not the Problem as much as how it is Used!!

Value/Governance Relationship

Effective IT Governance

is the single most

important predictor of the

value an organization

generates from IT.

Peter Weill and Jeanne W. Ross – IT

Governance, How Top Performers

Manage IT Decisions for Superior

Results

Strategic Question

Architecture Question

Value Question

Delivery Question

Adapted from Val IT Framework 2.0

[Input] [Outcome]

[Process] [Output]

IT Governance

1. Strategic Alignment

Aligning with the Business and

Providing Collaborative

Solutions

2. Value Delivery

Focus on IT Expenses and

Proof of Value

3. IT Asset Management

Knowledge, Infrastructure and

Partners

4. Risk Management

Safeguarding Assets and

Disaster Recovery

5. Performance Measurement

IT Scorecards

Value and Governance

IT Governance defines a structure of relationships,

processes and measures to direct and control IT

assets (e.g. people, finance, infrastructure) in order to

achieve the enterprise's goals by adding value while

balancing risk with return

It helps to define roles and responsibilities and

specify an accountability framework to encourage

desirable behaviour in IT and accountability for the

use of IT assets. IT governance also helps to

standardize best practices and define monitoring

methods

Value and Governance Issues

Heightened Management Expectations

Linkage of Managing IT Services and

Priorities to Business Risks, and Need for

Effective Internal Control

Best Practices - What are they, and are we as

an Organization Appropriately

Implementing?

Just how exactly do we know if IT is being

Managed Effectively?

Risk and

Opportunity

Risk

Management

Value

Management

IT Governance

and Process

Management

IT Related

Events

Risk and Opportunity

Risk IT Val IT

Cobit

IT Performance Framework

Source ISACA - 2009

IT Value Architecture

IT Architecture/Value Mapping

Funding People Equipment Tools

Inputs

Processes

Outcomes & Outputs

Strategic Business Objectives

IT Business Objectives IT Governance Board

Outcomes Corporate Profitability - Private Sector [Short and Long Term] Program Success - Public Sector [Citizen Satisfaction] Effectiveness

Outputs Value Capture – Increased IT Profits, Increased Service Delivery [External] Customer Loyalty and Retention – Increased Sales/Use from Existing Customers Customer Acquisition – Increased Sales/Use from New Customers Channel Optimization – Increased Site Traffic and Sales Efficiency

Outputs Direct Cost Savings – Reduced IT Costs and Other Direct Costs [Internal] Improved Quality – Reliable Information, Less Inspections, Lower Cost of Quality Increased Capacity Use – Optimal Use of Existing Resources Time Savings – Shortened Process Cycles Increased Productivity – Operational Improvements Efficiency

Processes IT Systems – Appropriate Processes for Effective Implementation IT Structure – Integration into Business Unit Structure IT Strategy – Coherent and Aligned Strategy Leadership – Commitment and Focus on IT Initiatives Efficiency

Inputs Resources – Adequate Capital & People Corporate Systems – Training, Processes and Culture Corporate Structure – Organization Structure Corporate Strategy – Alignment/Business Integration External Environment – External Force Adaptation Economy & Efficiency

IT Performance Measures

D1. Recognition of Staff

Suggestions

D2. Staff Absenteeism Rate

D3. Staff Credentials

D4. Staff Retention Rate

D5. Internal Promotion Index

D6. Development Hours Index

D7. Staff Satisfaction Survey

Help Executive Management, Operation

Management and staff fulfill their

stewardship responsibilities/

accountabilities.

Resource

Management

Ensure IT resources and infrastructure are appropriate.

Our People/Staff

Hire, motivate, develop, promote and retain quality staff.

Processes Maturity

Ensure process maturity level is appropriate for environment.

B1. In-house vs. contract considered

B2. Lease vs. Own Considered

B3. Build vs. Buy Considered

B4. Life Cycle Costs Considered

B5. Cost of Service Measures

C1. Process Mapping & Gap Assessment

C2. Risk Assessment & Management

C3. Quality Assurance Results

C4. Customer Satisfaction Survey Results C5. Service Levels Monitored & Reported

A1. Regular Meetings with ITSC

A2. Strategic Plan Approved

A3. Strategic Plan Updated Periodically

A4. Annual Business Plan Completed

A5. Executive Satisfaction Survey

IT Steering Committee

Ensure alignment of the IT function with corporate mission and goals.

Value-for-Money Scorecard for IT Services

Define

Objectives

Assess

Structure

Assess

Resources

Assess

Processes

Assess

Performance

Address

Issues

Business

Objectives/

Outcomes

IT Objectives/

Outcomes

IT

Performance

Measures

What Do You

Want to

Accomplish?

Strategic

Plan-Link

IT

Governance

Board

Risk

Management

Framework

Organization

Structure

Culture

People

(Capacity and

Capability

Technology

Education and

Development

Funding

Technology

Acquisition

[Procurement]

Technology

Management

Project

Management

Service Delivery

Standards

Benchmarking

Architecture

Link to

Objectives

Performance

Reporting

Information

Integrity

Compliance

[Laws and

Regulation]

Validity of

Measures

Economy

Improvements

Efficiency

Improvements

Effectiveness

Improvements

Reassess

Objectives [If

Needed]

Customer

Satisfaction

Governance?

Value For Money Review Approach

Risk

Define

Objectives

Identify

Risks

Analyze

Effect and

Cause

Determine

Significance

and

Likelihood

Risk Assessment/Control Design Process

Method for

Managing

Risk

Design

Control

System

Business

Objectives/

Outcomes

Performance

Measures

(KPI)

Risk Appetite

& Tolerance

(KRI)

What Do You

Want to

Accomplish?

Risk

Inventory

What Can

Go Wrong

to Prevent

Meeting

Objectives/

Outcomes?

Events

Potential

Harm (What

Might

Happen?)

Opportunity?

Why Does the

Risk Exist?

(Root Cause)

The Relative

Importance

Within the

Context it is

Being

Considered

(Impact)

A Probability

or Chance of

a Risk or

Event

Happening

Inherent Risk

Avoid Risk –

(Stay Out of

the Program

or Business)

Accept the

Risk (Take a

Chance)

Reduce to

Acceptable

Level

Transfer

(Insurance)

Controls

Mitigate Risk

Controls are

Cost Effective

If there is no

Risk, there is

no need for a

Control!!

Design to

Seize

Opportunity

Management - Develop Risk Mitigation Strategies either Strategically and/or Operationally

Internal Audit - Provide Control Advice to Clients

Useful Tools

Val IT

• Business/IT Partnership

• IT Investment Common

Language

• Supports Better

Investment Decisions

• Potential Cost Reductions

• Supports IT Enabled

Business Change

Management

Useful Tools

Risk IT

• Guiding Principle

Framework for Managing

Risk

• IT/Enterprise Risk

Integration

• Risk Common Language

• End to End IT Risk

Management – Tone from

the Top to Operations

Useful Tools

COBIT [Integrated with

Risk IT and Val IT]

• Improves IT Efficiency and

Effectiveness

• Better IT/Business

Integration

• Support Better Resource

Management

• Potentially Enable and

Maximizes the Business

Information Technology Supports the Enterprise in meeting Overall Business Goals and Priorities

Information Technology does not Exist for its Own Sake and for its Own Ends

Thoughts

Paul Wallis

Director - Internal Audit

Region of Peel

Brampton, Ontario

Canada

paul.wallis@peelregion.ca

Ron Foster

Auditor General

City of Oshawa

Oshawa, Ontario

Canada

RFoster@oshawa.ca