Design Your Own KPI Based Scorecard

KPI based Scorecard

Design your own

Wednesday September 23

2.00 PM – 6.00 PM

Mustang 6

Workshop overview

• This session is designed for professionals who operate in the field of IT

performance management and allows them to create a KPI based

scorecard using KPIs that actually make sense to the business

• The following are the objectives for this session:

– Understand the principles of successful IT Performance Management

– Are able to identify what KPIs are relevant to their business

– Learn to identify quality KPIs

– Using a Software as a Service environment can configure a scorecard and

dashboard

– Are able to interpret information in a dashboard and use this for effective

decision making

• This workshop focuses on the process for the creation of an IT KPI based

scorecard and will not go into the details of data collection, data quality or

data transformation

2

Workshop Agenda

3

Session Title Description Type Time

1 The current state

of IT Performance

management in

your organization

Introduction of the participants and

Discussion on maturity of IT Performance

Management in your organization

Assignment 2.00 – 2.10 PM

2 Current state of

the industry

An overview of the current state of the IT

Performance Management industry

Presentation and

Group Discussion

2.10 – 2.30PM

3 IT Performance

Management

An overview of the IT Performance

Management process, terminology and

concepts

Presentation 2.30 – 3.00 PM

4a How to Defining IT Performance management

objectives and identifying key processes

relevant to your organization

Presentation 3.00 – 3.45

4b Define / Measure Define requirements for IT Performance

Management: objectives, audience, scope

Individual Exercise 3.45 – 4.30 PM

5 Scorecard Design Create an IT KPI based Scorecard based

on the current state of your organization

and your requirements

Presentation and

Group Exercise

4.30 – 5.30 PM

6 Review of

Scorecards /

Dashboards

Analysis of a number of Scorecard and

Dashboard examples

Presentation and

Group Discussion

5.30 – 5.45 PM

7 Questions and

Answers

Q and A session and definition of next

steps

Group Discussion 5.45 – 6.00 PM

CURRENT STATE OF YOUR

ORGANIZATIONS‟ IT

PERFORMANCE

Session 1

The current state of IT Performance Management

in your organization

Agree

Agree to some extent

Disagree

Not sure

Q1) What is your opinion on the following statement:

“ For too many organizations IT is a black box. Projects and systems are so

complex that few CIOs can predict a direct impact on the business, making

it hard to win budget and resources even in prosperous times”

Source: Information Week 2008 - Hunting the Elusive IT Dashboard

http://www.informationweek.com/management/showArticle.jhtml?articleID=206900664



They don‟t have a clear picture

They could, but information is fragmented and dispersed over

tools and reports

They do, but a lot of effort goes into consolidating various

reports into one view

They have a holistic picture of their overall IT performance

thanks to one or few regular concise reports

There is no clear trend to be distinguished

Q2) To what extent does IT management in your organization / your

customers have a holistic picture on and decision making information

about the performance of IT?



Q3) How would you rate the maturity level of your organization (or that of

your customers) in terms of its IT performance management.

Level 0: IT Performance Management is non-existent.

Level 1: There is ad hoc monitoring in isolated areas.

Level 2: Some measures are set with a clear link to business

goals but are not communicated. Measurement processes

emerge, but are not consistently applied.

Level 3: Efficiency and effectiveness are measured and

communicated and linked to business goals and the IT

strategic plan. Continuous improvement is emerging.

Level 4: There is an integrated performance measurement

system linking IT performance to business goals by global

application of a documented framework. Continuous

improvement is a way of life.



Reduction of IT cost

Establish the progress toward achieving goals

Optimal resource allocation

Be compliant with internal or external regulations

Require an insight into performance against

service levels

Identify internal improvement opportunities

Other

Q4) What is the main driver for IT Performance Management?



Having support from the business

Proper guidance on the implementation

Having a formalized service management framework in

place (ITIL, COBIT)

Having a useful software application to capture and

convey performance measures

Having an industry standard for metrics (common

language)

Having the proper resources in place (budget, people,

infrastructure)

Other. Please specify:

Q5) What do you consider to be the key success factors for effective IT

performance management?



Business Intelligence (e.g. IBM, Hyperion, SAP)

Manual reporting (e.g. Excel)

Internally developed software solution

IT performance solution from external vendor (Metricus, M42 )

IT service management solutions (HP OpenView, BMC

Remedy…)

Service Level Management Monitoring Tools (Digital fuel,

Oblicore...)

Other. Please specify:

We don‟t use any tools to report IT performance

Q6) Which tools are you currently using to report IT performance?

CURRENT STATE OF IT

PERFORMANCE

MANAGEMENT

Session 2

IT Performance Management market maturity

12

“For too many organizations IT is a black box.

Projects and systems are so complex that few

CIOs can predict a direct impact on the business,

making it hard to win budget and resources even

in prosperous times.

And when the CIO can't get a clear picture of the

real-time data that underlies critical applications,

infrastructure, and projects, IT too often ends up

reacting to issues after users and customers are

having problems.” (Information Week March

2008)

Source: Information Week 2008 - Hunting the Elusive IT Dashboard

Source: “Trends in IT Performance Management,” 2008/9, ITpreneurs’

survey and interviews among 99 IT executives and consultants

65% of IT Managers agree with an InformationWeek statement that says IT managers

have no idea what is happening inside their IT organization (according to an ITpreneurs

survey)

http://www.informationweek.com/management/showArticle.jhtml?articleID=206900664


13

The majority of organizations are not reporting on IT Performance due to the time

and energy involved in doing this

Collecting data

from various sources

Converting data

into logical numbers

Building complex

Excel sheets

Highly Ineffective

0% 5% 10% 15% 20% 25% 30% 35% 40% 45%

They don't have a clear picture

They have a holistic picture of their overall IT performance thanks to one or few regular

concise reports

They do, but a lot of effort goes into consolidating various reports into one view

They could, but information is fragmented and dispersed over tools and reports

Percentage 2009 Percentage 2008

Source: “Trends in IT Performance Management,” 2008/9, ITpreneurs

Survey and interviews amongst IT executives and consultants


Level 0 IT Performance Management is non-existent.

Level 1 There is ad hoc monitoring in isolated areas.

Level 2 Some measures are set with a clear link to

business goals but are not communicated.

Measurement processes emerge, but are not

consistently applied.

Level 3 Efficiency and effectiveness are measured and

communicated and linked to business goals and

the IT strategic plan. Continuous improvement is

emerging.

Level 4 There is an integrated performance measurement

system linking IT performance to business goals by

global application of a documented framework.

Continuous improvement is a way of life.

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

Level 0 Level 1 Level 2 Level 3 Level 4

IT Performance Measurement is emerging and the basics are getting in place but

there is no proper communication as yet



Note: this is a self assessment

14


0% 20% 40% 60% 80%

Other

Be compliant with internal or external regulations

Identify internal improvement opportunities

Require an insight into performance against service levels

Optimal resource allocation

Establish the progress toward achieving goals

Reduction of IT cost


Survey and interviews IT executives and consultants

The reduction of cost and the creation of clarity on the performance of IT in relation

to business goals are the main drivers for IT Performance management initiatives

15


Organizations look at IT best practices for support in identifying metrics and realize

they need to have both resources in place and buy-in from the business



0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

Other

Proper guidance on the implementation

Having a useful software application to capture and convey performance measures

Having an industry standard for metrics (common language)

Having support from the business

Having the proper resources in place (budget, people, infrastructure)

Having a formalized service management framework in place (ITIL, COBIT)

percentage 2009 percentage 2008

16


Source: “Trends in IT Performance Management,” 2009, ITpreneurs

Survey and interviews IT executives and consultants

Manual reporting using Excel is favored in most organizations today followed

by the reporting capabilities provided by service management tools

0% 10% 20% 30% 40% 50% 60% 70% 80%

IT performance solution from external vendor (Metricus, M42 )

We don't use any tools to report IT performance

Other

Service Level Management Monitoring Tools (Digital fuel, Oblicore)

Business Intelligence (e.g. IBM, Hyperion, SAP)

Internally developed software solution

IT service management solutions (HP OpenView, BMC Remedy)

Manual reporting (e.g. Excel)

Percentage 2009 percentage 2008

17

IT PERFORMANCE

MANAGEMENT PROCESS

Session 3

18

The IT Performance Management Process

IT Performance Management is about measuring, improving, and

demonstrating the value of IT

“IT Performance Management is the effective combination of methods,

metrics, data, and tools that enables organizations to define KPIs that are

relevant to them, understand their current performance against

predetermined goals, and enables organizations to build on this information,

initiate improvement activities, and achieve optimal IT performance in line

with business requirements”

* Metricus definition for IT Performance Management

19


Performance measurement is the process of assessing progress toward achieving

predetermined goals*. Performance Management builds on that process, adding the relevant

communication and action on the progress achieved against these predetermined goals.

20

Measure Manage Improve

IT Performance Management is closely related to IT performance

measurement.

They are sometimes mistaken for each other. Strictly speaking, Performance

Management is the larger domain and includes performance measurement as a

component.

IT Performance Management

IT Performance Measurement

Define

* Wikipedia


21

Define

Measure

Manage

Improve

Define what is important

to you and what you

should measure

Measure your IT performance

through best practice KPIs

and performance data

Manage the ongoing process and

present decision making

information to relevant

stakeholders in your organization

Initiate improvement

activities based on historic

data and scenario planning

In this workshop we will touch on the first three phases of the IT Performance

Management lifecycle: define, measure and manage

What drives IT Performance Management

Just like other business departments, IT has to continuously improve and ensure alignment with

the business

Ultimately the only way for IT management to demonstrate value and control is by defining,

measuring and managing IT performance

A great idea, but the idea often gets stuck at not being able to successfully measure IT

performance and not being able to bring everything together into a view that allows the IT

management to take informed IT decisions.

What is not defined cannot be controlled.

What is not controlled cannot be measured

what is not measured cannot be managed or improved

“

”

22

What drives IT Performance Management

23

Business

Strategy

IT Strategy

IT Processes

IT ActivitiesIT Performance

Analysis

ITIL/COBIT

Process

Scorecards

CIO Dashboard /

IT process health

IT Balanced

Scorecards

drives

drives

drives

IT Performance

Metrics and

Measures

Business Strategy drives IT

strategy and results in

defining, managing and

optimizing IT processes and

activities

This needs to be balanced

against the availability of

quality data and processes

that can provide information

from the bottom up

Structured KPIs drive performance

management for various users

24

IT KPIs (that are aligned with the business strategy) play a role at each

decision-making level in the organization and enable the organization to

take informed IT decisions in line with business requirements

Business Leadership

IT management

IT departments

IT Balanced

Scorecard

IT process health

dashboards

Functional and

process scorecards

Timely,

accurate,

and

quality

IT metrics

and

measures

(KPIs)

Business

Strategy

IT

Strategy

Processes and

Activities

drives

drives

From Measures to Decisions

25

Measures KPIsOperational

Data

Dashboards

Scorecards

IT Balanced

ScorecardDecisions!

An Operational Data Source (ODS) represents data collected by a system or

process associated with the delivery and support of IT services.

A comprehensive understanding of ODS's within the context of the information ecosystem of an

organization is essential to the structured definition and development of IT Service Metrics.

e.g. FrontRange HEAT,

HP Service Manager,

ServiceCenter, CA

Unicenter, Service Plus,

Servicedesk, Numara

Footprint / Trackit, In

house applications, Open

Source applications

e.g. IBM Tivoli, HP

OpenView suite, CA

Unicenter, Nagios, Open

Source Applications

Enterprise Systems

Management

Decentralized,

disparate data related

to specific

users/processes

e.g. SAP, Oracle Financials

Enterprise Resource

Planning

Internal IT applications

developed specifically

for IT support/delivery

functions

Service Desk Manual Custom

ODS examples

Operational Data Source

26

IT

processes

and

activities

Data

WarehouseIT strategy

Data

Integration

KPI based decision

making information

Insight

Change

Data

integration

process

?

Data Management

Business objectives change, which has an impact on the requirements

for data. Thus keep in mind that this process is dynamic and requires

continual management

27

Measures are distinct sets of data derived from mathematical calculations.

Measures are quantifiable, for example, size, volume, or percentage,

and involve aggregation of data elements, for example, sum, average,

min, max, or count.

In and of themselves, measures may or may not be meaningful.

However, measures represent building blocks for the metrics

required to make business decisions.

Examples:

Calculation Measure

Sum (incidents) Volume of incidents

Sum (Incidents Resolved) where „Met Target‟ = „Y‟ Volume of incidents resolved within target resolution time

Average (Database Availability) % Database Availability

Sum (IT Resources) where „Trained‟ = „Y‟ Number of IT resources trained

Measures

28

Metrics consist of one or more measures combined with a mathematical

calculation and a standard presentation (format) for the output.

Volume of Incidents

Volume of Incidents Resolved

Within Target Resolution Time

Measures

Categorization

Priority = „Critical‟

Time

Daily

Dimensions

Metric

% Critical Incidents

Resolved

Within Target

division

facilitates

comparisonfunctional

categorization

Metrics are associated with two dimensions: a time dimension and a functional categorization

dimension.

Metrics are used in the quantitative and periodic assessment of a process that is to be

measured.

Metrics should be associated with targets that are set based on specific business objectives.

Metrics are associated with procedures to determine the measures required and procedures for

the interpretation of metric results.

Metrics

29

Acronyms that are used for „metrics‟

30

Impact related - Key Performance

Indicator (KPI), Critical Success Factor

(CSF), Outcome Measure, Performance

Indicator

Operational related – IT Service Level

Agreement – SLA, IT Operational Level

Agreement (OLA), and Service Level

Objective (SLO)

Term „Metric‟ often used interchangeably

with „Target‟, „Benchmark‟ and „Goal‟

Clear definitions are required in order to

facilitate the necessary communications,

and to set appropriate expectations, with

users.

IT Performance

Metric

CSFOutcome

MeasureKPI

OLO SLA SLO

Numerous terms and acronyms exists for labeling „metrics‟

Characteristics of good metrics

31

Metric Presentation

32

Concept Description

Dashboard • A dashboard is a graphical display of the status of a

selected set of key metrics

Scorecard • Scorecards are the consolidated tabular and graphical

display of sets of metrics related to particular business

functions.

Visualization • Visualization reporting solutions show relationships

between selected metrics and assist in performing

impact analysis

Interactive Data Analysis • Online interactive analysis of data providing „slice and

dice‟ and „drill-down‟, „drill-through‟ capabilities. Known

as OLAP (Online Analytical Processing).

Scheduled Reports • Individual reports related to specific data requirements.

Operational Reports • Reports available directly from operational systems

used to support processes

Monitoring • Near real-time display of detailed data related to the

delivery of a service.

Metric Presentation

33

Concept

Dashboard

Scorecard

Visualization

Interactive Data Analysis

Scheduled Reports

Operational Reports

MonitoringOperations

Executive

Managers

Functional

Managers

Process Managers

In theory various users in the organization have access to

information that is timely and relevant only to them

Metric Presentation

34

Concept

Dashboard

Scorecard

Visualization

Interactive Data Analysis

Scheduled Reports

Operational Reports

MonitoringOperations

Executive

Managers

Functional

Managers

Process Managers

In practice many IT managers want to have access to very

detailed reports. Also, scorecards or interactive data reports are

only sporadically used

“A dashboard is a visual

display of the most

important information

needed to achieve one or

more objectives;

consolidated and arranged

on a single screen so the

information can be

monitored at a glance”

Intelligent Enterprise, 2004

Green IT dashboard

Dashboards

35

Highly visual, data aggregated from

various sources

Provide a one glance view on the

current state of the

organization/process or activities.

Typically used by CIOs, IT

Management and business users.

„Scorecards‟ focus on the consolidated presentation of metrics and

presents an accurate view of the here and now compared with predefined

goals

Graphical representation of trends

and alignment with defined targets is

provided.

They are supported by static and

interactive reports, as well as

diagrammatic representations of

metric performance and linkages

between metrics.

Categorization of metrics within a

scorecard is typically related to

functional or process views.

Service Desk Scorecard

Metricus Scorecard from the Service Desk Module

Scorecards

36

DEFINE REQUIREMENTS FOR

IT PERFORMANCE

MANAGEMENT

Session 4

37

Method 1) GQM – goal, question, metrics

Source: L.Buglione & A.Abran © 2000

GQM is a technique to derive measures for project control starting from

high-level goals, passing through the decomposition in several questions

to answer.

Author: Victor Basili in the early „80s (NASA Goddard Space Flight Center)

Measurement object: software projects

Application: GQM is one of the most well-known and used measurement

approaches for establishing a measurement program

38

Method 1) GQM – goal, question, metrics

Process Governance

Complete Correct & Valid Detailed

Are mandatory

fields filled out

Are data entered according to process policies & standards

Do text fields contain

appropriate level of

detailsDoes the data reflect what actually happened

Process Effectiveness

Incident Problem Change

Reduce MTTR Decrease days to root cause determination

Decrease problems due to change

Decrease days to implement permanent fixes

Business Objectives

Increase Pace ofService Delivery

Improve ServiceAvailability & Stability

Deliver Projectson Time / on budget

… Reduce outage minutes

Reduce number of incidents

…

Reduce MTBF Decrease issues during change implementation

Using the Goal Driven Method

Source: David Chiu of BMO Financial 39

Method 2a) Balanced Scorecard

BSc is a multidimensional framework for “translating (organizational) strategyinto action” at all levels of an enterprise, by linking objectives, initiatives and measures to an organisation‟s strategy

Authors: Kaplan & Norton (HBS) in the early „90s, originating from a French methodcalled Tableau du Bord (turn of 20th century)

Main measurement object: whole organisation / a SBU

Application: BSc is one of the most relevant management practices of last 75 years, according Harvard Business Review Journal

Source: L.Buglione & A.Abran © 2000 40

Business strategy

IT strategy

Method 2b) Balanced Scorecard and COBIT

COBIT provides a bridge between business risks, control needs and technical by

providing a comprehensive framework for delivering value while managing risk

and control over data and information throughout the lifecycle.

Authors: ISACA in the „90s, originating from being an „auditors check list‟

Main measurement object: The entire IT organization: IT resources need to be managed by a set of naturally grouped processes. COBIT provides a framework that achieves this objective.

Source: COBIT 4.1, ISACA 42

Method 3) COBIT

Aligning business goals with IT goals using COBIT, example for DS5

Source: COBIT 4.1, ISACA 43

Assignment

Business Goals

IT Goals

IT process goals

IT KPIs

Step 1 define business goals

We are going to use the following model from COBIT to define a process based KPI

scorecard

IT

management

IT

departments

IT process

health

dashboards

Functional and

process

scorecards

drives

Step 2 define IT goals

Step 3 process

goals

Step

4

KPIsStep 5

Scorecard

44

Use the following template for step 1, 2, 3

45

Business goal

IT goals

Processes

45

Case description

At the Coffee Company, coffee is not merely a bean or a beverage.

It is the furthest thing from a product or a commodity. Coffee

involves relationships and responsibilities. It is a process involving

high standards and tough decisions. More than anything, coffee

has the connective force to enrich peoples‟ lives.

The Coffee Company is a gourmet coffee company, serving high-

quality coffee blends and a specialty selection of coffee by the cup.

Espresso and related food and beverage products complement the

offerings. Along with hot and cold liquid servings, the Coffee

Company offers gourmet beans in retail bags through a retail

channel as well as an online channel. All offerings are provided in an

atmosphere that whispers of the warmth and convenience of your

living room.

From 1991 to date, the Coffee Company experienced and realized

significant growth in size, revenue, and number of stores. From only

one store in downtown Toronto, the company has grown to 12,000

retail stores and 1 online store.

the current economic situation has made the Coffee Company

shift its focus from expansion to consolidating business with

its existing stores and focus on ensuring the uninterrupted

availability of its coffee products and services.

The coffee production process

Farming,

drying,

extractingRoasting Distribution

Retail

stores

Online

store

customers

After roasting, coffee is

packaged and sent to

one of our five regional

distribution centers in

Vancouver, Rotterdam,

Singapore, London and

Houston. The distribution

centers supply our retail

stores manage the

distribution from the

regional internet sales.

Our outlets are company

owned and franchise

locations worldwide. The

distribution centers

supply our stores twice a

week or more depending

on demand. Each

retailer is strongly

connected to the

community as well to

ensure delivery of fresh

products such as breads,

bagels, etc.

Coffee grows best in a

warm, humid climate

with a relatively stable

temperature of about

27C all year round.

The world's coffee

plantations are

therefore found in the

so-called coffee-belt

that straddles the

equator between the

tropics of Cancer and

Capricorn.

Un-roasted 'green' beans are

shipped to Rotterdam in the

Netherlands and Vancouver,

Canada from locations as

close as Hawaii and as far

away as Indonesia. Varietals

are then roasted or mixed

with other beans to create

blends, such as our popular

VOC or Da Vinci blends.

Fragmented information from IT

Retail store managers are getting

tired of constantly having to wait

more than 5 minutes on the

telephone with the service desk.

Before they get someone on the

line..They have a shop to run!

A change weekend is always

exciting for the IT organization..

Will there by any store with

problems with their online

systems or does everything go

well? In case of issues, the

challenge is to find out what went

wrong and how to revert. Testing

plans are not maintained

centrally and there is no review

of implemented changes

Our CMDB is not very

easy to use and as a

result its‟ not kept up to

date alwaysI am the incident manager, and I

am proud of closing incidents

within targets by my first level

support team

Change management is a

rush job, and our change

manager agrees to all

requests from the

business as they feel

everything is important.

So as a result they

don‟t call but bypass

the service desk and go

directly to the

specialists

It seems that we keep

reopening incidents as our

solutions don‟t seem to

work, what‟s going on. We

need a knowledge base..

CREATE AN IT KPI BASED

SCORECARD

Session 5

49

Tips and Tricks for selecting KPIs

CIO/Executive

ManagementConsulting

“Middlemen”IT Service

Transition Team

1. Show me how

well we are

managing issues

in IT

2a. Sure! Will

this scorecard

do?2b. We don‟t know

if it‟s possible but

that‟s not our

problem, we need

the sale.

3. We don‟t

have the

processes or

data in place…

There is a gap between theory and practice, and there are not that many metric practitioners out

there

What looks and sounds nice is often not practical and possible.

Don‟t be put in a situation with „middlemen.‟ Work direct with the users of KPIs to ensure that

what is requested is actually feasible.

When you are presented with a KPI wish list that looks too good to

be true, it probably is.

50


51

Use only use KPIs when you know the data is available and only use

KPIs if the results can be tied to higher level operational, tactical or

strategic objectives

The horse and the cart: Only use metrics when you

know the data is available.

Confirm with all stakeholders the actual need and

planned use of each of the reported metrics

The requirement for metrics should not drive the

requirement for data; process and strategy should.


52

Ensure that your „effort-to-insight ratio‟ remains positive

The cost for collecting data for a metric should weigh up against the insight that you are

retrieving from the metric.

The cost categories are data collection, Business Intelligence, and report development.

Costs are predominantly qualitative whilst benefits are both qualitative and quantitative.

Cost of

data collection

Insight

provided by

the KPI


53

Garbage in = Garbage out: Poor data quality in IT services negatively affects

the ability for informed decision making throughout the entire organization.

Data Quality

Quality of

IT Metrics

Low

Low High

High

IT metrics should be developed and

presented with the same rigor as financial

accounting metrics.

Consider: Integrity of data, consistency

with prior periods reporting, materiality

(Value of data must exceed cost of

reporting)


54

Focus on consolidating the data required to

support metrics and scorecards.

Don‟t assume „one approach fits all‟ for data

integration.

Build a 'data resource network,‟ including

developers, support and operations (DBAs),

and application SMEs.

Email

Word

Excel

PowerPoint

Face to Face

Telephone

Meetings

Memos/Letters

Help Desk

Monitoring

Systems

Unstructured Structured

Recognize that data integration will be the greatest challenge in developing

KPIs, scorecards, and supporting reports.


55

Less is more, and it is recommended to start with only a handful of KPIs

rather than going with dozens of them

KPIs are put in place to be used to manage and

too many KPIs make managing to them difficult

For every KPI processes need to be in place to

collect data for every KPI and do something

with the results of the KPI.

Start small, keep it simple, and build upon

achieved successes.

Recognize that metrics will change over time

based on the changing values required from the

metrics and the growing maturity of the

organization


56

Audience Rationale

Executive Senior

Management

• Implementing metrics and scorecards solutions is a strategic initiative that

requires C-level sponsorship (CIO, CEO).

• This is generally recognized as the #1 Critical Success Factor to incorporate

any Performance Management Initiative or implementing Business Intelligence

software

Identify Key People • The people that really matter to a process must always be part of it. This is

often a group so small that you can call the members out by name and count

them on one hand.

• Key people need to be identified for requirements gathering, ownership of

metrics and reports, assistance with developing metric/scorecard solutions,

assistance in architecting and configuring the infrastructure required, and

support for data collection and metrics presentation.

• Performance Management software solutions can provide guidance and

templates for scorecards and reports for specific users in your organization

Identify Metric Users • Ask users what they need!

• Collect key information for users.

• Map metrics, scorecards, and reports to users.

• Develop an authorization and application security model.

• Keep it simple! Start on the premise of allowing users to see all information

and selectively revoke rights where appropriate.

Involve the right people when you are designing your IT Performance

Management initiative

Metrics that are used need to be SMART:

Specific

Measurable

Action oriented

Realistic

Time bound

Now we have identified the Processes that have our key interest and that

are important for us when we want to meet the objectives of the business.

The next step is the identification of specific KPIs for the selected

processes.

First some guidelines around metrics:

Metric Requirements - SMART -

57

The definition of an IT Performance Metric needs to be structured in alignment

with defined attributes and valid measures

Good Example Poor Example

[% Calls Abandoned]. A fundamental Call

Center/Service Desk metric, this divides the

number of calls offered by the number of calls

abandoned. Both measures are readily available

and attributes such as time dimension,

categorization fields, targets, user base, and so on

are straightforward to obtain. (To be discussed

more in the section – „IT Performance Metrics –

Attributes.‟

Time used to resolve unavailable services.

What services are included? What is the definition of

unavailable? How is unavailable time equated with

effort to resolve? Is it linked through incidents? If so,

are the processes in place to record time?

This helps avoid the „100,000 feet‟ syndrome of documenting

„wish lists‟ of what sound like nice metrics to have, but in reality

are not meaningful, not possible, too costly, and so on.


58

IT Performance Metrics need to be quantitative, allowing users to actually

measure progress and performance against the IT Performance Metric .

The provision of an IT KPI should prompt the users to ask

questions.

Supporting data needs to be directly available and/or

mechanisms available to obtain support data.

Typically, quantitative metrics are supportable while qualitative

metrics are hard to support.


[SRs Open] is a valuable IT service metric

but needs to be supported by details of

open service requests.

% Projects with Predefined Benefit

is often based on after the fact

„guestimates‟ as to whether predefined

benefits were available. For the metric to

be of value, information on what predefined

project benefits need to be stored in

conjunction with baseline project details.


59

An IT Performance Metric should contain information that can be directly acted

upon.

This could involve questions being asked as to why a metric is a specific

value, providing information on achieving defined levels of service, or

automatically instigating remediation action within a particular IT service

process.


[% Service Requests Resolved On

Initial Contact] is often a metric contained

within a Service Level Agreement (SLA).

Failing to meet a defined contractual

target, say 85% will require action to be

taken in order to avoid possible penalties.

Incidents Created

While this is often interesting to look at,

particularly across a period of time, and it is

certainly easy to measure, it‟s hard to make

an educated decision based solely on the

results in isolation. This is actually a

measure that needs to be categorized with

IT performance reference data, such as

service or classification, in order to be of

value.


60

An IT Performance Metric should be realistic from a data perspective, that is, the

data associated with underlying measures in the calculation of the metric needs

to be available.

An IT Performance Metric also has to be justifiable from the initial and ongoing

costs. The effort in collecting the metric logically is lower than the value derived

from decisions related to the metric.


[% Incidents Escalated]

The volume of incidents and an indicator if

an incident was escalated should be

readily available from the operational data

source associated with Incident

Management.

% of Problems Recorded and Tracked

It is difficult to detect that a problem is not

recorded or tracked unless it actually has

been recorded and tracked!


61

An IT Performance Metric should be collected at regular time intervals, that is,

have an associated time dimension.

This may be hourly, weekly, monthly, quarterly, and so on.

Metrics without a time dimension should be referred to as milestones.

Good Example Milestones

[% Incidents Caused by Changes] is a

metric that should be analyzed over time

to ensure a downward trend. Data should

be available every day.

% of Data Elements Contained within

Enterprise Data Model. The creation of

an Enterprise Data Model is a largely

investigative activity aimed at

understanding what data elements exist.

Consequently, the denominator for this

metric, the number of data elements,

cannot be determined until an Enterprise

Data Model actually exists. As a result, this

should be an IT Performance Milestone –

„Enterprise Data Model Complete.‟

.


62

Assignment

Business Goals

IT Goals

IT process goals

IT KPIs

Step 1 define business goals

Based on identified processes, we are going to look at and select a number of IT KPIs

and create a scorecard for this:

IT

management

IT

departments

IT process

health

dashboards

Functional and

process

scorecards

drives

Step 2 define IT goals

Step 3 process

goals

Step

4

KPIsStep 5

Designing a

Scorecard63

Assignment

Step 1: identify the most relevant KPIs for

Change, Service desk / incident management

Process name:

Metric Importance (1-10) Rationale (why)

Step 2: write down which KPIs

are most relevant for your IT KPI

based scorecard and why

64

Note: keep the

„tips and tricks‟ in

mind!

Use the following template for step 4

Process name:

Metric Importance

(1-10)

Rationale (why)

65

% IT Costs - Travel Reduce travel cost by providing viable alternatives, such as Web conferences or VOIP and e-learning

education instead of classroom training.

Energy Cost per IT Facility Energy cost per IT facility gives insight into the energy cost per facility. Monitoring this information

periodically helps organizations in optimizing the energy consumption.

% Projects Related to

Business

Projects should not be created if they are not directly related to specific IT or business objectives. The

combination of [% Projects Related to IT] and [% Projects Related to Business] should be 100%.

% Inactive Application

Accounts

Inactive accounts represent overhead on security systems, both from a performance and an

administration perspective. Importantly, they make the task of auditing security more difficult due to

unnecessary overhead. If effective security management processes are in place, the number of

inactive accounts should be minimal.

% IT Resources - Contractors IT departments will almost always have a requirement for contractors. The level of contractors needs

to be balanced with the potential impact on overall IT resource moral as well as costs associated with

projects.

The % of IT costs that can be directly connected to business value drivers, for example, sales/services increase due to increased connectivity

This is a difficult metric to measure; however, the actual process of attempting to measure this metric will yield results through mapping IT costs to business value drivers.

The average cost per user license for a particular application and types of applications

It is common for an IT organization to purchase far more licenses than required. For example, it is estimated that 80% of Business Intelligence software licenses are not used. Reducing the average cost per user license is a combination of ensuring that the correct number of users are determined and that costs are negotiated efficiently with IT vendors.

A measure indicating how satisfied the business is with the allocation of costs from IT

This metric can be used to measure the effectiveness of the allocation of costs from IT to the business. While subjective in nature as the business will rarely be satisfied with IT costs, it can be used to gauge improvements in the distribution of costs.

The % of IT staff that has participated in IT-related training over the last 12 months.

It is important within any organization that resources receive the necessary training to ensure that they are aware of current knowledge and to assist with career development. This metric provides details on what % of IT personnel have received training over the previous 12 months. The level of training is also significantly proportional to effectiveness and the costs associated with IT support.

KPI Examples: Cost Control

% Availability SLAs Met The % of services delivered for which the requirements in the Service Level Agreement for the availability of

services is met

% CIs Monitored The % of configuration items that are actively monitored by an infrastructure monitoring tool. Only configuration

items that are candidates for monitoring i.e. infrastructure components such as servers, networks and databases

are included in this metric.

% Database Backup Failures The number of scheduled backups that failed in a selected time period

% Database Restoration

Failures

The % of database restoration attempts that failed during a selected time period.

% Databases Backup/Restore

Verification

The % of databases that are scheduled for backup for which backup testing and verification of restoration has

been performed

% Databases Scheduled for

Backup

The % of databases that required backup that have successfully been schedule for backup.

% Server Backup Failures The % of total server backups that failed during a selected time period. Failure is defined by the non completion

of a backup. This includes both scheduled backup's that did not start and backups that failed during the backup

process.

% Server Restoration Failures The % of server backup restoration attempts that failed during a selected time period

% Servers Backup/Restore

Verification

The % of total servers that have had a full backup and restoration tested and validated.

% Servers Scheduled for

Backup

The % of services in an IT infrastructure that are scheduled for regular backups. The nature of backups will vary

across servers based on the software and applications hosted on a server. This metric looks at all servers and

provides an enterprise IT perspective.

Average Training Days for IT

Resources

The average number of training days for IT resources over the previous 12 months

IT Continuity Plan Reviews Number of reviews per period of the IT continuity plan

KPI Examples: Ensure continuous service

Projects Completed on Time The number of projects completed that were completed within the target end date.

Projects Completed Within Budget The number of projects completed where the total actual costs were less than the total budgeted costs.

Projects Completed on Time and

Within Budget

The number of projects that were completed by the target completion date and had total actual costs less than

budgeted costs.

Projects Open with Milestones

Missed

The number of projects currently open where one or more milestones related to the project have not been met.

Projects Scheduled The number of projects at a given point in time that are scheduled to be started in the future. Scheduled projects

are those that have been created, subsequently approved, but not yet commenced.

Projects Open The number of projects that are currently open, that is, implementation or development has started but has not

been completed.

Projects Completed The number of projects marked as completed during a specified timeframe. The definition of completed is that

the intended functionality to be provided by the project has been accepted by the users of the functionality.

Projects Championed by Business The number of IT projects created were initiated from the business and that have funding and full support from

the business.

Projects Completed with

Documentation and Testing Plans

A measure of the number of projects that have appropriate documentation associated with implementation tasks,

such as testing and post-implementation tasks, such as support and training.

Projects Created The number of new projects created.

% Projects Related to Business Projects should not be created if they are not directly related to specific IT or business objectives. The

combination of [% Projects Related to IT] and [% Projects Related to Business] should be 100%.

Projects - Predefined Benefit The number of projects created that have specified defined tangible benefits to the business or IT if successfully

implemented.

KPI Examples: Project Management

% IT Processes Meeting QA

Objectives

The % of IT processes that meet the QQ objectives that were defined

% IT Processes Reviewed by QA The % of IT processes that have been reviewed by conducting a formal QA

% IT Resources - Quality Training The % of IT resources who have received IT Quality related training within the previous 12 months.

% Projects Meeting QA

Objectives

The % of IT projects that meet the QQ objectives that were defined

% Projects Receiving QA Review The % of IT projects that have been reviewed by conducting a formal QA

% Stakeholders Understanding IT

Policy

The % of stakeholders that understand the IT policy

KPI Examples: Manage Quality

% CIs with Operations

Documentation

The percentage of Configuration Items that have associated operations documentation

% Incidents Resulting from

Inadequate Documentation

The percentage of incidents that can be appointed to inadequacies in the documentation made available with

new or changed applications

% SRs Resulting from Inadequate

Training

The % of SRs created due to the requestor not having sufficient training in functionality related to an IT service

e.g. configuration of new email accounts in Outlook or accessing functionality within an ERP system or on the

corporate portal.

% SRs Resulting from Lack of

Documentation

The % of service requests created as a result of lack of documentation related to an IT service. For example, a

user calls the service desk because they cannot find information on the corporate portal on how to connect to the

instant messaging system.

% Training Attendance New

Applications

The % of stakeholders that attended training for a new application after release

Average Change Documentation

Update Time

Average time it takes for documentation to be updated after a change

Customer Satisfaction -

Training/Documentation

The average customer satisfaction score for IT Training/Documentation surveys sent in a selected period. This

is representative of the overall satisfaction and confident that the business has in IT's ability to delivery training

and IT related documentation required by the business to use IT services. A good range of values for

measurement is 0 to 10

Incidents Resulting from

Inadequate Documentation

The volume of incidents that can be appointed to inadequacies in the documentation made available with new or

changed applications

SRs Resulting from Inadequate

Training

The volume of Service Requests that can be appointed to inadequacies in the training that was presented to

stakeholders after release of the new application or after a change made to the applicatoin

KPI Examples: Enable Operation and use

Changes Created The number of requests for change (RFC's) created. In IT Service Support, a change is the addition,

modification, or removal of approved, supported or baseline hardware or software components. This can

include network, application, environment, and system components, or other IT components, including

documentation. All changes should relate to a configuration item.

Changes Implemented The % of critical priority changes implemented within an agreed target time.

Average Change implementation time The average length of time required to implement requests for change.

% Changes Implemented within Target - Critical The % of critical priority changes implemented within an agreed target time.

% Changes Implemented within Target - High The % of high priority changes implemented within an agreed target time.

% Changes Implemented within Target - Medium The % of medium priority changes implemented within an agreed target time.

Average Cost per Change Average cost per change

% changes failed The % of changes that failed during the implementation phase of Change Management.

% emergency changes The % of changes implemented that classified as emergency changes. Emergency changes are those

which require circumvention of routine change management processes due to the urgency of business

requirements and changes to the IT infrastructure. A comprehensive Change Management process will

include a process for handling Emergency Changes

% high risk changes The % of changes created where the risk of incidents occurring within the IT infrastructure is high. A

comprehensive Change Management process will include a process for handling High Risk Changes

% changes rejected The % of change requests that are analyzed by IT and subsequently rejected.

% changes due to CMBD issues The % of changes that were made as a result of incorrect information provided by the CMDB

% changes post implementation feedback The % changes that received feedback after completion of the implementatoin

% changes post implementation review The % of changes that were reviewed post implementatoin

% changes process compliance The % of changes that correctly followed the defined change management process

% changes audited The % of changes that were audited post implementation

% changes causing incidents The % of changes implemented that resulted in one or more incidents occurring.

% changes causing problems The % of changes implemented that resulted in one or more problems incurring

% changes implemented without a back out plan The % of changes that were implemented without a defined back out plan

KPI Examples: Change Management

% changes implemented without CI

testing

The % of changes that were implemented without proper CI testing

% changes implemented without testing The % of changes implemented that did not go through the formal testing phase incorporated within the

Change Management process.

% changes requiring scheduled outages The% of changes that require a service outage to be scheduled in order for implementation of the change to

occur.

% changes specified inaccurately The % of changes that were not specified correctly

% changes with incorrect data The % of changes that were closed but contained 1 or more incorrect data components e.g. wrong

categorization fields entered, incorrect history, timestamps entered incorrectly, missing solution or closure

description, etc.

% Changes without sign off The % of changes that were incorporated without formal sign off

KPI Examples: Change Management

% IT Level 1 Support with

Backup

An important IT metric that is often used as a benchmark between IT organizations. The lack of backup

personnel can directly effect the delivery and support of IT services, which in turn can effect the overall

operations of an organization. This is a particularly important metric for the Service Desk because of the

customer visibility involved.

% SRs Resolved on Initial

Contact

An important service desk metric as the ideal scenario is that service requests related to phone calls are

resolved during the first call, minimizing the impact on the customer and the amount of process required by IT

support. This metric is also a key driver of customer satisfaction because it is an 'interaction' metric - one

whereby the service desk is exposed to the customer.

IT Resources - Level 1 Support The number of IT resources classified as Level 1 Support. Level 1 Support typically refers to agents at the

Service Desk who are the first point of contact for service requests, incidents, and change requests from the

business. They will try to resolve 'how-to' questions, routine simple changes, such as password resets, and

ensure that more complex issues are assigned (dispatched) to the correct Level 2 and 3 resources.

% Calls Abandoned Provides insight into how efficient the Service Desk is at answering IT support telephone calls from customers. A

high abandonment rate may indicate inadequate staffing or inefficient processes in terms of the collection of

information from the customer. Also, an important metric in terms of customer satisfaction because while small

minorities of these abandoned calls are accidents or mistakes, the majority represent either dissatisfied

customers (those who hung up) or customers who are about to become dissatisfied (those whose calls were

terminated).

Calls Answered Measures the number of calls that were answered by the Service Desk. This is related to <Calls - Offered> and

<Calls - Abandoned> in that the difference between <Calls - Offered> and <Calls - Abandoned> are mostly the

calls that were answered (the exception are the calls that were terminated during the initial pre-recordings). So,

as the % of calls abandoned decreases, the % of calls answered should increase. Each call answered becomes

a candidate for a service request, incident, or change request to be created.

Call Answer Time A measure of how long it took to be answered from the time the customer was placed in a queue to the actual

answer time. The time placed in a queue is normally immediately after the automated menu messages.

Customer Satisfaction - Service

Operations

The results of survey responses for Service Delivery surveys sent.

Avg Daily Incidents Handled per

Service Desk Agent

Relates to the calls that Service Desks can handle and is an important metric relevant to the planning of Service

Desk resources.

Average Cost per Call Average cost per call to the Service Desk.

% IT Staff Turnover - Level 1

Support

High Service Desk staff turnover will have a negative effect on key service request resolution metrics, such as [%

SRs Resolved on Initial] and [% SRs Resolved within Target].

KPI Examples: Service Desk

% Incidents Misrouted 'Misrouted incidents add to the overall resolution time and, as such, impact the customer and add cost to IT operations. A high number and

% of misrouted service requests could indicate fundamental flaws and problems in the incident management process.

% Incidents Reopened Primarily an operational metric that provides information on the effectiveness of the Service Desk and IT Operations resolution process.

The ideal process is that an incident is resolved and then automatically closed within a set timeframe. If an issue recurs, a new incident is

created. Incidents should not be reopened after initial resolution if the issue logged has been appropriately actioned and resolved.

% Changes Causing Incidents If the change management process is functioning effectively and the CMDB is accurate, this metric should be very low because all changes

should occur during scheduled outage times. Note: When analyzing this metric, care needs to be taken as if the incident root-cause

analysis process is failing then this metric will be artificially low.

% Incidents Dispatched to Level 2 Costs associated with level 2 are typically higher than for level 1 due to the additional expertise required. As IT operations becomes more

efficient, this metric should fall, resulting in faster resolution times and reduced costs.

% Incidents Resolved by Known Errors Known error information assists in the incident resolution by providing an IT operations resource with answers to questions previously

asked. This assists in achieving target resolution rates and reducing overall costs. This is stated as an 'above target' metric, meaning that

while the overall volume of incidents and incidents resolved by known errors should reduce, but the % of incidents solved by known errors

should increase.

% Incidents Dispatched to Level 3 Costs associated with level 3 are typically higher than for level 1 and 2 because of the additional expertise required. As IT operations

become more efficient, this metric should fall, resulting in faster resolution times and reduced costs.

% Incidents Resolved by Workaround While workaround solutions play an important part in maintaining the overall stability of IT services delivery, this metric should decrease as

an IT environment matures.

% Incidents Dispatched Provides an indication of IT operations efficiency in processing incidents. Ideally, over time, the volume dispatched decreases as a % of the

total incidents created.

% Incidents Void Void incidents should be monitored to ensure that upward deviations from longer term averages do not occur. Such deviations may

represent data quality or data entry problems. They could also highlight issues with automated incident creation mechanisms.

Average Incident Dispatch to Own

Duration

This metric indicates how responsive IT support staff are at accepting ownership of incidents. Increasing trends can indicate issues with the

process of service request ownership. Hourly or daily spikes can indicate staff rostering issues. Over time, the [Incidents - Dispatch to Own

Duration] should decrease, but increase relative to the [Incidents - Create to Dispatch] duration. This will result in the [Incidents - Create to

Own] duration decreasing.

% Incidents Owned within Target -

Critical

Prompt ownership of critical priority incidents indicates efficient service support processes and increases the probability that incidents will

be resolved within the target time. Ownership of critical priority incidents within target has increased importance due to the customer

visibility into processes within IT, that is, because of the potential business of critical priority service requests, the customer will have high

expectations regarding ownership and subsequent resolution.

Incidents created The volume of incidents that are created

% incidents resolved by 1st

level The % of incidents that were resolved by the first level support team

% incidents resolved within target critical The % of critical incidents resolved within the defined target time. Also referred to as 'Resolution Met' or „TTR Met' (Time to Resolve Met).

A typical target time for resolution of critical priority incidents is 2 hours.

KPI Examples: Incident Management

% incidents resolved within target – high The % of high priority incidents resolved within the defined target time. Also referred to as 'Resolution Met' or

„TTR Met' (Time to Resolve Met). A typical target time for resolution of critical priority incidents is 2 hours.

% incidents resolved within target – low The % of low priority incidents that were owned within the defined target time. Also referred to as 'Ownership

Met‟, „TTO Met' (Time to Own Met), or 'Accept Met'. A typical target time for ownership of low incidents is 4

hours% incidents caused by changes The % of incidents caused by the implementation of a change. Refer [Changes - Created]

% incidents caused by CMDB issues The % of incidents that were caused by data errors within the configuration management database.

% incidents linked to testing errors The % of incidents that can be linked to errors made during testing

% incidents auto generated The % of incidents that were automatically generated by IT monitoring tools.

% incidents resulting from inadequate

documentation

The % of incidents that are the outcome of inadequate documentation

% incidents resolved by known errors The % of incidents that were resolved by access to information available regarding known errors. This

information is typically stored in a 'Known Error' or Knowledge Management database. The source of the

information should be Problem Management processes where known errors are identified.

KPI Examples: Incident Management

% Problems with Incorrect Data Shows the % of problems that were identified as having one or more invalid data components. Poor data quality

associated with operational IT systems adds significant cost to a business.

Problems Open - Missed Target

Resolution - Critical

Used to monitor the volume of critical problems that are open and have missed target resolution. Increasing

volumes highlight risk to the business and potential resource or IT infrastructure issues.

% Incidents Linked to Problems In theory, all incidents are caused by problems. But, in the practical context of implementing IT service

frameworks, not all incidents result in a problem record and resolution through the formal problem management

process. This metric provides an indication of the % of incidents that are linked to the resolution of a problem.

The goal of effective incident and problem management processes should be to increase the % over time while

decreasing the overall volume.

Average Problem Create to

Close Duration

Can be used over time to track the efficiency of IT operations in resolving and closing problems. Increasing

trends can indicate resource issues and instability of the IT infrastructure. However, care needs to be taken when

analyzing this measure because some problems can be open for extended periods of time due to difficulties in

capturing the data required for problem diagnosis

Problems Created Used to monitor the overall number of problems generated within an IT infrastructure. A high level of problems

indicates inherit instability and potential problems with all IT service processes as problems are minimized via

effective incident, configuration, and change management.

% Proactive Problems Used to monitor how many problems are initiated by IT operations in a pro-active manner, that is, they start a

new project because they believe that something could go wrong or they want to ensure that a given future

problem does not happen. The presence of proactive problems indicates a mature, efficient problem

management process.

Problems Open Used to help identify resource issues within IT operations and assist with analyzing the overall stability of the IT

infrastructure.

% Problems Completed within

Target - Critical

Provides an indication of the effectiveness of Problem Management processes. Can also be used as an indicator

of how many serious problem exists - those that require more time to resolve because of extensive analysis and

diagnosis requirements.

% Problems Root Cause

Identified

One of the fundamental goals of problem management is to identify the root cause of the related incidents and/or

service requests. Efforts can be made to rectify the root causes, and information can then subsequently be fed

into knowledge bases and known errors.

% Problems RFC Created Provides information on how many problems require requests for change in order to be closed. The

implementation of one of more changes associated with a problem can be a sign that tangible outcomes are

occurring because of problem management processes or that the IT infrastructure requires significant change to

correct issues. Given this, this metric needs to be analyzed with respect to the overall maturity of IT services

delivery.

Average Problems per Release Can be used to track the overall effectiveness with which IT implements releases. Problems related to releases

can indicate serious flaws in the Release Management process.

KPI Examples: Problem Management

ANALYSIS OF A NUMBER OF

SCORECARD AND

DASHBOARD EXAMPLES

Session 6

77

A number of example dashboards/scorecards for

review

View a number of examples from Metricus

78

CollaborateProvide comments with graphs, manage issues and documents

IT KPIs – 25 practical, pragmatic, usable KPIs

TrendingTrending information to analyze KPIs in detail

SettingsManage definition, set target, tolerance, data criteria

Metricus – IT Balanced Scorecard

79

Drill down into the detailsBenchmark different service desk sites across the world and drill down into the details of available data

IT KPIs – 25 practical, pragmatic, usable KPIs for managing your service desk operations across the world

DashboardsCall handling information, information on Service Requests and Incidents

Metricus – Service Desk

80

IT KPIs – Every Module includes at least 25 KPIs with information on the practical use, adoption and benefits of KPIs.

Analyze to the greatest extent possible Dynamic charts allow you to analyze data and create charts that are most relevant to you

Add your own KPIs

Simply add your own KPIs or modify KPIs to the Module and publish them to a dashboard

Metricus – Service Request Management

81

Comprehensive DashboardsA typical Metricus Dashboard Template that provides a comprehensive picture on your current IT performance and historical information. These two elements together allow you to take adequate decisions for the future

Six Sigma charts

For selected KPIs you have the ability to create Six Sigma control charts or histograms. This functionality helps you reduce defects in your processes

Metricus – Change Management

82

For Visio experts

Metricus allows you to create and upload Visio charts and link KPIs to Visio components. For every Module specific templates are available but you can also create your own

Manage KPI categories

For all areas in the IT balanced scorecard KPIs are available. We categorized them according to customer feedback, but you can create your own sub categories

Metricus – IT Balanced Scorecard

83

IT Performance Management is

strongly advocated in ITIL.

Measuring, Managing, and Improving ITIL

processes has its roots in the Deming Circle

that is advocated in ITIL. The Plan, Do,

Check, and Act (PDCA) Cycle requires to

define a baseline and continuously measure

progress against that baseline.

ITIL addresses the use and benefits of KPIs

and IT Performance Management in every

process, as well as having an entire lifecycle

phase fully dedicated to IT Performance

Management, Continual Service

Improvement (CSI).

ITIL Service Lifecycle © Crown Copyright 2007 Reproduced under license from OGC

IT Performance Management and best practices:

ITIL®

84

The 7-Step Improvement Process © Crown Copyright 2007 Reproduced under license from OGC

The 7-Step Improvement Process is the most important element of Continual

Service Improvement. This process follows the same logical steps as defined in the IT Performance Management lifecycle and

can help you define what is important to you, measure progress against defined goals, and effectively

utilize your resources to optimize your IT organization.


ITIL®

85

An improvement is not noted until measured

against a baseline

Justification for an ITIL project requires a

measured baseline and an improvement target

Without metrics, an ITIL project will soon lose

steam and eventually fail

A process is not implemented

until measured

ITIL Service Lifecycle © Crown Copyright 2007 Reproduced under license from OGC

Weaknesses

The ITIL publications are limited in the specification of

KPIs to be used.

There is not a lot of consistency in the definition of KPIs

and information on KPI utilization between the various

ITIL domains.

The Continual Service Improvement is intended to be

used throughout all phases of the service lifecycle,

however in reality is only considered late in most

implementation projects.

Strengths

ITIL is strong in terms of processes surrounding service

support. This has translated into broadly accepted sets of

KPIs to measure Incident Management, Change

Management, Problem Management, and so on.

ITIL emphasizes effective Configuration Management. An

accurate CMS is fundamental to the delivery of IT

Performance Management.


ITIL®

86

ISO/IEC 20000 provides an external

quality mark for IT Service

Management organizations

ISO/IEC 20000 promotes the adoption of an integrated process approach to effectively deliver managed services to meet business and customer requirements.

Adopting ISO/IEC 20000 formalizes the measurement component of IT processes for organizations because they have to demonstrate and attest control over IT processes.

This includes requirements such as “A process should be in place to identify, measure, report, and manage improvement activities,” and “Reports shall be produced to meet customer needs, including trend information, satisfaction analysis, and so on.”

Service Delivery Processes

Capacity Management

Service Continuity and

Availability Management

Service Level Management

Service Reporting

Information Security

Management

Budgeting and Accounting

for IT Services

Release Processes

Release Management

Resolution Processes

Incident Management

Problem Management

Relationship Processes

Business Relationship

Management

Supplier Management

Control Processes

Configuration Management

Change Management

Service Delivery Processes

Capacity Management

Service Continuity and

Availability Management

Service Level Management

Service Reporting

Information Security

Management

Budgeting and Accounting

for IT Services

Release Processes

Release Management

Resolution Processes

Incident Management

Problem Management

Relationship Processes

Business Relationship

Management

Supplier Management

Control Processes

Configuration Management

Change Management


ISO/IEC 20000

87

There are no KPIs in ISO/IEC 20000.

Even though ISO/IEC 20000 formalizes

many of the operational IT processes, it

does not provide guidance on good KPI

selection and adoption.

ISO/IEC 20000 formalizes the need for

well-defined and thought-through

process adoption.

There are many documentation

requirements in ISO/IEC 20000 and, as a

result, the chances of finding the data

required for effective IT Performance

Management are likely to be high.

A specific process called „service

reporting‟ dictates reporting

requirements.

Strengths Weaknesses


ISO/IEC 20000

88

Thank you

Contact details:Arjan Woertman

[email protected]

+31 (0)10 71 10 260insert photo

89

mailto:[email protected]

Design Your Own KPI Based Scorecard

Documents

Transcript of Design Your Own KPI Based Scorecard