An Introduction to Story Metrics 1 How to get the Answers to your most important Questions.

An Introduction to Story Metrics

1An Introduction to Story Metrics

How to get the Answers to your most important Questions

Martin KlubeckStrategy & Planning Consultant Office of Information Technologies University of Notre [email protected]


An Introduction to Story Metrics 3

Agenda • Metrics Overview• Demonstrate the Process• Hands-on


What Is A Metric?

A Common Language• Data• Measure• Information


Roots

Branches

LimbsLeaves

Trunk

Measures

Metric

Data Information

The Question


A Complete Story


Anthologies May Be Better


Metrics… • Are question driven• Tell a complete story• Include:

• Data• Measures• Information• Other Metrics


Help Desk Example

Number of trouble callsNumber of opened casesNumber of closed casesNumber of employees Number of survey responses

Number of calls per hourNumber or cases closed by worker

Number of calls for each hour compared to number of workers on shift. Average length of time to close a case, grouped by type

Average customer satisfaction rating

Data

Information

Measures


Help Desk Example

Aug Sep Oct

Nov Dec Jan

Feb

Mar Apr May

Jun Jul

manning

Average number of open cases from

1999-2002

0

25

50

100

125

150

175

200

225

0

5

6

7

8

9

10

11

12

Explanation: The manning over the academic year was not in line with the number of trouble calls received – based on data collected over the last three years. Result: We’ve re-aligned our manning for the coming year to match the level of need each month.

manning

open cases

Max manning

Min manning

ME

TR

IC


Why Use Metrics?

To Help You …• Gain support from above• Provide visibility• Improve• Make better “data informed” decisions


How Not To Use Metrics

To “support my case*”To “motivate” the staffTo “manage” the staff or othersTo evaluate individual performance


How To Use Metrics • Explain how they will • Investigate• Share – Close the loop

and WON’T be used


Who Will Use The Metric? • Customer Community• Management• Owners and Workers• Leadership


Top Five Warning Signs that this training has failed

The boss says – “I’ll know it when I see it.”

“We’ve been collecting this data for five years and no one is using it.”

“Do we have any data on...?”“They don’t trust the data.”“Sounds interesting, let’s collect it.”5.

4.

3.

2.

1.


The 5 Whys

What Is The Root Question?


Drawing A Picture • Focus on “how it looks”• A picture is worth

a thousand words• Send a clear message


Metrics Tools

• Implementation Guide – adds rigor and structure• Answer Key – “cheat sheet” for developing metrics• Process Documentation – How to


The Complete Metric “Pallet” • Metric Name• Purpose• Metric Area/Category• Customer• Graphical Representation• Explanation• Metrics Analysis• Measures used to Develop Metric• Collection Schema • Schedule• Assumptions and Constraints• Related Metrics and Data Dependencies• Lessons Learned


The Primary Metric Colors • Purpose• Graphical Representation• Explanation• Metrics Analysis• Measures used to develop Metric


Info

rmat

ion

Nee

ds

Value to Organization

Managementof Resources

Effectiveness

Efficiency

Human Resources

Visibility

Delivery

Customer Satisfaction

Customer Service

Cost

Time

Quality

Training

Resource Allocation

Reward & Recognition

Employee Satisfaction

Work Environment

Project/program Status

Strat. Planning & Goal Attainment

Priority Setting

Communications

Customer view

Business view

Worker view

Management view


Hands-On Session


Designing your first metric


1. What is your root question?

2. Visualize the Answer

3. Identify the Measures

4. Find the data

Summary

Embrace the OpportunityStart with the Question

Draw the PictureTell a Complete Story

Repeat - Tweak - Delete


Q & A


Martin KlubeckStrategy & Planning Consultant,

Office of Information Technologies, University of Notre Dame

[email protected]


Core/Critical service availability (f ile / storage, enterprise applications, email, and netw ork services)

Target is >= 99.90%

99.90% 99.90% 99.89%

99.81%

99.76%

99.78%

99.80%

99.82%

99.84%

99.86%

99.88%

99.90%

99.92%

2005 2004 2003 2002

Ratio of Promoters to detractorsReponse to Customer Satisfaction Survey

1.1

0.1

0.1

0.1

0.1

0.1

0.1

0.3

-1.0 -0.5 0.0 0.5 1.0 1.5 2.0

The higher the ratio the better - a ratio of 1 (promoter = detractor) is the threshold

Students (3574)

Fourth year (660)

Fifth year (40)

Grad/Professional (718)

Third year (592)

Second year (787)

First year (772)

Faculty & Staff (1280)

Significant security incidents

0 2 4 6 8 10

2005

2004

2003

Threshold = 2, Target is less than 2

Customer Satisfaction"Would you recommend OIT to a friend/coworker?"

35%

66%

77%

88%

69% 70%

85%

53%

26%

25%11%

6%

23% 21%

10%

33%

39%

10% 11% 6% 8% 8% 5%14%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Detractors Neutral Promoters

Effectiveness

S A M P L E


Efficiency

4.94.5

3.3

4.9

2.2

2.93.1

2.8

38,50437,260

34,268 30,782

0

1

2

3

4

5

6

2005 2004 2003 2002

0

5,000

10,000

15,000

20,000

25,000

30,000

35,000

40,000

45,000

Overall average time to resolve Average time to resolve - Help Desk Number of overall cases

in days

Time To Resolve

Target = <= 5 days Target = <= 2.75 days

OIT Spending

$2,476$2,498$2,351$2,884

$6,813

$7,803 $6,928

$6,473

1%0%0%

-6%

23%

21%

2%

-7%

$0

$1,000

$2,000

$3,000

$4,000

$5,000

$6,000

$7,000

$8,000

$9,000

2005 2004 2003 2002

-10%

-5%

0%

5%

10%

15%

20%

25%

Per student Per faculty and staff Per Student Change Per faculty and staff change

OIT Expenditures (in Millions)

$668.7$626.3

$589.2

$20.8 $19.6 $21.1$7.7 $10.0 $5.8

4.60%

4.30%

4.70%

$0.0

$100.0

$200.0

$300.0

$400.0

$500.0

$600.0

$700.0

$800.0

2005 2004 2003

4.10%

4.20%

4.30%

4.40%

4.50%

4.60%

4.70%

4.80%

Actual ND Expenditures (Millions) Actual OIT Expenditures (Millions)

Renovare (ERP) expenditures (Millions) OIT & Renovare (ERP) Expenditures / University Expenditures

Client-Staffing Ratio

52.9

59.561.8

66.6

5.5 6.2 6.3 7.0

14.1 15.4 15.917.8

217

192183

166

0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

2005 2004 2003 2002

0

50

100

150

200

250

Students to OIT employees Faculty to OIT employees Staff to OIT employees Centralized

S A M P L E


An Introduction to Story Metrics 1 How to get the Answers to your most important Questions.

Documents

Transcript of An Introduction to Story Metrics 1 How to get the Answers to your most important Questions.