Quality Management KPI Metric Determination.pdf

7/25/2019 Quality Management KPI Metric Determination.pdf

1/21

Quality Management KPI Metric Determination

February 2013


2/21

2011 SAP AG. All rights reserved. 2

Roadmap for KPI Identification (an example)

Replace manualprocedures

Automation in job scheduling

management

# manually planned or scheduled jobs

AutomationArea

Mission

Challenge

Metric

TargetReduce # manually planned or scheduled

jobs by 25% until end o f 2013.

Quality Scope Business Continuity

KPI


3/21


QM-BC: Areas for Improvement

Replace manual

procedures

Proactive

Support

Automation

Defined process

KPIs

Adjusted and

improved

documentation

Process Quality

Mainstream new

products /

technologies

Managed

Integration

Efficient

Operations

Reduced costs

of operationsArea

Mission

Increase IT

Value


4/21


QM-POI: Areas for Improvement

Solution is

maintained up to

date

Solution Transition

Quality

Changes are

analyzed for

impact

Tested business

processes

Master Release

Plan

custom

development

stability

Development

Management

Custom

development

standards

Custom

development

documentation

Authorizations

maintained

Security

Authorizations

reviewed

Managed access

(firewall)

Managed quality

gates

Project Control

Integrated

partners

Area

Mission

ICC

Updated footprint

Established

project

methodology

Communication


5/21


QM-BPI: Areas for Improvement

Business

deliverables

Continuous

Improvement

Business

feedback

Business

Alignment

Business signoff

Communication

Plan

Area

Mission

Innovation

Key Drivers

ICCQuality Awareness


6/21


QM-SIV: Areas for Improvement

Integration

Validation

Risk Mitigation

Risk Awareness

Documented

business

processes

Operations

readiness

Handover to

OperationsArea

MissionOperations

procedures

Operations

Planning

Operations

solution review

Operations skills


7/21


Standard Metrics for BC: Automation

Replace manual

procedures

Automation in job scheduling

management

Proactive

Support

Automation in test

management

Coverage of Business

Process Monitoring

Ratio of automated test cases vs.

overall test cases performed (a)

# EEM locations (a)

Ratio of business processes in

Business Process Monitor

ing (a)

Automation

Coverage of EEM

Area Miss ion Chal lenge Metric

Ratio of automated versus manually

scheduled jobs (a)

Coverage of business processes in

test cases (a)

Ratio of resource inolvment in job

scheduling (m)


8/21


Standard Metrics for BC: Process Quality

Defined process

KPIs

Process Maturity

Coverage of E2E standards (m)

Adjusted and

improved

documentation

Ratio of KPIs per IT process (m)

Frequency of adjustments of

process KPIs (m)

% test cases reviewed or changes

(after identification of affected

ones) (m)

Process Quality

Process Improvement

Change completeness - test case

adjustment with change request /

implementation and transport/ E2E

processes

% incidents with Priority 1/2 that are

caused by missing or wrong solution

documentation change or operations

handbook (m)

Change completeness Updated

Solution Documentation and

Operation Handbook


# of Process maturity assessments

(m)

Ratio of IT processes with

KPIs (m)


9/21


Standard Metrics for BC: Managed Integration

Mainstream newproducts /

technologies

New product incidents# incidents with priority 1/2 after go-live

of new products / technologies within

first 3 months (m)

Coverage of product trainings

sessions for end users (m)

Managed

Integration

New product skills assessed

New organizations reviewed# maturity assessments performed

(m)


Ratio of incidents identified as

training issues (m)

# up-to-date skill reviews (m)

Coverage of product trainings

sessions for IT staff (m)

New tool integration# maturity assessments performed

(m)


10/21


Standard Metrics for BC: Reduced Costs of Operations

Efficient Operations

Incident effort reductionTime for incident resolution (priority

1/2) (a)

Reduced costs

of operations

Incident solving experts

# experts required to solve critical

incidents within customersorganization (m)


IT Value Reduce TCO % TCO vs. revenue (m)

Increase transparency of IT

Value

Trend in escalation / Days

with an outage (m)


11/21


Standard Metrics for POI: Solut ion Transition Quality

Solution is maintained

up to dateSolution Maintenance

distance from your current release, EhP,

and switch (m)

Solution Transition

Quality

Changes are

analyzed for impactChange impact analysis

# change impact analysis projects in SAP

Solution Manager (a)

.

Tested business

processesBusiness processes tested

# critical business processes steps tested

for performance regularly (m)

Master Release Plan Master Release Planfrequency

# of MRP updates documented per annum

(m)

MRP update documented per SAP Release

Strategy update (m)



12/21


Standard Metrics for POI: Development Management

Custom development

stability

Upgrade change impact

analysis Ratio of completed analysis projects inUpgrade Change Impact Analysis (m)

Development

Management

Custom code clearingRatio of completed analysis projects in

Custom Code Clearing (m)

Custom development

standards

Custom development

standards

Level of clear development standards

documented (m)

Custom development

documentation

Custom development

documentation

% custom developments documented

according to SAP Standard (Sol.

Documentation for Custom

Development) (m)


# development projects without review

of development standard compliance

(m)

% of custom code objects reviewed

incl. documentation standard (m)

% of custom code objects not

compliant to documentation standard

(m)

ICC

ratio of incidents caused by custom

development (a)

% of incidents (prio 1/2 ) caused by

custom developments (m)

Reduce modification

% of reduction of unnecessary

modifications (m)


13/21


Standard Metrics for POI: Security

Authorizations

maintained

Profile changesRatio of profile changes for users in

one year (m)

Security

Security notes

Distance to implemented security note

from released security note (m)

Authorizations

reviewedSecurity notes

% security reviews vs new releases

(m)

Managed access

(firewall)Infrastructure assessment

# infrastructure assessment per annum

(m)


# incidents caused by not implemented

security notes (m)

# reviews of assigned roles vs. actual

requirements (m)

# alignment meetings (m)Communication Alignment Meetings

Updated Standards/PoliciesUpdated FootprintRatio of changes in standards/policies

leading to IT reviews (m)


14/21


Standard Metrics for POI: Project Control

Managed quality

gates

Quality Gates Defined% projects with quality gates defined

(m)

Project Control

Quality Gate Criteria# of criteria defined for each quality

gate (m)

Integrated partnersRisk Management of

Partners

% of quality gates passed by 1st

attempt (m)


# of quality gates passed by mgmt.decision (m)

# incidents caused by ignored quality

gates (m)

Established ProjectMethodology

Project Methodologyaligned

% projects executed according toproject methodology (m)

Alignment meetings/ Participation in

risk mitigation reviews


15/21


Standard Metrics for BPI: Continuous Improvement

Business deliverables

Usability - Enhancements# usability enhancements resulting

from Business Feedback (m)

Continuous

Improvement

Usability - Problems # review of usability with users (m)

Functionality - Assessments # SMA per major release (m)

Functionality - Prototyping


Trend: # prototypes

Ratio of prototypes built in sandbox

systems to implemented ones in

Business (m)


16/21


Standard Metrics for BPI: Continuous Improvement

Quality Awareness

Establish Quality Roles% Completeness of Quality Challenges

(m)

Continuous

Improvement

Execution of Quality Plan Measurement frequency of KPIs (m)

Continuous Learning

# Lessons Learned performed with

Business (m)


# Success Stories with Businesscommunicated in the organization (m)


17/21


Standard Metrics for BPI: Business Alignement

Business feedback

Service Catalog

# offered (service catalog) IT-services

Business

Alignement

% usage of offered services (m)

Alignment meetings

# alignment meetings between Business & IT (m)

# action items from alignment meetings between

Business & IT (m)

Business signoff

Business testing % test cases performed by Business (m)

Business change

acceptance

% change requests confirmed/accepted by

Business (m)

Communication

PlanInformed business # newsletters /events o inform business users (m)

average duration until completion of action items

(m)



18/21


Standard Metrics for BPI: Innovation

Key Drivers

Speed to Respond to

market opportunityInnovation

Time for completion of Rapid

prototyping (m)

Business Feedback rating (m)

ICCReduce modification

% of reduction of unnecessary

modifications (m)

# projects resulting from Rapid

prototyping (m)


19/21


Standard Metrics for SIV: Risk Mitigation

Integration Validation

Integration test without

critical issues

# of new issues of type

functional/technical identified during

integration test phase (m)

Risk Mitigation

Volume/load test without

critical issues# of issues of type ** identified during

performance test phase (m)

Reduced duration of

stabilization period

Days after go live untilstabilization/hypercare is no longer

required (m)

Documented

business processes

Completeness of solution

documentation

Stabilization by add.

Resources/incidents or end user (m)


% critical business processes

documented

Minimize costs for

stabilization

Risk Awareness

Risk tracking Frequency of risk reviews (m)


20/21


Standard Metrics for SIV: Handover to Operations

Operations

readiness

Completeness of solution

operations documentation% of changes of operation handbook after

GoLive

Handover to

Operations


% of training schedules performed

Sufficient skills

SignOff % of completion SignOffs by Business

Monitoring procedures

establishedCoverage of automated monitoring for critical

business processes


21/21


Standard Metrics for SIV: Operations Planning

Operations

procedures

Business Process

Monitoring # business processes for which monitoringobjects have been defined and instrumented

(m)

Operations

Planning

Job Scheduling# jobs for which monitoring objects have

been defined and instrumented (m)

Operations solution

reviewOperations review

# operations reviews for solutions to be

implemented (SMA approach) (m)

Operations skills Operations trainings

# operations handbook training / skills

transfer for new topics (m)


Ratio of business processes in

Business Process Monitoring (m)

% of employees with proven skills (m)

Quality Management KPI Metric Determination.pdf

Documents

Transcript of Quality Management KPI Metric Determination.pdf