What is your PMI? A Model for Assessing the Maturity of Performance Management in Organizations
description
Transcript of What is your PMI? A Model for Assessing the Maturity of Performance Management in Organizations
What is your PMI?A Model for Assessing the Maturity of Performance Management in Organizations
Dr. Mika Aho
PMA 2012 Conference: From Strategy to Delivery
Fitzwilliam College, University of Cambridge, UK
11.07.2012
Presentation Outline
Rationale for Performance Management
Maturity Models
Background for the study
Performance Management Index (PM-Index, PMI)ComponentsMaturity LevelsKey CapabilitiesExample Scores and Results
Summary
Rationale for Performance Management (PM)
Central part of business management and the mechanisms that support decision-making
It broadly encompasses all those measures that an organization needs to take in order to achieve specific aims and objectives
The range of methods and procedures that an organisation uses to manage its personnel effectively
The actions people take are closely related to the organization’s overall strategy
Effective PM aims to integrate all these procedures, methods and measures into a single, organization-wide management system by leveraging modern ICT
Rationale for Performance Management(from Information and Knowledge point of view)
Organizations need reliable, accurate and timely data; these being the fundamental tools of performance management
Sometimes, the concept of PM means dealing only with the data for certain pre-defined standards and key performance indicators (KPI)
This perception of PM leaves the rest of the data and information in the organization untouched
To implement a really comprehensive PM strategy organizations should leverage their data by adding analytical functions to their traditional measurement and monitoring processes
The use of analytics helps organizations to proactively seek out and identify trends, and to create alternative scenarios for what might happen in the business
Performance Management and Information Technology
Information technology acts as an enabler and integrator to support the overall PM initiative in an organisation
It integrates all of an organisation’s processes, measures, information and intangible assets so that they support the different phases involved in implementing a strategy, e.g.
Visualize and monitor the strategy in a form of dashboards, reports, and analytical tools
Create rolling sales forecasts and use those results in the budgeting process
In mature organizations, the performance data is often stored in an organization-wide data warehouse
PM furnish the decision-makers with all the information they need to make really informed decisions
Background for Maturity Models
There is nothing new in the idea that the evolution of an organization, or any one part of it, can be described by using maturity models
For example the Capability Maturity Model (CMM), have already been in use for several years to demonstrate the evolution of different areas such as
Corporate Governance (Weill & Ross, 2004)
IT and Strategy Alignment (Luftman & Kempaiah, 2007)
Service Oriented Architectures (Perko, 2008)
Performance Management (Aho, 2009; 2011)
Maturity Models
Over the years, organizations have adopted models by which they can improve their own maturity and capability, and that of their procedures
Maturity = development from some initial stage to an improved, target stage
Capability = what an organization can do and is capable of doing
When moving from a stage to another, the target area, or the whole organization, improves in terms of its qualitative and quantitative capabilities over time
The higher the maturity level, the more efficiently the organization is performing
At the higher maturity levels any procedure is controlled, foreseeable and the targets that have been set for it are achieved
At the lower maturity levels, however, there might be considerable variation; the process might be unstable and there are risks associated with it
Maturity Models and Performance Management
PM and maturity models have not been studied much academically, although some attempts had been made in related areas, such as
business intelligence (e.g. Eckerson, 2007; Hostmann, 2006; van Roekel et al., 2009)
analytical competition (Davenport & Harris, 2010)
performance culture (Dresner, 2010)
information technology growth (Nolan, 1979)
IT-business alignment (Luftman & Kempaiah, 2007)
There are some drawbacks in the existing maturity models
They are very seldom based on empirical evidence
There are few indications as to why certain components and maturity levels were selected
They only concentrate on particular aspects of PM, such as business intelligence or analytical capability, but fail to examine the concept as a whole
How to create a Maturity Model for Performance Management?
In order to create a maturity model for performance management
1. One first needs to identify the components that comprise PM
2. Secondly, to define a set of levels that denotes the evolution of PM in organisations
3. Thirdly, to identify the key capabilities for an each component and maturity level combination
4. And finally, to build a metric that can be used to assess the PM maturity, and to present the results in a convenient and understandable way
1
2
3
4
”A Construct for Performance Management Maturity Assessment”
The study was conducted during 2009-2011
A Constructive Research Approach
Empirical evidence was collected from five large Finnish manufacturing companies
Methods: interviews, questionnaire, participant observation, literature study
Doctoral dissertation published in November 2011
Tampere University of Technology, Faculty of Business and Technology Management
Maturity levels
Overview of the model (PMI, PM-Index, Performance Management Index)
Questions
Calculation engine
Maturity Model Metric
User interface
Web-based toolMain components
Supporting components
Sub components
The focus of this paper
Main components
Information
Performance
Strategy and business
Technology
Business value
Decision support
Strategy and objectives
Data and information analysis
Performance management processes
Measurement and monitoring
Common structures and master data
Data and information quality
Information production, distribution and availability
Data warehouse architecture
Business and technology alignment
Application architecture
Sta
keho
lder
s
Org
aniz
ation
al c
ultu
re
Com
pete
ncie
s
Com
mun
icati
on
Man
agem
ent a
nd R
espo
nsib
ility
Methods and Tools
Scale and Scope
Inta
ngib
le
asse
ts
Sub components
Information
Performance
Strategy and business
Technology
Business value
Decision support
Strategy and objectives
Data and information analysis
Performance management processes
Measurement and monitoring
Common structures and master data
Data and information quality
Information production, distribution and availability
Data warehouse architecture
Business and technology alignment
Application architecture
Sta
keho
lder
s
Org
aniz
ation
al c
ultu
re
Com
pete
ncie
s
Com
mun
icati
on
Man
agem
ent a
nd R
espo
nsib
ility
Methods and Tools
Scale and Scope
Inta
ngib
le
asse
ts
Supporting components
Information
Performance
Strategy and business
Technology
Business value
Decision support
Strategy and objectives
Data and information analysis
Performance management processes
Measurement and monitoring
Common structures and master data
Data and information quality
Information production, distribution and availability
Data warehouse architecture
Business and technology alignment
Application architecture
Sta
keho
lder
s
Org
aniz
ation
al c
ultu
re
Com
pete
ncie
s
Com
mun
icati
on
Man
agem
ent a
nd R
espo
nsib
ility
Methods and Tools
Scale and Scope
Inta
ngib
le
asse
ts
Maturity Levels
In CMM PMI Maturity Level PMI Maturity Level Description5 – Optimized Level 5
Strategic toolPerformance management has become a strategic tool for management, and it is a central part of the organization’s control and management system. Performance management is also strongly connected to the different phases of strategy implementation in the organization.
4 – Managed Level 4Analytical business
Business becomes more analytical in key business areas. The metrics and scorecards are closely aligned to the organization’s strategy.
3 – Defined Level 3Fact-based decision-making
Decisions are more often made based on facts, rather than management instinct. The organization’s data is stored in a centralized data warehouse.
2 – Repeatable Level 2Understanding the value
The organization understands the value of performance management for its business. Organizational goals and objectives are defined.
1 – Initial Level 1Information silos
The solutions are local; there are no common standards, no shared resources or management. As a result the management does not get a clear and consistent picture of the organization as a whole.
Maturity Levels
In CMM PMI Maturity Level PMI Maturity Level Description5 – Optimized Level 5
Strategic toolPerformance management has become a strategic tool for management, and it is a central part of the organization’s control and management system. Performance management is also strongly connected to the different phases of strategy implementation in the organization.
4 – Managed Level 4Analytical business
Business becomes more analytical in key business areas. The metrics and scorecards are closely aligned to the organization’s strategy.
3 – Defined Level 3Fact-based decision-making
Decisions are more often made based on facts, rather than management instinct. The organization’s data is stored in a centralized data warehouse.
2 – Repeatable Level 2Understanding the value
The organization understands the value of performance management for its business. Organizational goals and objectives are defined.
1 – Initial Level 1Information silos
The solutions are local; there are no common standards, no shared resources or management. As a result the management does not get a clear and consistent picture of the organization as a whole.
Create understanding
Build foundation
Leverage data and information
Align with strategy process
Key Capabilities
LEVEL 1 Information silos
• Poor quality data• No defined measures• Temporary ad-hoc
solutions• Solutions made for
single business cases• Reports built on
operational information systems
LEVEL 2Understanding the
value
• Operations and development plan
• Data marts• Budgeting• Creation of strategic
goals• Financially focused
solution• Interactive reporting
systems• Formal PM method in
use
LEVEL 3Fact-based
decison-making
• Top management support
• Management dashboards
• Common business vocabulary and master data
• Strategy execution and monitoring
• Planning and forecasting
• Key Performance Indicators
• Data warehouses
LEVEL 4Analytical business
• Competence centre• Balanced metrics• Individual and external
stakeholder metrics• Strategy planning and
analysis• Rolling forecasts• Enterprise data
warehouse (EDW)
LEVEL 5Strategic tool
• High quality data• Strategy feedback and
refinement• Continuous strategic
planning• Strategy scorecard• External data sources• Integration between
architectures and processes
• Service-oriented architecture (SOA)
For a complete matrix, please visit http://pm-index.com/matrix/
3,22 3,53
2,61 2,91 3,07
0,0
1,0
2,0
3,0
4,0
5,0
Case 1 Case 2 Case 3 Case 4 Case 5
case companies
Descriptions for each maturity level and component combination
” At the third maturity level a common organizational culture starts to evolve. The management, in particular, begins to appreciate the benefits to be gained from performance management. People in the organization start to adapt fact-based decision-making, and their decisions and actions are based on their understanding of performance management systems. People are starting to understand better the factors that have an impact on the business.
The performance management initiative gets support from top management. The organization begins to form a performance management competence centre where specialists from both business and IT are joined together to meet the needs of the end users.
The organization has a common set of standards and vocabulary. The shared terms and metrics are standardized across the organization. From the technology point of view, one characteristic of this level is that there are data warehouses that have more actionable and better quality data… ”
Structured results
Structured results
Structured results
Structured results
Questionnaireresults
Questionnaireresults
Questionnaireresults
Questionnaireresults
Summary by Case company
Summary by Case company
Summary by Case company
Summary by Case company
Structured interview
(1)
Transcribed interview
Structured interview
(2)
Summary by Case company
Field journal
Other sources
Literature study
Analysis of the empirical evidenceStructured
interview (2)
Structured results
Questionnaireresults
Process for identifying the components
Interview (15)
Participant observation(~125 days)
Project plans, minutesof meetings, internal documentation etc.
Questionnaireand interviews
Process for identifying the components
Literature
Empiricalevidence
Methods:• Quantifying• Theming• Typing• Content analysis
Analytical unit (n=475) Grouped analytical units (n=56)
Category
• Timeliness of information helps decision-making
• SAP forces to go towards common vocabulary
• MDM system in place• Master data has accountable• Common business vocabulary
from ERP project• Data quality is challenging• Data producer is responsible
for its validity• …
• Common business vocabulary
• Master Data• Data quality
and validity
Data and information
Process for identifying the components, cont.
Data andInformation
Data input Data quality and validity
Information
Information distribution
and availability
Information production
Sourcesystem
Infrastructure
Process for identifying the maturity levels and the key capabilities
E.g. Brudan, 2009; Balanced Scorecard Institute, 2011; Moncla, 2004; Hostmann, 2007; Luftman & Kempaiah, 2007; Eckerson, 2007; Hagerty, 2006; Davenport & Harris, 2007; Davenport & Harris, 2010; Wettstein & Kueng, 2002; Curtis et al., 2009
Empirical evidence
N=374
N=385
N=759
Literature and existing maturity models in the area
Level 1 Level 2 Level 3 Level 4 …Technology Application
architectureSpreadmarts(Eckerson, 2007a)
Traditional back-office applications(Luftman & Kempaiah, 2007)
Spreadsheetapplications arefrequently used(Hostmann, 2007)
Data Marts(Eckerson, 2007a)
Applications aresiloed(Hostmann, 2007)
DataWarehouses(Eckerson, 2007a)
Hybridtechnologies(Hostmann, 2007)
Improvedreportingsystems (Brudan, 2009)
Organization-widedata warehouse(Eckerson, 2007a)
Focus on applications thatimprove business process for a long-term benefit(Luftman & Kempaiah, 2007)
…
… … … … …
Validating the Model
Building the maturity model
Building the metric Questionnaire Analysis
Maturity model validation Metric validation
Group interviews
Construct validation
PrestudyLiterature
ObservationsInterviews
Changes to the maturity model
Metric calibration
Maturity model validation• No statistical methods were used• Construct validity increased when the construct was
incrementally developed• The components and their suitability were examined based
on literature, direct observations and interviews• The selection of the components are justified well
Metric validation• Definition of the 283 statements• Questionnaire was sent to key employees in the case-
companies in four parts• Analysis of the data, selection of proper calculation model• Interview with the case companies• Later the index was further refined and calibrated
Validating the Construct, cont.The dimensions of a weak market test
One person Team or department
Strategicbusiness
unit
Divisionor country
Entireorganization
Regular usereplacing old
system(s)
Regular use in parallel with old
system(s)
Ad hocusage
Used once
The intensityof usage
The extent of usage
Weakmarket test
passed
Weakmarket testnot passed
Rejected before considering implementation
Rejected after considering implementation
Reject after unsuccessful implementation trial
Tried once but not actually used
Case companies
Case companiesLogica
Rongo
(Labro & Tuomela, 2003)
A web-based tool for assessing the PM maturity and to present the results
A web-based tool for assessing the PM maturity and to present the results
Statement 1 2 3 4 5 0
Performance Management solution supports strategy monitoring x
The performance metrics used in our organization are in balance x
Performance data is used to devise alternative scenarios x
The technological architecture and infrastructure are flexible x
Data and information are high quality x
Data quality problems are openly communicated in our organization x
Top management supports performance management initiative x
The owner of Performance Management comes from the business, not IT x
Through scorecards, the employees participate to strategy execution every day x
Example scores and results
1
2
3
4
5Strategy and business
Performance
Intangible assets
Information
TechnologyCommunication
Methods and tools
Scale and scope
Management andresponsibility 3,0
3,84,1
0
1
2
3
4
5
Business value Strategy and objectives Decision support
Strategy and business
2,53,0
3,43,7 3,8 3,9 4,1
0
1
2
3
4
5
Managementand
responsibility
Business value Methods andtools
Scale and scope Strategy andobjectives
Communication Decisionsupport
Strategy and business
Example scores and results
1
2
3
4
5Strategy and business
Performance
Intangible assets
Information
TechnologyCommunication
Methods and tools
Scale and scope
Management andresponsibility
1,7
3,2 3,4 3,7 3,9
0
1
2
3
4
5
Technology Information Intangibleassets
Performance Strategy andbusiness
Communication
Communication
Summary
The study
presents a novel model for assessing the maturity of PM
further extend PM research by providing a deeper understanding of the underlying components of PM
attempts to strengthen the existing theoretical framework in this field
increases the understanding of what is a relatively new area of PM research
In the model
the identified nine components are used to describe the key areas of PM
the five maturity levels identify the key characteristics of each development stage
In contrast to existing models, in which PM is usually seen only in conjunction with strategic management, this model examines PM particularly from an information and knowledge management point of view
For more information, please visitwww.pm-index.com
What is Your PMI?http://your.pm-index.com
(available in the autumn 2012)
Good at Finnish? Check out my dissertationhttp://bit.ly/PMDissertation