The Rise of Analytical Performance Management
Transcript of The Rise of Analytical Performance Management
The Rise of Analytical Performance Management
Tom DavenportCFO Preconference10 June 2009
Thomas H. Davenport – Analytical Performance Management
Information and Performance Management
► Perhaps the most successful area of information management► The first area attacked by IT
► Unit of performance—currency—is clear for many organizations
► CIO often reports to CFO
► Still problematic in many ways► Information not attended to or acted upon
► Still “multiple versions of the truth”
► Too difficult for many to use effectively
► Not sufficiently analytical
2 | 2009 © All Rights Reserved.
Thomas H. Davenport – Analytical Performance Management
Paying Attention to Performance Information
3 | 2009 © All Rights Reserved.
Vol
ume
Time
Data, Information,Knowledge
Information ProcessingTechnology
AccessNetwork Bandwidth
Attention
► Attention: the most important resource in business
► A finite resource► A zero-sum game
► Attention is a two way street:
► Seekers of attention try to capture it
► Givers of attention try to allocate and preserve it
Thomas H. Davenport – Analytical Performance Management
How Attention Works
4 | 2009 © All Rights Reserved.
Awareness Attention Action Decision Action
Meaning/Context
BEHAVIORDATA
Human Attention
Thomas H. Davenport – Analytical Performance Management
Making Performance Information Attention-Getting
5 | 2009 © All Rights Reserved.
► Personalize information to the role, if not the individual
► Decide what information is really important to get across
► Know what information is really critical to the recipient’s strategy
► Embed information in compelling stories► Send out information (if you must) in
small chunks at regular frequencies► Show movement and trends► Measure information consumption
Thomas H. Davenport – Analytical Performance Management6 | 2009 © All Rights Reserved.
LinkingLinkingPerformance Information Performance Information and and DecisionsDecisions
6
InformationInformation DecisionsDecisions
Requires linkage to decision process &
behaviors• Daimler Truck
• GE Energy Finance• New York schools
Requires tight process/ system integration • Zurich Insurance
• JM Family
Structured HumanStructured Human
InformationInformation DecisionsDecisions
Requires information
infrastructure
• BlueCross BlueShield of Tennessee
• Wollongong• Miami schoolsLoosely-CoupledLoosely-Coupled
More decisions
AutomatedAutomatedInformationDecisions
Thomas H. Davenport – Analytical Performance Management7 | 2009 © All Rights Reserved.
Technologies Linking Technologies Linking Information and DecisionsInformation and Decisions
7
•Data warehouses
• Information Integration
•BI packages
•Specialized displays & dashboards
•Recommendations•Scorecards
• Scoring algorithms• Rules engines
• Workflow
InformationInformation DecisionsDecisions
Structured HumanStructured Human
InformationInformation DecisionsDecisions
Loosely-CoupledLoosely-CoupledMore decisions
AutomatedAutomatedInformationDecisions
Thomas H. Davenport – Analytical Performance Management8 | 2009 © All Rights Reserved.
LooselyLoosely--Coupled May Be Too Loose!Coupled May Be Too Loose!
► Requires lots of data sophistication and analytical skill (Aberdeen study—6 to 7%)
► How often do “ad hoc query” and “drill down” really happen?
► No way to assess value of information and tools if not tied to a specific decision
► “BI historically has been about dashboards and scorecards developed for specific uses. But that's changing. All of a sudden it's about integrated analytics within applications. The conversation is starting to shift to looking at information in the context of specific decisions and roles.” AMR Research analyst John Haggerty in InformationWeek, “Performance Management Links Strategy and Operations,” Nov. 22, 2009.
Thomas H. Davenport – Analytical Performance Management9 | 2009 © All Rights Reserved.
Ways to Make Decisions Ways to Make Decisions More StructuredMore Structured
9
Equations/ Scores
Rules
Forms/Displays/Scorecards
Customization by rolesProcesses/Lanes
Tests
Degree of Structure
Thomas H. Davenport – Analytical Performance Management10 | 2009 © All Rights Reserved.
The Organizational Context for Linking The Organizational Context for Linking Information and DecisionsInformation and Decisions
10
Decision mgmt. groupsBICC organizations
Information groups not in ITIT/business alignment on key issues
IT and business groups playing well together
Thomas H. Davenport – Analytical Performance Management
Nirvana in Performance Data Management
11 | 2009 © All Rights Reserved.
► Major performance-related information entities are defined commonly across the enterprise
► There is one version of the performance truth
► Most data quality problems have been addressed
► Data is easily accessible in a warehouse or mart
► Data that need to be private and secure are private and secure
► There are some unique performance metrics
Thomas H. Davenport – Analytical Performance Management
Two Key Business Management Roles for Data
12 | 2009 © All Rights Reserved.
Senior Management
Team
► Data governance:► Definitions► Commonality► Political models► Appointments
Senior Functional Managers
► Data stewardship► Key domains► Initial structures► Coalitions► Policing
Thomas H. Davenport – Analytical Performance Management
Data Stewardship at the Bank of Montreal
13 | 2009 © All Rights Reserved.
► Business definitions and standards► Consistent interpretation of
information and ability to integrate► Information quality
► Accuracy, consistency, timeliness, validity, completeness of information
► Information protection► Appropriate controls to address
security and privacy requirements► Information lifecycle
► Treatment of information from creation or collection to retention or disposal
All at the strategic,
operational, and tactical
levels
Thomas H. Davenport – Analytical Performance Management
Analytics in Performance Management
► Not very sophisticated thus far► Primarily standard reports, not analytics
► Scorecards considered state of the art
► Mostly financial performance indicators
► Consistent measures of non-financial metrics are a problem
► Some demand for non-financial and “intangibles” metrics from analysts and financial accounting bodies► Loyalty, brand, sustainability
► A few leading firms are beginning to explore more analytical territory
14 | 2009 © All Rights Reserved.
Thomas H. Davenport – Analytical Performance Management
Performance Management Analytics Nirvana
► We’d have predictions of future corporate performance, not reports on the past
► We’d know why the items on our scorecards were there
► We’d be able to confirm our strategies
► We’d know how and where to intervene if performance began to suffer
► We’d simulate and optimize resources to implement strategy
► We’d report externally on “measures that matter” to performance
15 | 2009 © All Rights Reserved.
16 | 2009 © All Rights Reserved.
Analytics + Reporting = Analytical Performance Management
AnalyticsWhat’s the best that can happen?
What will happen next?
What if these trends continue?
Why is this happening?
What actions are needed?
Where exactly is the problem?
How many, how often, where?
What happened?Com
petit
ive
Adv
anta
ge
Reporting
Decision Optimization
Predictive Analytics
Forecasting
Statistical models
Alerts
Query/drill down
Ad hoc reports
Standard reports
AnalyticalPerformanceManagement
Thomas H. Davenport – Analytical Performance Management
Thomas H. Davenport – Analytical Performance Management
Analytical Performance Management Stories“For every 5% improvement in customer retention this year, we will grow revenue by 1.1% next year.”
“If we grow our share of customer gaming budgets by 1%, our share price increases by $1.10.”
“For every 10th of a point increase in employee engagement, we increase operating income by $100,000.”
“Raising the conversion rate by 1% brings more than $35 million in sales and more than $15 million in operating profit.”
17 | 2009 © All Rights Reserved.
Thomas H. Davenport – Analytical Performance Management
What Do These Stories Have in Common?
► Bivariate relationships
► Intermediate or control variables not considered yet
► In service businesses
► Baby steps along the service profit chain
► Relate non-financial measures to financial
► Employ relatively new non-financial measures
► Require time-series or cross-sectional data
18 | 2009 © All Rights Reserved.
Thomas H. Davenport – Analytical Performance Management
Pick Your Favorite Variable
More common
Less common
Innovation metricsSustainability metrics
Customer metricsEmployee metrics
19 | 2009 © All Rights Reserved.
Thomas H. Davenport – Analytical Performance Management
Key Stages in Analytical Performance Management
Stage 5Incentives and Actions
Stage 4Analytical Model
Stage 3Strategy Map
Stage 2Balanced Scorecard
Stage 1Financial reports
Accurate, timelyfinancial reports
Scorecard with non-financial & financial measures
Logical relationships amongnon-financial & financial measures
Statistical relationships amongnon-financial and financial measures
Steps to align behavior with goals
20 | 2009 © All Rights Reserved.
Thomas H. Davenport – Analytical Performance Management
Steps Toward Analytical Performance Management
Define Intangibles
Common Information
Strategy Map
Unit Comparison
Quant Scorecard
Run and Refine
Display and report variables that matter to performance
Create and refine a statistical (path) model
One version of key metrics and
information
Evaluate and compare
business units on key metrics
Hypothesis of logical relations among variables
Agree on metrics for non-financial
variables
21 | 2009 © All Rights Reserved.
Thomas H. Davenport – Analytical Performance Management
The Toronto Dominion Story
► Strong focus on customer service post-merger
► Developed proprietary index and scorecard
► Factor analysis identifed four service factors
► Related branch service to branch profit
► Controlled for other branch-level variables
► Explained 19% of branch profitability variance
► Created new incentive plan based on service index
► Found that service increases only worked in the middle of the distribution
22 | 2009 © All Rights Reserved.
Thomas H. Davenport – Analytical Performance Management
The Store24 Story
► Management had “strategy map” hypothesis about entertaining service experience
► Metrics of entertainment by store created► Also employee skill levels
► Analysis by academics found that enter-tainment was negatively related to store profit
► Controlling for population, competition, etc.
► When skill levels were high, entertainment did work as a strategy; when they weren’t it didn’t
► After two years management discontinued the strategy, but a year later than necessary
23 | 2009 © All Rights Reserved.
Thomas H. Davenport – Analytical Performance Management24 | 2009 © All Rights Reserved.
When It Works
► When you have a relatively local operational unit (store, branch, business, etc.) with clear financial and non-financial metrics
► When you’re comparing like operations using cross-sectional data
► When there is a clear hypothesis about what drives your business results (e.g., the service/profit chain in retail, or uptime in a refinery)
► When there is a senior executive to lead the charge
► When there is a clear relation to action!
Thomas H. Davenport – Analytical Performance Management25 | 2009 © All Rights Reserved.
What Doesn’t Work
► One big model with all possible explanatory variables
► Aggregation to the level of the large, complex enterprise
► Gathering long time series (because businesses and metrics change)
► Total reliance on somebody else’s data
► Analytics before metrics, e.g., sustainability
Thomas H. Davenport – Analytical Performance Management26 | 2009 © All Rights Reserved.
What We Still Need
Data . . . . . . . . consensus on nonfinancial metricsEnterprise . . . . . . . . defining performance consistentlyLeadership . . . . . . . . . . . . …a clear leader/owner for EPMTargets . . . . . . . . . . . clarity on what drives the businessAnalysts . . . .analytical people in performance mgmt functions