#GMSQL

Why Won’t Managers Use My Data?Or: An Invitation to Become a

Decision EngineerDr. Lorien Pratt, Chief Scientist, Quantellia

Mark Zangari, CEO, Quantellia

About Me

• Based in Denver• Former college professor• Research focus: applied analytics/neural networks• Wrote Learning to Learn and a lot of articles• Ran market analyst team with Frost and Sullivan• Co-founded Quantellia in 2008• Chief Scientist

• US Government spending• Community Justice Advisors analysis / Liberia

Agenda

1. Decision Engineering: Research showing the importance of this need

2. Research results for what’s needed to fill this need

3. How to do it: key steps

Global research study:Q: What is the biggest

problem that technology should be solving, that it

is not?

Global research study:Q: What is the biggest

problem that technology should be solving, that it

is not?A: Decision making

Where all this great data could

be used

Where the data is actually

used

Strong Demand for Better Use of Data

"We use predictive analytics in the following areas"

Strongly Agree

Agree

Neutral

Disagree

Strongly Disagree

0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0%

"Better use of our data and analytics could produce substantially more value (cost savings

and/or revenue growth) than it does today"

Ineffective Navigation Structure the Norm

"We use predictive analytics in the following areas"

Strongly agree

Agree

Neutral

Disagree

Strongly disagree

0% 5% 10% 15% 20% 25% 30% 35%

"We have an effective business navigation structure in place, where we make decisions,

monitor their outcomes, then adjust decisions as needed to achieve our business goals"

Human Resources2%

Environment2%

Pharmaceuticals2%

Financial Services2%

Nonprofit3%

Manufacturing3%

Defense7%

Public Health7%

Media10%

Information Technology

11%

Telecom-munications

52%

Source: Quantellia (2008) Number of samples = 61

Market Research

Decision Making

Source: Quantellia (2008). N = 28

Approximately 86% of organizations do not consistently follow a formal methodology for ensuring sound decisions.

We have a formal

methodology and we gener-

ally follow it14%

We have a formal

methodology but we do not adhere to it

very closely or consistently

29%

We follow an in-formal "rule of

thumb" methodology

32%

All decisions are made in an ad

hoc manner25%

How carefully do organizations make decisions today?

So why don’t managers use my data?

Because their most essential needs aren’t

met

Wanted: Decision Engineers

This can be you.

What is Difficult in Your Organization About Making Decisions?

Source: Quantellia (2008) N= 61

Decision making problems involve many business factors: especially communication, collaboration, and visualization

Mine Unstructured Data Sources

Include domain expertise

Common Methodology for Visualization

Sensitivity Analysis

KPI Identification / Dashboard

Integrate with Excel

Model Building Wizard

Handle uncertainty, e.g. by visualizing confidence levels

High powered quantitative engine

Multiple bottom lines / objective functions

User Friendly

Need for decision maker to tweak models themselves

Templates / pre-canned models and/or data

Social / Value Network Visibility

Iterative Methodology

Organize information / Help with overload

Need to represent intangibles

Qualitative plus quantitative data together

0% 2% 4% 6% 8% 10% 12% 14% 16%What features would be most valuable in software that supports decision making?

Source: Quantellia (2008) N = 61

Decision makers have many needs that are not met by current decision support systems

Systematic Decision Making Problems• “We focus on only one measure, when there are really

multiple objectives.”• “We make decisions that assume a predictable

unchanging future.”• “Our focus is on short-term goals,

ignoring long-term ones.”• “We are unable to reason about long

cause-and-effect chains.”• “We ignore intangibles like morale, reputation, trust, and

brand.• “We plan for only a single future scenario when

radically different courses of action may be appropriate, depending on how the future unfolds.”

Revenue Communit

y Service

Cost

“Five years from now, the market for our product will have grown by 30%”

“I can barely plan for next quarter, how can I think about the future, too?”

Reduce Time We Spend on Customer Care Telephone Calls

Lower Customer Care Costs

Improved Contribution Margin

Unhappier Customers

Reduced Knowledge of our Customers

Greater Customer ChurnWorse Contribution Margin

Brand

GAP

Decision Makers

Decision Makers

What will be the impact of today’s

decision, tomorrow?

“What price should I charge for this product?”

“Is my money better spent on more servers or more iPads?”

“Which buildings should I transform to cloud/VOIP first, to

maximize business benefit?

How can I design a new democracy to meet the health and legal needs of rural

populations, given limited funds?

What price should I charge for my new mobile service?

Data

Data

What will be the impact of today’s

decision, tomorrow?

Decision Makers

Q: So how can I get my data more widely used?

Q: So how can I get my data more widely used?

A: Realize that a decision (like software) can be engineered, and apply

engineering principles to its creation and

management

What have we done in the past when the complexity of a problem eventually exceeded our ability to manage it?

Analogies from History

• Small structures require little planning, commit few resources, and have relatively few consequences if they fail.

• As we try to build larger structures, we need more is needed.

• There is a ceiling beyond which the complexity becomes too great.

• An engineering discipline provides the organizational and communications tools that enable much larger structures to be reliably erected.

Example: Construction.

Decision making has reached its own complexity ceiling…

To overcome the complexity ceiling, we need to create a structured paradigm for decision making…

We need Decision Engineering.

Previous times we’ve introduced visual engineering approaches

Manufacturing

Incr

easi

ng v

isu

aliz

ati

on /

inte

ract

ivit

y

over

tim

e

Software Decision Making

“[It is essential] to visualize not just the data used to support decisions, but also the decisions themselves. [This is an] essential need in both the commercial and nonprofit worlds.”

-Lynn Langit, Developer Evangelist at Microsoft and author of the book Smart Business Intelligence Solutions with SQL Server 2008

Quantellia: Winner of the 2009 Microsoft Windows 7

Innovation Award

"In an age of global complexity, the time for making decisions is ever-shrinking, and the cost of bad choices too great to tolerate.  Quantellia created a tool for making the right decisions in this environment.”

-Guy Pfeffermann, former Chief Economist of the International Finance Corporation (World Bank); Founder and

CEO of the Global Business School Network (www.gbsnonline.org).

“Telecommunications companies, along with other businesses challenged by the rapid pace of a global environment, recognize the competitive value of applying Business Intelligence and analytic tools to the vast stores of data they generate.  Visual, actionable decision engineering solutions are the next evolutionary step in BI, to help get at what decision makers need and how they think, rather than on what data managers can provide.” - Susan McNeice, Vice President - Software

Research, Yankee Group (www.yankeegroup.com)).

“Anyone facing complex decisions with many participants and stakeholders, mounds of data, and limited resources to address the decision-making process, should look closer at visualization tools …  Visualized decision support—decision engineering—is fast becoming a key part of effective business management.”

-Karl Whitelock, Director Strategy – OSS/BSS, Stratecast, a Division of Frost and Sullivan (www.frost.com ).

What does all this mean in practice? Some keys

To make the best use of data, you have to start by setting all the

data aside.Really.

Time for a blueprint for decisions

Key Elements of a Decision ModelDecision Levers

Outcome #1

Outcome #2

Outcome #3

External Factors

Decision Levers

Decision levers: Factors over which we have control.Examples:• Price of a product• Features of a

product• Investment in sales• Investment in

marketing• Investment in OSS

External Factors: impact the outcome but over which we have no control Examples: • Competitor price• Market demand

Intermediate Values

Intermediate Values: Facts and values that are calculated along the way to determining outcomesExamples: sales volume, mean time to respond, sales expertise level, fallout rate

f

f

f

f

Outcomes: Measures of successExamples: Margin, Brand, Share Price

Dependencies: how one part of the model depends upon another, through cause-and-effect or other flows. Examples:How does MTTR respond to investment in CSR training?How does brand respond to sales staff expertise level?Note: these can be determined through traditional analytics, staff expertise, or industry benchmarks

Goals: targets against outcomes. Example: 5% margin growth in 2 years.

f

f

f

Data

Analytics

Analytics

Analytics

Predictive

analytics

Understand time

Understand how feedback loops end up dominating many

systems

Proprietary and Confidential Not for Reproduction Without Permission of Quantellia Copyright © 2010, 2011 Quantellia Inc All rights reserved.

Demonstration #1: Carbon Tax

Understand that Situational Data + Decisions + Time = Outcomes

Use Human Intelligence (especially when data is

imperfect)

Planning Phase

Implementation Phase

Objectives

Specification

Design

Qu

ality Assu

rance

Alignment

Execution & Monitoring

ChangeManagement

Secu

rity

Apply best practices of the engineering lifecycle

Beware the Whack-a-Mole

“When I lower costs in one part of my business, it ends up creating bigger problems in another.”

My decision is only as good as the data that

supports it

My decision is only as good as the data that

supports it

Not

How: Since only 10% of the data impacts 90% of

the decision, problems with the 90% matter much less. Know which is which

Use sampling / statistical to extract excellent analytics from messy data

Use human expertise when data is imperfect

Good Decisions from Imperfect Data

Start with the decision maker, not the data

Follow the decision value chain / connect the dots

Customer experience investment

Improvement to a KPI

Improvement to brand

Changes to demand

curve: sell same product

at a higher price

More revenue for the same

cost

Keep asking why

Understand time

Demonstration #2: Blue Jeans

Decision Engineering

• Like automobile design• Key competency: being

able to understand how the system will work

• Key competency: using judgment where data is missing

• Like monitoring a working vehicle

• Key competency: detecting problems accurately and quickly

• Key competency: diagnosis

Operational Monitoring

vs.

Data Is a key element, because Situational Data + Decisions +

Time = Outcomes

Decision Engineering is the Next Generation of Business Intelligence

Decision EngineeringPredictive

AnalyticsReporting/Business

Intelligence

Data Management

Wanted: Decision Engineers.

An invitation: change the world.

(or, just do the next cool thing)

THANK YOU.Lorien.pratt@quantellia.com

303 589 7476@LorienPratt

Please fill out the evaluation and turn it in to this session’s host.

#GMSQL