How to Create Measurement Dashboards That
Communicate PR’s Business Value
Katie Delahaye Paine
CEO
PainePublishing.com
PR News’ Measurement Conference
Chicago, IL
November 18, 2015
Paine Publishing: Providing communications professionals the knowledge and information they need to navigate the journey to great measurement.
The 4 Pillars of Paine Publishing
Education
Training
Consulting
Publishing
Katie Delahaye Paine: Helping communications professionals define and measure success for 30 years.
Founder of:
The Delahaye Group (now Cision)
KDPaine & Partners (now Carma)
Paine Publishing
Author of:
Measuring the Networked Nonprofit
Measure What Matters
Measuring Public Relationships
50 Shades of PR ROI
Measurement 101
2
#powerofPR @queenofmetrics
► Big data raises false hopes
► Measurement is no longer an
option
► Impressions are not sales or
awareness
► Everyone is “media”
► Sentiment is overrated
► It’s not about the media, it’s
about your mission & what your
stakeholders do with the info
when you put it out there
► The Barcelona Principles +
Standards & Best Practices for
Measurement
What has changed in communications?
#powerofPR @queenofmetrics
The Coalition
Definition of an Item of Content 1
Circulation 2
Mentions 3
Sentiment 4
Quality Elements 5
The Coalition published, tested, and validated standards for:
Definition of an Item of Content
The Conclave
Content & Sourcing 1
Reach & Impressions 2
Engagement & Conversation 3
Opinion & Advocacy 4
Influence 5
Impact & Value 6
For more info go to http://painepublishing.com/standards-central-2/
The Conclave published standards for:
#powerofPR @queenofmetrics
Why a Dashboard?
You can’t separate social and traditional media
You can’t isolate employees from customers, prospects or other stakeholders
Engagement delivers a 23% premium in terms of share of wallet, profitability, revenue, and relationship growth over the average customer
Companies that engage both their employees and their customers gain a 240 percent boost in performance-related business outcomes
No one has time to flip between 3-4 different platforms and apps to figure out what’s working and not working and whether you have enough resources to get where you’re going
#powerofPR @queenofmetrics
6 Steps to a Dashboard Leadership will Love
Step 1: Define your goal
What outcomes is this strategy or tactic
going to achieve? What are your measurable
objectives?
Step 2: Define the parameters
Who are you are trying to reach? How do
your efforts connect with those audiences to
achieve the goal?
Step 3: Define your benchmarks
Who/what are you going to compare your
results to?
Step 4: Define your metrics
What are the indicators to judge your
progress?
Step 5: Select your data collection tool
Step 6: Analyze your data.
Turn it into action, measure again!
1
2
3
4
5
6
#powerofPR @queenofmetrics
Outtakes (Intermediary Effects)
• Awareness
• Knowledge/Education
• Understanding
Outcomes (Target Audience Action)
• Revenue
• Leads
• Engagement
• Advocacy
Activities
How does what you do
contribute to the bottom line?
Step 1: Define the Goals
#powerofPR @queenofmetrics
What’s your champagne moment?
#powerofPR @queenofmetrics
Goals Goal 1: Meet Sales
Targets
Goal 2: Reduce
Risk/Threats
Goal 3: Increase
market share in new
market
Communications’
Contribution
• Expand the
marketable
universe
• Reduce sales cycle
• Increase trust
• Increase advocacy
• Increase employee
engagement
Expand the marketable
universe
Metrics
• % Increase in
desirable share of
voice
• % in awareness
• % decrease in
undesirable voice
• % increase in trust
scores
• % increase in
employee
engagement
• % increase in
desirable voice in
new market
• % increase in
awareness of
brand in new
market
Goals & Suggested Metrics
#powerofPR @queenofmetrics
First: Understand how you contribute to the business or the
mission
High-quality media coverage
Improved reputation
Awareness/Engagement/Consideration
Increased revenue
Definitions of “Success”
#powerofPR @queenofmetrics
Employees
Customers
Prospects
Investors
Media
Who are the Stakeholders?
What are management’s
priorities?
Who are you are trying to
reach?
How do your efforts connect
with those audiences to
achieve the goal?
What influences their
decisions?
What’s important to them?
What makes them act?
Step 2: Understand the Parameters
#powerofPR @queenofmetrics
Measurement is a comparative tool
Compare to:
Past performance over time
Peers/Competitors
Whatever keeps leadership awake at night
Step 3: Establish Benchmarks
#powerofPR @queenofmetrics
Step 4: Define your Kick Butt Index
You become what you measure,
so pick your metrics carefully
The Perfect KBI
Is actionable
Is there when you need it
Continuously improves your
processes & gets you where
you want to go
#powerofPR @queenofmetrics
Desirable Criteria Score Undesirable Criteria Score
Contains a key message
3.50 No key message -1.0
Contains a desirable visual
0.75
Negative message,
negative myth reinforced -3.0
Contains a quote from a
spokesperson
2.50
Contains a competitor
quote -1.0
Positions your brand as best in
class
0.75
A story or a headline that
leaves the reader less
likely to do support the
organization
-3.0
Dispels a myth
0.75
Organization omitted
from story that includes
competitors mentioned
-2.0
The story or headline leaves a
reader more likely to support the
organization
1.75
Total
10.00 -10.0
14
Media Quality Score
#powerofPR @queenofmetrics
Social Media Engagement Index
Action Score
“Like”/Follow/Opens/+1 0.5
Favorite or Opens or Views 1
Comment 1.5
Share content 2
Signs up to receive email or other owned content 2.5
Shares a link to an owned site 2.5
Total 10 24
#powerofPR @queenofmetrics
Elements in an Employee Engagement Index
Element Score
More likely to invest discretionary time 0.5
More likely to recommend to family & friends as a great place to
work 1
Greater understanding of organizational mission vision & values 1.5
Greater understanding of key organizational messages 2
Lower retention rate 2.5
Lower recruitment costs 2.5
Total 10 24
#powerofPR @queenofmetrics
Step 5: Pick the Right Dashboard Platform
If you want to measure messaging,
positioning, themes, sentiment:
Content analysis
If you want to measure awareness,
perception, relationships, preference:
Survey research
If you want to measure engagement,
action, purchase:
Web analytics
If you want correlations, find the
Dashboard Platform that can
integrate all three
#powerofPR @queenofmetrics
Goals Determine Tools
Communications’
Role
Interim Metric
Outcome
Metric
Tools Required
Increase
understanding of key
messages
Increase in % of
quality coverage
% increase in
understanding
Media Quality Analysis
Qualitative Survey
Enlarge & improve
relationships with
NGOs & other
influencers
% increase in
share of influencer
voice
% improvement
in relationships
with influencers
Media Content
Analysis Relationship
Survey
#powerofPR @queenofmetrics
#powerofPR @queenofmetrics
A Good Dashboard Tool is More Than Pretty Charts
Valid data
Easy to find answers to your questions
Metrics aligned to goals
Integration of social, traditional,
digital, web, survey data, CRM,
etc.
Ability to easily find the data
and/or stories behind the
charts/numbers
Comparison of Tonality Between Vendors
33.12%
58.00%
7.84%
11.95%
26.53%
13.00%
0.32%
9.46%
40.35%
30.00%
92.11%
78.00%
Vendor 1
Vendor 2
Vendor 3
Vendor 4
% positive %negative %neutral
#powerofPR @queenofmetrics
Step 6: Be Data Informed, not Data Driven
Rank order results from worst to best
Ask “So What?” at least three times
Put your data into an overall
framework consistent with C-Suite
expectations
Find your “Data Geek” (or someone
who is)
Compare to last month, last quarter,
13-month average
#powerofPR @queenofmetrics
Dashboard Examples
Engage As Owners
Create Advocacy Support the Business
With Employees
• Employee Engagement Index
• Pulse point survey scores for Proud, Recommend, and Understand
• % increase in employee sentiment in blog comments
• % increase in net positive sentiment on Glassdoor
• Progress against Survey Action Plan
With Customers
• External Engagement Index
• Awareness of community involvement
• Media Quality Index • Overall increase in
message communications • Revenue generated
• Increase revenue from Comms’ tagged URLs
Measurement Framework
#powerofPR @queenofmetrics
Improve Perceptions & Advocacy
*See glossary for definitions
48% 49% 50%
53%
18% 19% 20% 21%
2% 2% 2% 3%
13% 11%
9% 8%
Q1 Q2 Q3 Q4
Internal & External Sentiment Over Time
Positive Internal Positive External
Negative Internal Negative External
What happened? What we did: What we will do next:
Engage As Owners - External
Engagement with INTERNAL channels
0
10
20
30
40
50
60
0
10
20
30
40
50
60
70
Employee Engagement vs Sentiment Over Time
Employee Positive Sentiment Employee Engagement Index
What happened? What we did:
What we will do next:
Summer Event
Engagement with EXTERNAL channels
0.00
20.00
40.00
60.00
80.00
100.00
120.00
0.00%
20.00%
40.00%
60.00%
80.00%
100.00%
Jan Feb Mar Apr May Jun Jul Aug Sept Oct Nov Dec Jan
MQ
I
% o
f it
ems
Key Metrics Over Time
Messaging Sentiment
Top stories by MQI: What happened? What we did:
What we will do next:
Engage As Owners – Internal
Number of Employees who participate in EXTRA EFFORT Quarterly Events:
1Q 2Q 3Q 4Q Notes
Preparedness Engagement
72 584 -- -- 584 attended/896
opportunities
Volunteerism 3,631 -- -- 57,988 hours
Annual Events:
SWA Rally (1Q) Banquet (2Q) Gala (3Q)
Key Contact Training (3Q)
Spirit Party (4Q)
8,067 851 -- -- --
Engagement with INTERNAL channels
0
10
20
30
40
50
60
0
10
20
30
40
50
60
70
Employee Engagement vs Sentiment Over Time
Employee Positive Sentiment Employee Engagement Index
What happened? What we did: What we will do next:
Summer Event
The A-Ha moments Come from Integrating Data
Correlations shown between media quality and unique traffic to the destination site
High Resource Events do not increase trust in the organization
Photo Event
High Message Content
Resource Use
Low
Hig
h
Med
ium
Ver
y h
igh
Level of Engagement
Ver
y H
igh
Med
ium
Hig
h
Low
High Resources
No Message
Low Resources
Webinar
Status update
Link
Ultimate Road Trip
Google + Chat
Media Day
Corporate Video
Resource Use
Low
Hig
h
Med
ium
Ver
y h
igh
Ver
y H
igh
Med
ium
Hig
h
Low
High Resources Low Resources
Webinar
Status update
Link
Ultimate Road Trip
Google + Chat
Media Day
Corporate Video
Advocacy vs. Resource Use
#powerofPR @queenofmetrics
-6
-4
-2
0
2
4
6
Jan Feb Mar Apr May June
% point change since last month
Share of desirable voice Share of undesirable voice
Unique visits to Website Engagement Index
Brand Metrics
-5
0
5
10
Jan Feb Mar Apr May June
% point change since last month
Share of desirable voice in biologics Visits to UnderstandingBiologics.com
Share of desriable HepC voice Share of undersirable voice in HepC
Visits to HepCInfo.com Peception change
-5
0
5
10
Jan Feb Mar Apr May June
% point change since last month
Desirable Oncology SOV
Innovative positioning in media
Visits to abbvie.comresearch-innovation
Innovation social engagement index
Perception of AbbVIe as Oncology leader
-6
-4
-2
0
2
4
6
Jan Feb Mar Apr May June
% point change since last quarter
Ratings on trust
Employee knowledge of "Way We Work"
Understanding of Strategic Objectives
Reduction in Say/Do gap
Culture Metrics Science & Innovation Metrics
On-Market Products
Goal: Measure All Four Departments
#powerofPR @queenofmetrics
KPI Last month/period This Period Difference
Share of Desirable Voice 10% 12% 2pts
Share of Undesirable Voice 5% 2% 2pts
% increase in Unique Visits 100k 137K 2%
Engagement Index 1357 2568 2%
% aware 35% 37% 2pts
% familiar 50% 52% 2pts
RMI 75 77 2
What Worked:
What Didn’t:
What We Will Do Next:
-6
-4
-2
0
2
4
6
Jan Feb Mar Apr May June
% point change since last month
Share of desirable voice
Share of undesirable voice
Unique visits to Website
Engagement Index
Brand Metrics
#powerofPR @queenofmetrics
KPI Last quarter This Period Difference
% increase in ratings on communications trustworthiness
3% 5% 2pts
% increase in employee knowledge of organization “Way we work”
12% 14% 2pts
% employees strongly understanding strategic objectives
2% 8% 25%
% reduction in Say/Do gap on agility metrics 50% 48% 2pts
What Worked:
What Didn’t:
What We Will Do Next:
-4
-3
-2
-1
0
1
2
3
4
5
Q1 Q2 Q3 Q4
% change since last period
% increase inratings oncommuniationstrustworthiness
%increase inemployeeknowledge of Waywe Work
% employeesunderstandingstrategicobjectives
Culture Metrics
#powerofPR @queenofmetrics
Measure Performance Against Three Top Goals
Generate inbound inquiries Facilitate user adoption of comms resources
Establish ourselves as a reputable thought leader
45
50
55
60
020406080
100
Inbound Inquiries Over Time By Product
# of inbound inquiries
# of inbound inquiries for EIS
# of conversions
% increase in conversions & inbound inquiries
Last Month
This Month
Change
10% 15% +5 pts
% increase in inbound inquiries for EIS
Last Month
This Month
Change
10% 9% -1 pts
% increase in conversions from Comms activities
Last Month This Month Change
10% 15% +5 pts
50 75 46 89 99 2.00
5 4.00 5 6.00
0.00
5.00
10.00
0
100
200
Perceptions vs Media Impact Over Time
% would recommend
% preferring
% would consider
% aware
MQI
% increase in a recommend
Last Month
This Month
Change
10% 15% +5 pts
0
50
100
150
200
Jan
Feb
Mar
Ap
rM
ay Jun
Jul
Au
gSe
pt
Oct
No
vD
ec Jan
Conversions Over Time
Conversions
% increase in a preferring
Last Month
This Month
Change
10% 15% +5 pts
% increase in a consideration
Last Month
This Month
Change
10% 15% +5 pts
% increase in awareness
Last Month
This Month
Change
10% 15% +5 pts
% increase MQI
Last Month
This Month
Change
3 5 +2 pts
2
Business Development Dashboard
45
50
55
60
020406080
100
Inbound Inquiries Over Time By Product vs Conversions
# of inbound inquiries
What happened? What we did: What we will do next:
3
% increase in conversions & inbound inquiries
Last Month This Month Change
10% 15% +5 pts
% increase in inbound inquiries for EIS
Last Month This Month Change
10% 9% -1 pts
0
1
2
3
4
5
0
1
2
3
4
5
6
Jan
Feb
Mar
Ap
r
May Jun
Jul
Au
g
Sep
Oct
No
v
Dec
Conversions Over Time By Source
Social Content
Web Email
Traditional
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
8.00
$0
$50
$100
$150
$200
$250
$300
$350
$400
Jan Feb Mar Apr May Jun Jul Aug Sept Oct Nov Dec JanM
QI
Val
ue
of
Co
nve
rsio
ns
Conversion by Source Over Time Relative to Media Quality
General Web Site Conversion E-Mail Click Thrus
Social Conversions Content Conversions
MQI
What happened? What we did: What we will do next:
Engagement Dashboard
5
% increase in conversions from Comms activities
Last Month This Month Change
10% 15% +5 pts
0
1
2
3
4
5
6
7
8
9
10
0102030405060708090
100
Jan Mar May Jul Sept Nov Jan
MQ
I
% o
f R
esp
on
den
ts
Change in Perceptions Over Time
MQI Aware
Recommend Prefer
Consider
Recommend
Q1 Q2 ∆
10% 15%
+5 pts
What happened? What we did: What we will do next:
Reputation/Image Management Dashboard
Prefer
Q1 Q2 ∆ 10%
15%
+5 pts
Consider
Q1 Q2 ∆
10% 15% +5 pts
Awareness
Q1 Q2 ∆
10% 15% +5 pts
Perception Survey Changes (Quarterly) Media Image Changes (Monthly)
MQI
Q1 Q2 ∆
10%
15%
+5 pts
Messaging
Q1 Q2 ∆
10%
15%
+5 pts
% Positive
Q1 Q2 ∆
10%
15%
+5 pts
Social Engagement
Q1 Q2 ∆
10% 15% +5 pts
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
8.00
9.00
10.00
0
20
40
60
80
100
120
MQ
I
% o
f to
tal a
rtic
les
Media Quality Over Time
Positive Messaging MQI
4
Messaging Sentiment Media Quality Index(MQI) Social Engagement Index
Top Ten Stories by MQI
Oct Nov Change Goal
3.5 4.5 1 pt. 5
Oct Nov Change Goal
20% 30% 10 pts 50%
Oct Nov Change Goal
3.5 4.5 1 pt. 5
Oct Nov Change Goal
12% 15% 3 pts 20%
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
8.00
0.00%
5.00%
10.00%
15.00%
20.00%
25.00%
30.00%
Jan Feb Mar Apr May Jun Jul Aug Sept Oct Nov Dec Jan
MQ
I
% o
f it
ems
Key Metrics Over Time
MQI Sentiment Messaging
Volume of Coverage
Oct Nov Change Goal
2.5K 4.5K 1K 10K
6
Engage As Owners – External
What happened? What we did: What we will do next:
#powerofPR @queenofmetrics
Dashboard Dos and Don’ts
Don’t use metrics that you
don’t have buy-in for
Don’t measure what’s easy
Don’t clutter up your
dashboard
Don’t put numbers on it you
can’t explain
Don’t use charts that people
can’t read or understand
Get consensus on definitions of
Success
Measure what matters –how
you contribute to the business
Make your metrics tell a story
Make sure your data is valid
and accurate
Test any indexes or algorithms
with real data before presenting
them
#powerofPR @queenofmetrics
Thank You!
For more resources on measurement, check out Paine
Publishing’s Measurement Mall
For a copy of this presentation or for any questions, email
Follow me on Twitter: @queenofmetrics
Follow Paine Publishing on Facebook
Or call me: 1-603-682-0735
39
#powerofPR @queenofmetrics
Top Related