© 2004 Keynote Systems

Customer Experience Management (CEM)

Bonny Brown, Ph.D.Director, Research & Public Services

2

Customer Experience vs. Usability

Web sites - part software, part marketing medium

Usability is a subset of customer experience Usability – Can users use it? How can we make it easier? Customer Experience – Do users like it? How can we better drive

conversion, loyalty, and brand?

Areas for Customer Experience insights: Customer needs, preferences Barriers to adoption Drivers of retention and loyalty Concept testing of new features Feedback and impact of policies A/B Testing to make volume predictions

3

Research Phases

1. Discover: Define strategic direction Business objectives Core business metrics (sales, leads, traffic, support center data) Competitive assessment / benchmarking (baseline) Drivers / Customer insight Segmentation

2. Explore: Develop concepts and assess user reactions User interaction / reaction Unanticipated insights / obstacles / opportunities

3. Evaluate: Refine and perfect solution Predict user response to implemented solution, prove product success

before launch

4. Track: Set goals and monitor success Competitive assessment / benchmarking (ongoing) Optimization

4

Metrics to Manage Customer Experience

• Metrics to measure progress

• Strategic direction

• Understand relative strengths and weaknesses

• In-depth customer feedback

• Actionable recommendations and best practices

• Affordable research

Customer Needs

• Ongoing success metrics

• Driver analysis

• Apples-to-apples comparison to competitors

• Attitudes and behaviors reveal “why’s”

• Analysis conducted by experts

• Multi-client study reduces research cost per client

Keynote Syndicated

5

The Complete Customer Experience

• Customer Conversion

• Brand

• Retention

• Productivity

• Customer Service

• Revenue Maximization

Continuous Measurement and Improvement

ImpactImpact== CustomerExperienceCustomerExperience

Are yourcustomers:

Engaged?

Frustrated?

Confused?

Why?

++

AdvertisingPrevious

Experience

Word of Mouth

AdvertisingPrevious

Experience

Word of Mouth

ExpectationExpectation

What do users think about you?

How do they expect to be treated?

What do theywant to do?

Behavior

Thoughts

InteractionInteraction

What do users do?

What are they thinking?

Do they succeed?

6

Success Indices and Drivers

SITE EXPERIENCE METRICS(250+ quantitative metrics gathered per site)

DRIVER ANALYSIS(Aspects of the experience are grouped into factors and then regressed

against the indices to identify factors most related to brand and conversion)

DRIVER ANALYSIS(Aspects of the experience are grouped into factors and then regressed

against the indices to identify factors most related to brand and conversion)

CONVERSIONINDEX

CONVERSIONINDEX

BRANDIMPACTINDEX

BRANDIMPACTINDEX

CUSTOMERSATISFACTION

INDEX

CUSTOMERSATISFACTION

INDEX

OVERALL CE RANKINGS (Overall site effectiveness based on equal weighting across each of the three indices)

OVERALL CE RANKINGS (Overall site effectiveness based on equal weighting across each of the three indices)

7

Traditional Methods

IntuitionIntuition

Focus GroupsFocus Groups

UsabilityUsability

Log Server AnalysisLog Server Analysis

Web-based SurveysWeb-based Surveys

Robotic AgentsRobotic Agents

CustomerExperienceCustomer

Experience

8

Integrate the Best of Each Method

Users Intent

Known

LargeSamples

QualitativeData

BehaviorData

NaturalSetting

AnalysisTools

Focus Groups

Usability Labs

Surveys

Traffic Analysis

Keynote

9

..using our browser companion that captures events

…to evaluate site(s)

…providing comments and

feedback

…which are analyzed with our powerful software

…performing tasks

& answering questions

“Find information that meets your needs.”

“How satisfied are you with this site?”

How Keynote CEM works

Panelists are invited or

intercepted…

10

Canon vs. Xerox

The “client,” Canon, turned to Keynote to evaluate Customer Experience on the Canon.com website…

Home Page

Product Search Capabilities

Online Customer Support Options

…benchmarked against top competitor site, Xerox.com

11

A large sample of target and actual customers participated in the evaluation:

800 panelists from the Keynote Research Panel

Majority of panelists significantly involved in purchasing printers, fax machines and copiers for company

37% of panelists actual Canon customers

…providing credible Customer Experience data

Study Participants

12

InsightPoor Customer Experience on the Canon.com website is resulting in lost revenue

1. Canon.com site is cluttered and difficult to navigate

2. Customers find search engine unhelpful

3. Inadequate product information is provided for customers to evaluate products

4. As a result, customers can’t find products that meet their needs on the site

Customer Insight

13

The participant’s view

14

Panelists commonly

describe Canon homepage as “busy” and “cluttered” compared to

“clean” Xerox homepage

15

Panelists comments reveal that

homepage does not provide

clear navigation to

product information

16

Majority of panelists

failed or gave up product search on

Canon.com

Panelists took very scattered

paths trying to find

product

17

In contrast, the majority of panelists

succeeded in product

search on Xerox.com!

18

The majority of panelists used the search

engine on Canon.com

…

19

…and 99% encountered problems

with Canon.com’

s search engine!

20

80% of panelists did

not find adequate product

information to select a product on Canon.com

21

Product Search

80% of all panelists were unable to find a

product on Canon.com

22

Panelist provides detailed

description of

frustrations on

Canon.com

23

•Spent over 8 minutes

•Couldn’t find the product

•Rated Canon.com “Extremely Difficult”

24

25

Only panelists who authorize purchases of

faxes, printers and copiers

Poor Customer Experience is costly!

26

Actionable Recommendations

Improve ease of finding products:

Create clearer product classification categories for browsing customers

Categorize search engine results Expand product information

Impact

Increase transactions volume and revenue

Canon vs. Xerox

27

Web customer experience is a combination of excellent application design and marketing

Customer Experience research needs to: Capture the complete Web experience Explain the “Why” behind the “What” Combine the best of multiple methods

New tools and methods allow better insight into how to improve the customer experience and the bottom line

Conclusions