How are you keeping up with AI?...6/26/2018 1 How Are You Keeping Up with AI? The Creation of...
Transcript of How are you keeping up with AI?...6/26/2018 1 How Are You Keeping Up with AI? The Creation of...
6/26/2018
1
How Are You Keeping Up with AI? The Creation of Actionable Insights and
Dashboards Around Chatbot Performance
Beth CorsoManager, Customer Insights and
Measurement, American Family Insurance
How are you
keeping up with AI? The Creation of Actionable Insights and Dashboards Around Chatbot Performance
6/26/2018
2
• American Family Insurance
• American Family Insurance and Clarabridge History
• Overview of the Session
• Chatbot and Virtual Assistant Background
• American Family Insurance Customer Bot, Abby
• Utilizing Clarabridge for Abby Performance and Deep Learning Analytics to Drive Prioritization
• LearningsAG
EN
DA
$8.8 Billion in
Premium
2017 American Family Insurance
$10.5 Billion
Active
Policies
Fortune 500
Ranked 311
(moved 92 spots in 22 yrs.)
6/26/2018
3
American Family Insurance
American Family Insurance + ClarabridgeOUR JOURNEY SINCE SUMMER OF 2015
• Started with 2 call centers
• Focused on speech calls
• Voice to text evaluations
• Quality Assurance for
Care Center
• Root cause analysis
• Chatbot monitoring and
intent building
• Multi-Channel Data
Source Injections
• Elimination of call
dispositioning
• Expansion to surveys,
webchats, social data,
digital feedback
BUILD CLOUDIFY & EXPAND LEARN & OPERATIONALIZE
6/26/2018
4
Conversational
CommerceCustomer Chatbot via Webchat
Artificial Intelligence, ChatbotWHAT IS A CHATBOT AND IS YOUR ORGANIZATION UTILIZING AI TODAY?
Software RobotsIncreasingly being
used to automate
business processes
6/26/2018
5
Meet Abby, Our Customer BotBUSINESS GOAL: Create a conversational virtual assistant our customers will benefit
from in their busy daily lives by answering their insurance questions and provide
support to their every day needs.
Wait time optimization
for calls and representative webchats
Continuous customer service
to allow customer toget support they need
TOOLS
• Customer Empathy
• Dialog Testing
• Usability Testing
• Customer Insights
• Chatbot to answer the
customer’s questions
correctly without a wait
time
• Reduce wait times for chat
• 24/7 customer care via chat
• Provide customers consistent
answers
SOLUTION SUCCESS METRICS
CUSTOMER PAIN POINTS TO SOLVE W/ AN AI CUSTOMER CHATBOT
Answer consistency to our
customers regardless of the channel contacted for support
Who is Abby?
LIFE MOTTO: Where there’s a will, there’s a way.
BRAND PERSONA
LIFE MISSION: Helping other people realize and achieve their vision of success.
DEMOGRAPHICS:
• Age: 38 (old enough to have
enough life experience, but young
enough to be relevant)
• Mom of 3
• Life Coach and Licensed Insurance
Agent – protection expert!
BACKSTORY:
• Humble upbringing in the
Midwest. Raised by
grounded and diligent
small farmers who value
and understand the
importance of hard work.
ABBY…
• Inspirationally guides
• Transparently
navigates
• Humbly coaches
• Proactively plans
and manages
6/26/2018
6
How Does Abby Work?
Customer
Utterance NLP
Utterance to Intent
Intent to Content
Abby’s Response
Abby
Intent: Category of what the customer is asking
Add/Remove Vehicle
Utterance: How the customer asks the question
“I just got a new car and would
like to remove my old one and add my new one.”
American Family
Insurance + ClarabridgePerformance and Deep Learning Analytics
6/26/2018
7
Clarabridge and Abby Performance3 STEP IN CREATING AN AI CHATBOT
1 -
Go
als •Containment: How many
chats start with the bot and end with the bot
•Building Knowledge: How are we increasing our intents and utterances through customer interactions
•Correct Intent Mapping: How are the utterance to intent performing
•Conversational Sentiment: How is the bot sentiment compared to the customer sentiment
2 -
Pe
rfo
rma
nc
e •Increasing containment as we increase our knowledge base of utterances and intents, 22%
•Started with 76 FAQ intents and increased to 180+ which consists of 30,000 utterances
•Correct utterance to intent mapping has increased since launch and is at 80%
•Conversational sentiment with the bot and customer is neutral 3
-D
ee
p L
ea
rnin
g •Verification success vs failure
•Intent mapping to categorization through Clarabridge modeling
•Total webchat conversation vs Abby conversations
•Intent sequencing: Where does speak to rep fall in the conversation and what intent came prior to the transfer
•Webchat profiling: Who is our customer that is utilizing webchat compared to phone calls
Clarabridge and Abby PerformanceWHAT ARE OUR MEASUREMENT GOALS AND TRACKING?
• Total Webchats vs. Chatbot• Webchats vs. Chatbot Categorization
Metric Overview
Abby Contained Chats
• Total Contained and Penetration to Total• Chatbot Intents Comparison to Model Categories
Abby to Rep Transfers
• Total and % of Transferred Chats + % of Initials Trans• Chatbot Intents Comparison to Model Categories
Abby Verification• Total and % of Prompted and Successful• % to Success, Failure, Opt Out, Abandon, and Intent Vol
6/26/2018
8
Clarabridge and ABBY INTENT SEQUENCINGSEQUENCING DASHBOARD, DEEPER DIVE INTO THE BOT CONVERSATION
• Total Webchats vs. Chatbot• Contained Chats and Transfer Chats
• Care Center Drillable Intents to Lowest Level
Metric Overview
Webchat Intent Sequencing
• Intent Volume• Speak to Representative sequence evaluation
• 1-4 Intent Sequence Volume
Intent Category Deep Dive
• By category: Billing and Payments, Claims, Policy, Account Management, etc
6/26/2018
9
Abby Customer BotWHAT HAS AMERICAN FAMILY INSURANCE LEARNED AND WHAT IS NEXT 1-2-3 …
1 -
Lea
rnin
gs
•Monitor Daily: if we hadn’t
monitored Abby daily we
wouldn’t have found key insights
•Greeting: does your customer
read it?
•Content: are you explaining
things correctly?
•Verification: does your customer
follow directions?
•Customers need guidance
•You don’t need a big team to
succeed
2 -
Exp
an
sio
n •2018: research, learning,
development
•Understand what metrics
drive a customer
•What platforms best enables
the business to scale and
succeed?
•Expansion
•Data Access
•Channels
•Voice – Alexa, Google, etc
3 -
Re
co
mm
en
da
tio
ns •Monitor Daily
•Empathy Work – do what the
customer tells you they want
•Establish Goals
•Customers need parameters
to work in
•Test and Learn
•Don’t be afraid to fail
•Have fun
6/26/2018
10
Actionable insights from every
customer interaction, in one platform.
www.clarabridge.com