BOT-O-LOGY - Entropia · Facebook's Messenger platform – which explains the reduced emphasis...

41
the contrarian BOT-O-LOGY a whitepaper by

Transcript of BOT-O-LOGY - Entropia · Facebook's Messenger platform – which explains the reduced emphasis...

Page 1: BOT-O-LOGY - Entropia · Facebook's Messenger platform – which explains the reduced emphasis companies seem to place on their chatbot's user experience. The Messenger platform provides

the contrarian

BOT-O-LOGYa whitepaper by

Page 2: BOT-O-LOGY - Entropia · Facebook's Messenger platform – which explains the reduced emphasis companies seem to place on their chatbot's user experience. The Messenger platform provides

1.0Introduction

Page 3: BOT-O-LOGY - Entropia · Facebook's Messenger platform – which explains the reduced emphasis companies seem to place on their chatbot's user experience. The Messenger platform provides

Bots are not new, but they are bursting into the marketing buzzword library with renewed vigour. After all, in today’s connected world, it is virtuallyimpossible to navigate the internet withoutencountering a chatbot. These artificially intelligent conversationalists have been around for over 50 years, but saw a recent surge in interest as many brands have realised by automating recurring and repetitious customer service tasks, they can reduce costs and improve responsiveness, or perhaps even be proactive without additional expense.

To perform these tasks, pre-programmed commands or scripts are coded in to reply to a user’s enquiry when that question is relevant. It is more pertinent

now than ever before, as this feeds into the ever-demanding need for users to get real-time replies at any time of the day. Consequently, chatbots have become smarter, more natural, efficient, responsive, and useful.

Think simple stand-alone digital assistants like Siri and Alexa, natural language processing (NLP) tools such as wit.ai, cognitive technologies such as IBM’s Watson and bot developer frameworks like Face-book’s Messenger Bot API. With so many big players willing to dive into this space, we are seeing more and more brands willing to dip their toes into the pool to provide a service to consumers that is personal, contextual and cost efficient.

The Google Trends graph above shows how searches for "chatbot" have grown worldwide between 2013 and 2019. After a huge jump in search volume in the past three years, "chatbot" stayed a popular search term, as indicated by its healthy year-on-year incline.

Developers and brands often give chatbots apersonality, so much that it operates as if there is a real person behind the chat interface. With this in mind, consumers nowadays are expecting more and more from every interaction they have with a chatbot.

Today, the execution of a ‘bot’ is still blurry and encompasses a number of different interpretations, though the technology has come a long way from simply replying to Frequently Asked Questions.

Chatbots can now perform highly complicated tasks and initiate customer-specific conversations based on historical and behavioural data, making them invaluable for enhancing business efficiency. Adding artificial intelligence capabilities into the mix will enhance chatbots to further address consumer frustrations, automate manual processes, and provide ongoing support.

At ENTROPIA, we spent a lot of time looking at chatbots and how they impact the industry. This report aims to look at some of the existing chatbots in the market today.

03

CHATBOTS TODAY CAN PERFORM HIGHLY COMPLICATED TASKS AND INITIATE CUSTOMER-SPECIFIC CONVERSATIONS BASED ON HISTORICAL AND BEHAVIOURAL DATA, MAKING THEM INVALUABLE FOR ENHANCING BUSINESS EFFICIENCY.

Page 4: BOT-O-LOGY - Entropia · Facebook's Messenger platform – which explains the reduced emphasis companies seem to place on their chatbot's user experience. The Messenger platform provides

2.0The Study

Page 5: BOT-O-LOGY - Entropia · Facebook's Messenger platform – which explains the reduced emphasis companies seem to place on their chatbot's user experience. The Messenger platform provides

We selected a total of 30 chatbots for study from various industries. These chatbots consist of a mix of brand chatbots built to answer simple FAQs and tend to customer service, to AI-powered chatbots designed to collect data and close sales.

Ultimately, we developed three evaluation criteria groups for our analysis: Reliability, Experience, and Personality. In our analysis, we kept in mind the reason chatbots came to be in the first place – as a way to automate query responses in a faster and more efficient way.

05

ULTIMATELY, WE DEVELOPED THREE EVALUATION CRITERIA GROUPS FOR OUR ANALYSIS: RELIABILITY, EXPERIENCE, AND PERSONALITY.

Reliability

PersonalityExperience

Page 6: BOT-O-LOGY - Entropia · Facebook's Messenger platform – which explains the reduced emphasis companies seem to place on their chatbot's user experience. The Messenger platform provides

ReliabilityAs the ultimate goal for a chatbot is to make communication faster and more effective, reliability is key. To measure Reliability, we evaluated parameters like information availability on the chatbot, speed of replies to user queries, answer accuracy, and the level of integration of the database.

We used a total of 11 parameters to evaluate Reliability:

Navigation (Guided) Ease of navigation through pre-allocated commands

Messaging (Non-Guided) Ease of navigation through freestyle conversations

FAQ Searches FAQs that are outside the chatbot commands

Human Intervention Level of depth without the need for intervention

Level of Integration Amount of database information that is integrated on to the bot

404 Response Chatbot's response when there is no answer

Response Time Amount of time needed to crawl through the database

Task Completion Percentage of successful task completion

Information Availability Amount of information on the bot

Accuracy Accuracy of answers to query

Conversation Depth Ability to continue conversations on the topic

06

WE EVALUATED ALL 30 OF OUR SELECTED CHATBOTS BASED ON THESE PARAMETERS AND ASSIGNED THEM POINTS – FROM 1 POINT FOR AREAS IN WHICH THEY PERFORM POORLY TO A MAXIMUM OF 5 POINTS FOR AREAS IN WHICH THEY EXCEL.

2.1

Page 7: BOT-O-LOGY - Entropia · Facebook's Messenger platform – which explains the reduced emphasis companies seem to place on their chatbot's user experience. The Messenger platform provides

PersonalityEven though users are aware that chatbots are inorganic computer programs, conversations still need to be as organic as possible. Personality helps a chatbot speak to users more naturally, in the voice of the brand they represent. To rate a chatbot Personality, we looked at localised language and culture support, as well as how relevant it is to the brand.

We evaluated these 6 parameters to assess Personality:

A chatbot can score a maximum of 140 points (55 points each for Reliability and Experience and 30 points for Personality) with each parameter weighted individually.

Brand Relevance Level of personality integration between chatbot and brand

Character How much of an individual the chatbot has inherited

Localisation Language and culture support for the different countries

Functionality Ability to exist without an internet connection

Out of Scope Conversation How the chatbot reacts to random conversations

Personalisation Ability to customise communication styles to different individuals

Navigation (Guided) Messaging interface

Messaging (Non-Guided) Messaging interface

Redirection Amount of times users need to be redirected to external information

Offline Capabilities Ability to exist without an internet connection

Accessibility Accessibility of chatbot

Dependencies Ability of chatbot to exist as a standalone

User Recognition Ability to customise conversations based on user

Machine Learning Ability to detect and correct mistakes

Cross Channel Availability across different channels

Cross Device Interface optimisation across different devices

Cross Operating System Availability of chat across the different OS

ExperienceUser experience plays a huge part in chatbot interaction, so Experience is a key aspect of our evaluation as well. To gauge Experience, we looked at factors like ease of communication, user interface design,accessibility and availability, and compatibility with various operating systems and devices.

We used these 11 parameters to evaluate Experience:

07

2.2

2.3

Page 8: BOT-O-LOGY - Entropia · Facebook's Messenger platform – which explains the reduced emphasis companies seem to place on their chatbot's user experience. The Messenger platform provides

3.0Overview

Page 9: BOT-O-LOGY - Entropia · Facebook's Messenger platform – which explains the reduced emphasis companies seem to place on their chatbot's user experience. The Messenger platform provides

A survey conducted among users who have interacted with chatbots shows that a majority of users are not satisfied with the overall performance of these chatbots. Even though chatbots today have baseline functionality thanks to ready-made platforms that facilitate chatbot development, there is clearly room for improvement.

This tells us that even though most chatbots are fulfilling basic functions, they are not excelling at what they do. Users have built up expectations for top-level service after seeing how much chatbots have evolved over the years – at ENTROPIA, we believe being "Somewhat Satisfying" no longer cuts it in this day and age.

However, some chatbots do excel in their respective areas, and there is much for us to learn there. The next part of this whitepaper deep dives into the criteria in which the chatbots were scored.

09

EVEN THOUGH CHATBOTS TODAY HAVE BASELINE FUNCTIONALITY THANKS TO READY-MADE PLATFORMS THAT FACILITATE CHATBOT DEVELOPMENT, THERE IS CLEARLY ROOMFOR IMPROVEMENT.

Not at All Satisfying

Overall Performance

Not Satisfying Somewhat Satisfying Satisfying Highly Satisfying

26%

30%

33%

7%4%

Page 10: BOT-O-LOGY - Entropia · Facebook's Messenger platform – which explains the reduced emphasis companies seem to place on their chatbot's user experience. The Messenger platform provides

10

Here, we collected data on the areas of focus for companies that are developing their chatbots. Brands tend to emphasise personalisation and character most – hardly surprising in an age where brand identity matters. The chatbot frequently acts as a "customer representative" and is often the first point of contact between a brand and its audience. Any external communication is given top priority and needs to reflect the company's vision and brand values.

Companies also focus on reliability in their chatbots so that conversations with consumers can come to a productive end. Most chatbots that score well in this area have concrete amounts of information in their database, which lets them provide more relevant responses to user queries. By integrating natural language with a wealth of user information, chatbots can handle conversations that escalate outside their territory of expertise and respond to users who are there just to chat.

Most chatbots in the market today are built onFacebook's Messenger platform – which explains the reduced emphasis companies seem to place on their chatbot's user experience. The Messenger platform provides a simple chatbot engine that brands and individuals can use to create their bot in a matter of hours. Paired with Facebook's proprietary AI, brands can create a basic chatbot with little to zeroknowledge in the area.

BRANDS TEND TO EMPHASISE PERSONALISATION AND CHARACTER MOST – HARDLY SURPRISING IN AN AGE WHERE BRAND IDENTITY MATTERS.

PersonalityExperienceReliability

Page 11: BOT-O-LOGY - Entropia · Facebook's Messenger platform – which explains the reduced emphasis companies seem to place on their chatbot's user experience. The Messenger platform provides

UNFORTUNATELY, RELYING ON FACEBOOK'S MESSENGER PLATFORM TO BUILD A CHATBOT MEANS FACING LIMITATIONS IN TERMS OF PERFORMANCE AND CUSTOMISABILITY.

Even though these basic chatbots can be created at a fraction of the development cost compared tocreating it from scratch, developers will be limited in terms of what they can do to create a truly seamless chatbot experience.

Although many brands choose to compromise on chatbot experience to manage development cost, some brands have not. Chatbots that excel inExperience tend to be from brands that went the extra mile to build their chatbots on reputable platforms outside of Facebook that offer more options for customisation. These chatbots tend to learn and react much faster, as the needs of the brand are taken into account without compromise.

Page 12: BOT-O-LOGY - Entropia · Facebook's Messenger platform – which explains the reduced emphasis companies seem to place on their chatbot's user experience. The Messenger platform provides

4.0The Ratings

Page 13: BOT-O-LOGY - Entropia · Facebook's Messenger platform – which explains the reduced emphasis companies seem to place on their chatbot's user experience. The Messenger platform provides

Reliability

Rule-based: Conversations are prefixed and users are not able to enter text outside of a predefined script.NLP-based: Conversational but can onlyunderstand natural language by learning via preset content.AI-based: Learns through conversation and automatically builds responses.

This section lists the results for all parameters that were evaluated for each criteria group.

4.1

Within the Reliability criteria, our sampling of chatbots scored highest on Ease of Navigation and Response Time. This is an indicator that most brands place more emphasis on certain aspects of the customer journey.

In general, there are 3 types of chatbots:

Rule-based chatbots are the easiest to build as users are made to follow a predetermined journey. With predefined user responses, these chatbots can walk users through to a goal state set by the developer. However, guided chats can never build up to support natural conversation, nor handle new and non-linear user queries.

Conversation Depth scored lowest across the board for Reliability. Only a few chatbots have managed to handle and predict the many directions conversations can go. In this case, Facebook's Messenger platform is both a blessing and a curse - developers are forced to compromise on integration with the company's local environment (and the depth of their chatbot interactions) to minimise development cost and time spent.

a)

b)

c)

13

MOST BRANDS PLACE MORE IMPORTANCE ON CERTAIN ASPECTS OF THE CUSTOMER JOURNEY – MOST CHATBOTS SCORED HIGHEST ON EASE OF NAVIGATION AND RESPONSE TIME.

404 Response

Ease of Nav (Non)

Integration Level

Accuracy

FAQ

Response Time

Convo Depth

Human Intervention

Task Completion

Ease of Nav (Guide)

Info Availability

Reliability

Page 14: BOT-O-LOGY - Entropia · Facebook's Messenger platform – which explains the reduced emphasis companies seem to place on their chatbot's user experience. The Messenger platform provides

Experience4.2

14

In terms of Experience, our sampling of chatbots scored overwhelmingly well on Quick Selections and Ease of Messaging. This comes as no surprise, as a majority of chatbots today are built on established chat platforms like Facebook's Messenger. These chatbots all use a standard user interface, making it easy to navigate for most users who are already familiar with the format.

By using platforms like Messenger to develop chatbots, developers can focus more on populating the chatbot with content and not worry about technical issues such as load time, refresh rate and database crawling. The Messenger platform is already well-optimised, making crawling through and pulling information much faster, compared to developing a chatbot from an original platform.

Similarly, we can see why Accessibility and Cross Devices score the lowest in Experience. Chatbots on the Messenger platform are not able to exist outside of the Facebook environment – users must log in to Facebook to initiate a chat. Brands and developers will have to rely solely on Facebook to function across more devices.

CHATBOTS SCORED OVERWHELMINGLY WELL ON QUICK SELECTIONS AND EASE OF MESSAGING THANKS TO ESTABLISHED CHAT PLATFORMS LIKE FACEBOOK'S MESSENGER.

Accessibility

Cross OS

Ease of Quick Selections

Redirection

Cross Channel

Dependencies

Machine Learning

User Recognition

Cross Devices

Ease of Messaging

Offline Compatibility

Experience

Page 15: BOT-O-LOGY - Entropia · Facebook's Messenger platform – which explains the reduced emphasis companies seem to place on their chatbot's user experience. The Messenger platform provides

Personality4.3

15

WITH CHATBOTS BEING ONE OF THE FIRST CONSUMER TOUCHPOINTS WHEN IT COMES TO INTERACTING WITH A BRAND, IT'S ENCOURAGING TO SEE BRANDS MAKING SURE THEIR RELEVANCE SHINES THROUGH, EVEN IN CONVERSATION.

When we evaluated Personality among our sampling of chatbots, Brand Relevance scored the highest. With chatbots being one of the first consumer touchpoints when it comes to interacting with a brand, it's encouraging to see brands making sure their relevance shines through, even in conversation.

While the idea of having an AI chatbot as an employee can be new to brands, it is possible to inject a brand's employee criteria into AI-based chatbots. Whether or not the chatbot works well or provides enough information, brand essence should be present in the conversation.

So far, brands have not managed to crack Localisation, which has the lowest scores for Personality. Many brands have opted to use just Standard English for their chatbots due to difficulties in implementing local languages and slang.

Personality

Brand Revelance Character Functionality Localisation Personalisation

Page 16: BOT-O-LOGY - Entropia · Facebook's Messenger platform – which explains the reduced emphasis companies seem to place on their chatbot's user experience. The Messenger platform provides

5.0BEST IN CLASS

Page 17: BOT-O-LOGY - Entropia · Facebook's Messenger platform – which explains the reduced emphasis companies seem to place on their chatbot's user experience. The Messenger platform provides

Mitsuku is a virtual character developed by AIML Technology in Leeds, England and has been active since 2005. She portrays a character of her own, possesses a wide range of knowledge,and has a continuously updated database. From answering random questions to highly specific ones, she can carry out productive conversations.

Thus far, Mitsuku has won many awards, most notably the Loebner Prize in 2013, 2016, 2017, and 2018. She has also been featured in numerous publications like The Wall Street Journal and has been compared to many Smart Intelligent Systems,including Siri and Microsoft’s XiaoIce.

17

FOR OUR EVALUATION, THE CHATBOT MITSUKU SCORED EXCEPTIONALLY WELL ACROSS ALL THREE OF OUR CRITERIA, PARTICULARLY IN RELIABILITY.

Page 18: BOT-O-LOGY - Entropia · Facebook's Messenger platform – which explains the reduced emphasis companies seem to place on their chatbot's user experience. The Messenger platform provides

Reliability5.1

18

THANKS TO HER HUGE DATABASE AND EXCELLENT INFORMATION INTEGRATION, MITSUKU RARELY RUNS OUT OF CONVERSATION TOPICS.

Unlike most chatbots, Mitsuku does not have a series of set commands. Instead, she guides the conversation by initiating topics with the user, making for a more natural flow to the conversation. Even when tested with random brand-related questions, Mitsuku was able to provide detailed answers, sometimes even better than the chatbots created for a specific brand.

Her responses are consistently factually correct and she is able to move from one topic to another with ease, without human intervention. Thanks to her huge database and excellent information integration, Mitsuku rarely runs out ofconversation topics.

Even when she is unable to understand or provide an answer to a user query, Mitsuku is programmed to respond in a fun and entertaining way. Instead of just providing standard responses, Mitsuku's developers and content team balanced cheeky ways of saying "I don't know" with factual responses designed to trigger further conversation.

1 2 3 4 5

Page 19: BOT-O-LOGY - Entropia · Facebook's Messenger platform – which explains the reduced emphasis companies seem to place on their chatbot's user experience. The Messenger platform provides

Experience5.2

19

Chatting with Mitsuku is truly a one-of-a-kind experience. Users can customise their chat interface based on how they want to interact with Mitsuku, whether it's via notepad or an old school mobile phone. Desktop users can even chat with Mitsuku via Flash, which allows for more immersive effects and interfaces.

Mitsuku is accessible across a variety of platforms, from browsers to Telegram, to Twitch group chat, and even Flash. Users can access the chat on whichever platform they are most comfortable with. However, due to her large database of knowledge, Mitsuku cannot chat without a network connection.

MITSUKU IS ACCESSIBLE ACROSS A VARIETY OF PLATFORMS, FROM BROWSERS TO TELEGRAM, TO TWITCH GROUP CHAT, AND EVEN FLASH.

1 2 3 4 5

Page 20: BOT-O-LOGY - Entropia · Facebook's Messenger platform – which explains the reduced emphasis companies seem to place on their chatbot's user experience. The Messenger platform provides

1 2 3 4 5

Personality5.3

20

Mitsuku truly is a character of her own. Although she is unable to score high on Brand Relevance as she does not speak for a specific brand, Mitsuku shows that chatbots can carry a personality of their own.

For Localisation, Mitsuku scores for localising based on how users choose to interact with her. For example, she appears as an anime character in the Anime interface.

Chat with Mitsuku at http://www.square-bear.co.uk/mitsuku/home.html

MITSUKU SHOWS THAT CHATBOTS CAN CARRY A PERSONALITY OF THEIR OWN.

PersonalisationLocalisation

Functionality

Brand Relevance

Character

Page 21: BOT-O-LOGY - Entropia · Facebook's Messenger platform – which explains the reduced emphasis companies seem to place on their chatbot's user experience. The Messenger platform provides

Chat Journey5.4

21

Page 22: BOT-O-LOGY - Entropia · Facebook's Messenger platform – which explains the reduced emphasis companies seem to place on their chatbot's user experience. The Messenger platform provides

6.0LOCAL HEROES

Page 23: BOT-O-LOGY - Entropia · Facebook's Messenger platform – which explains the reduced emphasis companies seem to place on their chatbot's user experience. The Messenger platform provides

THROUGH PLATFORM INTEGRATION, AVA PROVIDES A SEAMLESS CUSTOMER SERVICE JOURNEY QUICKLY AND EFFICIENTLY.

23

Among chatbots developed in Malaysia, AirAsia's AVA placed at the top of the pack. AVA, or AirAsia Virtual Allstar, is part of AirAsia’s efforts to transform the way consumers interact with the brand and ultimately influence the way people book flights. The chatbot was built by AirAsia's software engineering team and customer happiness team using technology built specifically for AVA.

AVA is integrated into AirAsia’s website and mobile application. It allows users to check their flight details, book flights, view information on flight schedules, and select the nearest departure and arrival airports.

Through platform integration, AVA provides a seamless customer service journey quickly and efficiently.

Page 24: BOT-O-LOGY - Entropia · Facebook's Messenger platform – which explains the reduced emphasis companies seem to place on their chatbot's user experience. The Messenger platform provides

1 2 3 4 5

Reliability6.1

24

Customer service centres for airlines are constantly flooded with calls – AVA is designed to help reduce the call load by automating the simplest and most frequent customer queries.

By integrating AVA with AirAsia's internal database, AVA is able to answer frequently asked questions exceptionally well, provide common flight-related queries with updated information, and answer surprisingly specific questions.

Even when AVA is unable to provide an answer, it is programmed to direct the user to a human customer representative who will be able to assist. In most cases though, AVA managed to handle queries satisfactorily.

AVA IS DESIGNED TO HELP REDUCE THE CALL LOAD BY AUTOMATING THE SIMPLEST AND MOST FREQUENT CUSTOMER QUERIES.

Page 25: BOT-O-LOGY - Entropia · Facebook's Messenger platform – which explains the reduced emphasis companies seem to place on their chatbot's user experience. The Messenger platform provides

1 2 3 4 5

Experience 6.2

25

Chatting with AVA is a pleasant experience, which contributed to its high scores in this parameter. Users are very likely to have their queries addressed within the chat itself, thus minimising redirection to other sites.

However, despite integration done by Air Asia, it does not personalise communication towards each of its customers. Through a deeper integration with its first-party data, personalisation could be done easily and this would provide an even better experience to its customers.

Due to its deep integration with the AirAsia database, AVA cannot exist outside of AirAsia’s digital ecosystem. Users can only access the chatbot on AirAsia's website and mobile app when they are connected to the internet – this means users cannot access AVA on Facebook, one of the most common and accessible platforms. For AVA, AirAsia prioritised data integration and chat experience over accessibility.

FOR AVA, AIRASIA PRIORITISED DATA INTEGRATION AND CHAT EXPERIENCE OVER ACCESSIBILITY.

Page 26: BOT-O-LOGY - Entropia · Facebook's Messenger platform – which explains the reduced emphasis companies seem to place on their chatbot's user experience. The Messenger platform provides

1 2 3 4 5

Personality6.3

26

AirAsia is known for their strong brand voice, which they have successfully translated into their chatbot Personality. For AVA, AirAsia injected their signature stewardess hospitality – fun, bold, and helpful, AVA comes to life as a digital assistant with a pleasant personality.

If there is anything more we can ask for, it is for AVA to talk less like a digital assistant and more like a human being. With more character and personal flair, users can be driven to chat with AVA beyond asking questions about flights.

FOR AVA, AIRASIA INJECTED THEIR SIGNATURE STEWARDESS HOSPITALITY – FUN, BOLD, AND HELPFUL, AVA COMES TO LIFE AS A DIGITAL ASSISTANT WITH A PLEASANT PERSONALITY.

1 2 3 4 5

PersonalisationLocalisation

Functionality

Brand Relevance

Character

Page 27: BOT-O-LOGY - Entropia · Facebook's Messenger platform – which explains the reduced emphasis companies seem to place on their chatbot's user experience. The Messenger platform provides

Chat Journey6.4

27

Page 28: BOT-O-LOGY - Entropia · Facebook's Messenger platform – which explains the reduced emphasis companies seem to place on their chatbot's user experience. The Messenger platform provides

7.0GROWING UP

Page 29: BOT-O-LOGY - Entropia · Facebook's Messenger platform – which explains the reduced emphasis companies seem to place on their chatbot's user experience. The Messenger platform provides

29

ADAM CAN RESPOND TO FAQS AND QUERIES ABOUT ONGOING PROMOTIONS, AS WELL AS CHAT ABOUT PLANNING, PREGNANCY, AND NUTRITION WITH CMC’S TARGET AUDIENCE:CONNECTED MUMS.

ADAM - THE CONNECTED KIDAdam - The Connected Kid is another Malaysian chatbot developed for the Connected Mums Club website. This NLP-based chatbot is updated frequently and regularly, with the ability to provide relevant information and carry out casual chats with mothers.

Adam can respond to FAQs and queries about ongoing promotions on the website, as well as chat about planning, pregnancy, and nutrition with CMC’s target audience: connected mums. For now, users can only chat with Adam on the CMC website, as the chatbot is integrated into the CMC website database.

Adam was designed with the personality of an energetic 4-year-old to make him more appealing and relatable. As a result, connected mums seem to love chatting with Adam, with 90% of conversations registering as positive.

Chat with Adam here:https://www.connectedmums.com.my/

Sentiment analysis is based on sample data from Jan-May’19

93%

7%

Sentiment

Positive Neutral Negative

Page 30: BOT-O-LOGY - Entropia · Facebook's Messenger platform – which explains the reduced emphasis companies seem to place on their chatbot's user experience. The Messenger platform provides

8.0LEAST DESIRABLE

Page 31: BOT-O-LOGY - Entropia · Facebook's Messenger platform – which explains the reduced emphasis companies seem to place on their chatbot's user experience. The Messenger platform provides

31

The Endurance chatbot was developed primarily to be a virtual companion to the elderly - it is supposed to ask questions, react to answers, and share interesting information from Google. Its programming included conversations on general topics like the weather, hobbies, movies, music, the news, nature, and more.

Reliability8.1

THE ENDURANCE CHATBOT PERFORMED POORLY ACROSS ALL THREE CRITERIA, PARTICULARLY IN RELIABILITY AND EXPERIENCE.

However, in our evaluation, we found the Endurance chatbot performed poorly across all three criteria, particularly in Reliability and Experience. This is not ideal, particularly for a chatbot designed to interact with elderly users.

The Endurance chatbot does not house frequently asked questions on its database. Users are instead frequently redirected to an external webpage in response to most queries – an indication that the

chatbot does not have a coherent information database integrated into its systems.Even though its stated function is to assist the elderly, the Endurance chatbot mostly pushes Endurance products and services, without offering much value in return. It asks highly detailed questions when users attempt to make an order without providing any information in return. More often than not, it was easier to make orders via the website instead.

1 2 3 4 5

Page 32: BOT-O-LOGY - Entropia · Facebook's Messenger platform – which explains the reduced emphasis companies seem to place on their chatbot's user experience. The Messenger platform provides

Experience 8.2

32

Chatting with the Endurance chatbot is quite an unpleasant experience, as seen in its low Experience scores across the board. The interface is poorly designed and cluttered, to the point where even the spaces between chat texts are inconsistent and under optimised.

Without properly guided conversation trails, chatting with the Endurance chatbot comes across as unfriendly and inorganic. To access the guided menu, users have to refresh the webpage and initiate a new chat every time they want to talk about a new topic.

WITHOUT PROPERLY GUIDED CONVERSATION TRAILS, CHATTING WITH THE ENDURANCE CHATBOT COMES ACROSS AS UNFRIENDLYAND INORGANIC.

1 2 3 4 5

Page 33: BOT-O-LOGY - Entropia · Facebook's Messenger platform – which explains the reduced emphasis companies seem to place on their chatbot's user experience. The Messenger platform provides

1 2 3 4 5

PersonalisationLocalisation

Functionality

Brand Relevance

Character

Personality8.3

33

Unfortunately, a human avatar picture does almost nothing to give the Endurance chatbot points in Personality. The chatbot does not seem to have a character of its own and randomly switches to different modes mid-chat. Chatting with the Endurance chatbot feels no different from interacting with basic applications.

The chatbot does attempt to localise for audiences in different countries by integrating Google Translate into the chat. However, Google Translate does not always pick up on local nuances and slang, making non-guided conversations almost impossible.

Chat with the Endurance chatbot here: http://endurancerobots.com/azbnmaterial/a-robot-companion-for-senior-people-and-patients-with-alzheimer-s-disease/

UNFORTUNATELY, A HUMAN AVATAR PICTURE DOES ALMOST NOTHING TO GIVE THE ENDURANCE CHATBOT POINTS IN PERSONALITY.

Page 34: BOT-O-LOGY - Entropia · Facebook's Messenger platform – which explains the reduced emphasis companies seem to place on their chatbot's user experience. The Messenger platform provides

9.0CHATBOTPLATFORMS

Page 35: BOT-O-LOGY - Entropia · Facebook's Messenger platform – which explains the reduced emphasis companies seem to place on their chatbot's user experience. The Messenger platform provides

Bot Name Features Technical details

-IBM Watson ConversationService

35

Although Facebook is the most popular platform to showcase how easy it is to get a chatbot to work, there are numerous other platforms that could work even better, depending on the requirements of the developer. This section explores some of the more popular options:

• Built on a neural network (one billion Wikipedia words).• Has three main components: Intents, Entities, Dialog

Pypestream • Pypestream uses a patented framework of ‘Pypes’ and ‘Streams’.• Natural Language Processing and keyword parsing.

AgentBot • Understands natural language.• Memory to maintain coherence during long conversations.• Gathers customer information to deliver customized solutions.• Continuous evolution.• Clarifies intent.

Integrates with any CRM, internal system, human chat and third-party application.

Msg.ai • Intent models and tone classifications• Deep learning• Interactive Smart Cards• A/B Testing

-

DigitalGenius • AI predicts case meta-data and suggests the right answers to your agents.• AI learns from every agent interaction.• Uses a deep neural network model, word vectors, statistical operations.• Deep learning algorithms.

Human + AI Customer Service is integrated directly into the contact center’s existing software.

Semantic Machines • Conversation Engine• Speech Synthesis• Deep Learning• Reinforcement Learning• Speech Recognition• Goes beyond understanding commands to understanding conversations.• Extracts semantic intent.• Uses the Conversation Engine's natural language generation (NLG) technology.

-

Twyla • Learns from agent/customer live chats.• Blends machine learning and rule-based methods.• Answers questions, deflects tickets.

Integrated with most major cloud Helpdesk and Live Chat solutions, like Zendesk, Salesforce and Liveperson.

An open and flexible API platform allows for custom integrations and development of 3rd party connectors, plugins and extensions.

Live Agent - -

Octane.ai • Real-time analytics -

Page 36: BOT-O-LOGY - Entropia · Facebook's Messenger platform – which explains the reduced emphasis companies seem to place on their chatbot's user experience. The Messenger platform provides

36

Bot Name Features Technical details

wit.ai Allows to use:• Entities• Intents• Context• Actions• Natural Language Process (NLP)

Microsoft Bot Framework • Understands the user’s intent.• To give your bot more human-like senses, you can incorporate LUIS for natural language understanding, Cortana for voice, and the Bing APIs for search.

rasa NLU • Intent classification• Entity extraction

You can use rasa as a drop-in replacement for wit, LUIS, or api.ai

Is available for developers to use with iOS, Android, Windows Phone, Raspberry Pi, Python, C and Rust.

JavaScript plugin.

Pandorabots • AIML (Artificial Intelligence Markup Language) • Includes A.L.I.C.E.

The Pandorabots API allows you to integrate our bot hosting service and natural language processing engine into your own application.

Microsoft LanguageUnderstanding IntelligentService (LUIS)

• Uses intents and entities.• All LUIS applications are centered around a domain-specific topic or content related.• Active learning.• You can use pre-existing, world-class, pre-built models from Bing and Cortana.• Deploy models to an HTTP endpoint with one click. LUIS returns easy-to-use JSON.

LUIS offers a set of programmatic REST APIs that can be used by developers to automate the application creation process.

Chatfuel • A block is a basic building block of your bot. It consists of one or more message cards that are sent together to a bot user.• You can link blocks with each other using buttons in text cards or in gallery cards.

-

ChatterBot • The program selects the closest matching response by searching for the closest matching known statement that matches the input, it then chooses a response from the selection of known responses to that statement.

http://chatterbot.readthed-ocs.io/en/stable/#process-flow-diagram

Api.ai • API.AI matches the query to the most suitable intent based on information contained in the intent (examples, entities used for annotations, contexts, parameters, events) and the agent's machine learning model. • API.AI transforms the query text into actionable data and returns output data as a JSON response object.• Leverage predefined knowledge packages collected over several years.

-

The framework provides the Direct Line REST API, which you can use to host your bot in an app or website.

Page 37: BOT-O-LOGY - Entropia · Facebook's Messenger platform – which explains the reduced emphasis companies seem to place on their chatbot's user experience. The Messenger platform provides

37

Bot Name Features Technical details

Rebot.mez • The platform allows you to create a chatbot and train it by entering questions and corresponding answers. • It also offers a chat log so that you can see the results of previous conversations.

ManyChat • The basic functionality lets you welcome new users, send them content, schedule posts, set up keyword auto-responses (text, pictures, menus), automatically broadcast your RSS feed and much more.

-

-

Reply.ai • Visual Bot builder• Easily leverage Natural Language Processing (NLP) engines wit.ai and api.ai for advanced use cases• Analytics

ChatScript • Powerful pattern matching• Simple rule layout• C-style general scripting• Built-in WordNet dictionary• Extensive extensible ontology• Data as fact triples enables inferencing and supports JSON representation• Rules can examine and alter engine and script behavior• Remembers user interactions

• Ability to control local machines via popen/tcpopen/jsonopen.• Ability to read structured JSON data from websites.• Postgres and Mongo databases support for big data or large-user-volume chatbots. ChatScript runs on Windows, Linux, Mac, iOS, or Android• It has integrated tools to support maintaining and testing large systems.

KITT.AI • Hotword Detection (no internet required)• Semantic Parsing• Natural Language • Understanding • Conversational Engine (multi-turn)• Neural network powered machine learning model

LUIS offers a set of programmatic REST APIs that can be used by developers to automate the application creation process.

It's Alive • Keyword detection• Recipes• RSS Feeds• Variables• Subscriptions• Instant notification• Schedule• Manage & organize• Activity • Analytics

The team believes in decision trees and buttons that drive users towards the answer they are looking for.

Gubshup • A built-in wit.ai integration for Natural Language Processing• Templates available for popular business categories.

IDE Bot Builder for developersFlow Bot Builder for non-developers

-

FlowXO - -

Page 38: BOT-O-LOGY - Entropia · Facebook's Messenger platform – which explains the reduced emphasis companies seem to place on their chatbot's user experience. The Messenger platform provides

10.0SUMMARY

Page 39: BOT-O-LOGY - Entropia · Facebook's Messenger platform – which explains the reduced emphasis companies seem to place on their chatbot's user experience. The Messenger platform provides

39

IN MALAYSIA, CHATBOT TECHNOLOGY IS STILL RELATIVELY NEW, BUT PROGRESSIVE COMPANIES, ALWAYS EAGER TO INNOVATE AND AUTOMATE, HAVE ADOPTED CHATBOTS AS AN EFFICIENT ALTERNATIVE TO IMPROVE THEIR CUSTOMER COMMUNICATIONS.

After over five decades in existence, consumers have become more accustomed to chatbots, with companies proactively engaging the technology. In Malaysia, chatbot technology is still relatively new and the user experience oftentimes frustrating, but progressive companies like AirAsia, always eager to innovate and automate, have adopted chatbots as an efficient alternative to improve their customer communications.

And as we are all acutely aware, game changers in technology come hard and fast. While the ROI on complex AI bots may prove a step too far at this moment in time, we need to remain mindful that we are on the cusp of normalising a new way to interact with digital systems – even with the most basic execution.

This may sound like cutting out the heart of creative execution, but there is still a lot to be learnt from even the simplest bots. Our ability to effectively A/B test creative execution, tone of voice, and extract information of contextual relevance to the user, provides us with a deeper understanding of the customer journey and emotional lifecycle. From here, effective and contextual content andconversation can be successfully mapped, programmed, and learned from.

As the report shows, there are key criteria that make some chatbots more notable than the rest. To create a great chatbot experience, brands and developers must keep these in mind:

Are continuously improvingBrands should consider chatbots an ongoing investment. A chatbot is only as smart as the information it has access to, yet we found that many chatbots are not even integrated into the brand's information database. Chatbots shouldbe regularly updated to include new customer information; and improvements should be made to the way it handles natural language queries.

Go beyond FAQsToo many chatbots today exist purely as FAQ machines. Plenty of users prefer to bypass automated responses to get to a human customer representative because they feel like they can ask more specific and personalised questions.Chatbots should be prepared to interact and respond to users naturally, beyond scripted queries, to satisfy a broader range of user queries.

Represent their brand wellBrands want chatbots to be almost indistinguishable from their human customer service representatives. Although most brands know enough to get the tone of voice right, chatbots today need to be integrated into the company's ecosystem to provide personalised responses according to individual needs and circumstances, minimising redirection to human representatives and external sites.

Make themselves availableGreat chatbots connect with users the way their friends do. Making a chatbot available on the users' most preferred platforms will naturally facilitate more interactions and queries. Chatbots that incorporate new accessibility technologies like voice-to-text and image recognition will also be able to enhance the way brands and customers communicate.

Great chatbots

Page 40: BOT-O-LOGY - Entropia · Facebook's Messenger platform – which explains the reduced emphasis companies seem to place on their chatbot's user experience. The Messenger platform provides

The advent of deep data and marketing technology has caused massive chaos and complexity in the marketing ecosystem. Marketing lives at the cusp of business and people - the most powerful insights and ideas can be fragile and elusive. Complexity can kill the soul of marketing. At the very least, it can take focus away from what truly matters.

Brands must enhance human happiness, and so must data and technology. In the age of anticipatory data, creative prototyping, and curative technology, there is a historic opportunity for brands to be different. And to make it simple. Entropia hopes to chance upon ways we can do that - with simplicity, consistency, and scale.

We hope to bring to the play an agile, hybrid, and ecosystem approach that is native to the digital age - made in future. In the process, we hope to break a few norms and call a spade a spade, even at the risk of being wrong.

About

40

Page 41: BOT-O-LOGY - Entropia · Facebook's Messenger platform – which explains the reduced emphasis companies seem to place on their chatbot's user experience. The Messenger platform provides

Copyright © 2019 Entropia Pte. Ltd. All rights reserved.

Write [email protected]

Check outwww.entropia.com