Future of the Web with Conversational Interface
-
Upload
tomomi-imura -
Category
Technology
-
view
4.588 -
download
0
Transcript of Future of the Web with Conversational Interface
![Page 1: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/1.jpg)
Future of the Web with Conversational Interface
BOTS, AI, AND JAVASCRIPT
Tomomi Imura (@girlie_mac)#ModernWeb2017
![Page 2: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/2.jpg)
@ girlie_mac
● advocate open web & technology● code JavaScript & Node.js● write & speak about tech● hack useless stuff● dev relations at Slack● live in foggy San Francisco
tomomi imura
![Page 3: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/3.jpg)
"Bots are like new applications that you can converse with. "-- Satya Nadella, Microsoft
![Page 4: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/4.jpg)
@ girlie_mac
“We will evolve in computing from a mobile first to an AI first world.”-- Sundar Pichai, Google
![Page 5: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/5.jpg)
@ girlie_mac CC BY-SA: nextdayblinds.com
![Page 6: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/6.jpg)
@ girlie_mac
Traditional Web & App Interactions
![Page 7: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/7.jpg)
@ girlie_mac
Modern Web & Apps with Social Interactions
![Page 8: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/8.jpg)
@ girlie_mac
Conversational User Interactions: Siri and Alexa (Voice Assistants)
Alexa, how is the weather?
Hey Siri
In various form-factors
![Page 9: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/9.jpg)
@ girlie_mac
Conversational User Interactions for Kids - with Voice
CogniToys Dino - https://cognitoys.com
![Page 10: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/10.jpg)
@ girlie_mac
Conversational User Interactions in a robot shape - with Voice
![Page 11: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/11.jpg)
@ girlie_mac
![Page 12: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/12.jpg)
@ girlie_mac
Conversational User Interactions: Goole Assistant (Voice & Text)
![Page 13: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/13.jpg)
@ girlie_mac
Conversational User Interactions: SlackBots (Text)
!
![Page 14: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/14.jpg)
Slack Integrations & Bots for Better Productivity
+
+
+
![Page 15: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/15.jpg)
Graphic Interface to Conversational Interface
![Page 16: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/16.jpg)
@ girlie_mac
Deliver me a large margherita pizza!
What kind of crust do you want?1. Regular crust2. Thin crust3. Gluten free crust
Thin crust
![Page 17: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/17.jpg)
Natural user interactions with a minimal visual interface
![Page 18: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/18.jpg)
No UI ClutterLess Time Spent
![Page 19: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/19.jpg)
@ girlie_mac
Where is the address of the place?
It’s 325 Pearl Street. Come on now!
Request a ride
![Page 20: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/20.jpg)
@ girlie_mac
Yes, get me a ride now
Your driver, Sam will pick you up in 6 minutes. Look for the red Toyota Prius!
![Page 21: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/21.jpg)
Old concepts.
Technology caught up with the ideas.
![Page 22: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/22.jpg)
@ girlie_mac
“I'm sorry Dave, I'm afraid I can't do that”
![Page 23: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/23.jpg)
@ girlie_mac
“I'm sorry Dave, I'm afraid I can't do that”
![Page 24: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/24.jpg)
Conversational Interface is:
![Page 25: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/25.jpg)
● Intuitive● Accessible● Productive
![Page 26: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/26.jpg)
@ girlie_mac
Messaging Platforms
● Slack● Facebook Messenger● Telegram● WeChat● Kik● Viver● LINE etc.
![Page 27: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/27.jpg)
Messaging + Bots for More Interactive Communications
![Page 28: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/28.jpg)
Slackbots at
![Page 30: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/30.jpg)
TacoBot by Taco Bell
https://www.tacobell.com/feed/tacobot
![Page 31: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/31.jpg)
@ girlie_mac
Chatbot Building APIs
● API.ai (Google)● Wit.ai (Facebook)● Microsoft Bot
Framework● Motion.ai● Converse AI
● Chatbots.io● Recast.ai● Flow XO● ManyChat● Fluent.ai (Voice UI)
etc.
![Page 32: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/32.jpg)
AIaaS
Motion.ai
API.ai
Artificial Intelligence as a Service
![Page 33: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/33.jpg)
@ girlie_mac
Hi, I want to book a flight!
Yes, from SFO.
Where are you flying to?
Taipei
Hi Linda! Are you flying from San Francisco, as usual?
An airline customer service bot
Linda may not know she is talking to a bot!
![Page 34: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/34.jpg)
@ girlie_mac
NLP & Conversational Platforms / APIs
Natural Language Processing & Cognitive platforms:
● IBM Watson● Google Cloud Natural Language API● Microsoft LUIS● Amazon Lex● Baidu UNIT
![Page 35: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/35.jpg)
Build Your Own Conversational Interface
![Page 36: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/36.jpg)
...with JavaScript
![Page 37: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/37.jpg)
@ girlie_mac
{ REST }
JSSDK
The APIs are mostly accessible with JS
![Page 38: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/38.jpg)
@ girlie_mac
Example: API.ai + FB Messenger
const app = require('apiai')(CLIENT_ACCESS_TOKEN);
function sendMessage(event) {
let sender = event.sender.id;
let text = event.message.text;
let ai = app.textRequest(text, {
sessionId: SESSION_STRING });
ai.on('response', (response) => {
// Got a response. Let's POST to Facebook Messenger
});
ai.end();
}
API.ai Node.js SDK
![Page 39: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/39.jpg)
@ girlie_mac
Example: IBM Watson + Slack
![Page 41: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/41.jpg)
@ girlie_mac
Example: IBM Watson + Slack
The bot workflow:
1. Read each message on a Slack channel 2. Send the message to IBM Watson for examination 3. If the likelihood of an emotion is above the given
confidence threshold post the most prominent emotion
![Page 42: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/42.jpg)
@ girlie_mac
Example: IBM Watson + Slack
Slackbot Config:
1. Set up a bot user2. Set OAuth permission
scopes 3. Subscribe a bot event, so
the bot can post messages on Slack channels
IBM Bluemix Config for Watson
![Page 43: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/43.jpg)
@ girlie_mac
Example: IBM Watson + Slack
app.post('/events', (req, res) => {
let q = req.body;
if (q.type === 'event_callback') {
if(!q.event.text) return;
analyzeTone(q.event);
}
});
Use Slack Events API to grab the text when a user post a message
Pass the text data to Watson to analyze
HTTP POST using ExpressJS
![Page 44: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/44.jpg)
@ girlie_mac
Example: IBM Watson + Slack
const watson = require('watson-developer-cloud');
let tone_analyzer = watson.tone_analyzer({
username: process.env.WATSON_USERNAME,
password: process.env.WATSON_PASSWORD,
});
const confidencethreshold = 0.55;
tone_analyzer.tone({text: text}, (err, tone) => {
tone.document_tone.tone_categories.forEach((tonecategory) => {
if(tonecategory.category_id === 'emotion_tone'){
tonecategory.tones.forEach((emotion) => {
if(emotion.score >= confidencethreshold) {
postEmotionOnSlackChannel(emotion);
}});
}});
});
Returns emotions score in0 to 1
Just initializing it w/ your API credentials
Post the result on Slack
![Page 45: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/45.jpg)
@ girlie_mac
Example: IBM Watson + Slack + Raspberry Pi (for fun)
function colorEmotion(emotion) {
if (emotion.tone_id === 'anger') {
setLED(red);
} else if(emotion.tone_id === 'joy') {
setLED(yellow);
} else if(emotion.tone_id === 'fear') {
setLED(purple);
} else if(emotion.tone_id === 'disgust') {
setLED(green);
} else if(emotion.tone_id === 'sadness') {
setLED(blue);
}
} Change the LED color to match the emotion
![Page 46: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/46.jpg)
@ girlie_mac
Ack, this sucks! I want my money back!
An angry customer detected. Connect the customer with a human!
![Page 47: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/47.jpg)
Conversational Interface with Voice in Browser?
![Page 48: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/48.jpg)
@ girlie_mac
Hello!1
2
Howdy
3rd Party NLP API for artificial conversations
Voice command:Voice -> Text 3
Text -> Synthetic Voice
Text
![Page 49: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/49.jpg)
@ girlie_mac
Web Speech API
Speech Recognition & Speech Synthesis
http://caniuse.com/#feat=speech-recognitionhttp://caniuse.com/#feat=speech-synthesis
1 3
![Page 50: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/50.jpg)
@ girlie_mac
Web Speech API: Speech Recognition
const SpeechRecognition =
window.SpeechRecognition || window.webkitSpeechRecognition;
const recognition = new SpeechRecognition();
Get an instance of the SpeechRecognition, the controller interface
In the current Chromium, it is still prefixed
![Page 51: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/51.jpg)
@ girlie_mac
Web Speech API: Speech Recognition (Cont’d)
recognition.lang = 'en-US';
recognition.interimResults = false;
recognition.start();
recognition.addEventListener('result', (e) => {
let last = e.results.length - 1;
let text = e.results[last][0].transcript;
});
Some properties
Methods: start(), stop(), abort()
Events: onresult, onerror, onaudiostarted, onaudioend, etc.
![Page 52: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/52.jpg)
@ girlie_mac
Web Speech API: Speech Synthesis
const synth = window.speechSynthesis;
const utterance = new SpeechSynthesisUtterance();
utterance.text = 'I am a robot';
utterance.pitch = 1.5;
utterance.voice = 'Alex';
synth.speak(utterance);
No vendor prefix
Properties of the SpeechSynthesisUtterance interface
Get available voices with synth.getVoices() method
![Page 53: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/53.jpg)
@ girlie_mac
![Page 55: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/55.jpg)
@ girlie_machttps://www.smashingmagazine.com/2017/08/ai-chatbot-web-speech-api-node-js/
![Page 56: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/56.jpg)
@ girlie_mac
![Page 57: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/57.jpg)
The future was here already.
Long Live Clippy!
1997 - 2007
![Page 58: Future of the Web with Conversational Interface](https://reader033.fdocuments.net/reader033/viewer/2022050613/5aaceb887f8b9a1f368b486f/html5/thumbnails/58.jpg)
@ girlie_mac
謝謝 !@girlie_mac
girliemac.com
github.com/girliemac
slideshare.net/tomomi