Building blocks for building bots

PowerPoint Presentation

Rita ZhangOpen Source Engineer@ritazzhangBuilding Blocks for Building Bot

Bhargav NookalaOpen Source Engineer@bhargav

BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)Introduction: oss engr, sf, working closely with the product team to provide a better framework and services for developers like yourself, building bots with startups and enterprises, share my experience

1

By the endWhen is a bot a good idea

Available tools and services

Build your own bot

BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)Agenda: what are bots, when to build a bot, many options, but here are some tools and services from msft you can use to build your own bots, last lets look at how we developed a few bot examples, including bots that leverages payment gateways and cognitive servicesExec Retreat 2016

2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.2/22/17 3:24 AM2

Bots areBots are notHave been around for a long timeNew interface to engage with usersTask automation when possibleMore InteractiveEasily accessibleMore naturalReplacing humansWe do not need a bot for everythingA catch allReplacing existing solutions

BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)Agenda: what are bots, when to build a bot, what are some of the available tools you can use to build your own bots

Bots have been around: clippy, IRC bots, MSN, ICQIts just a new way to engage users; it used to be flat, e.g. forms to get information from users. but now, its more interactive and more customized and more natural ways, like speaking, listening, looking into a camera

Slackbot?Exec Retreat 2016


People

Digital assistants

Bots

Conversationsas a PlatformHuman language is the new UIBots are the new apps;digital assistants are meta appsIntelligence infused into all interactionsBots are just another form of user interface

BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)

Exec Retreat 2016


So When is a Bot a Good Idea?QnA, Knowledge base, SearchWhen text input is not enoughWhen productivity can be increased by listening in on group conversationsWhen user engagement can be maximized using the apps they already love

BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)Bot framework aims to unify existing bot development.Write once, connect to many things

5

www.botframework.com

BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)Bot Framework provides everything you need to build and connect intelligent bots that interact naturally wherever your users are talking, from text/sms to Skype, Slack, Office 365 mail and other popular services.

Bot Framework consists of three main components: Bot Connector, Bot Builder, and Bot Directory

6

Bot Framework provides what you need to

BuildConnectManagePublish

BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)Bot framework aims to unify existing bot development.Write once, connect to many things

7

Benefits

For developersEasiest way to reach the broadest set of users where they already are conversingBots are more capable because of supporting services (translation, profile, history, memory, etc.)Bring your own bot or build your own bot with the Bot Builder SDKsFor end usersUsers can choose from a variety of conversation channelsUsers have trust and control of their dataFor businessesBroad access to their customers, new experiencesReduced cost of developmentwww.botframework.com


Your Bot Framework Bot

Bot ConnectorBot DirectoryBot Builder SDKsConnect your bot(s) to text/sms, Office 365 mail, Skype, Slack, and other services.Build great dialogs within your Node.js or C# botTry, use, and add published bots to the worlds top conversation experiencesRegister, connect, publish and manage your bot through the bot dashboardMessage routingAutomatic translation to 30+ languagesUser and state managementEmbeddable web chat controlDebugging toolsOpen source SDK on GithubFrom simple built-in prompts and command dialogs to simple to use yet sophisticated FormFlow dialogsLibraries, samples and tools to make a great conversationalistChat emulatorLeverage related services available in Cognitive ServicesPublic directory of bots registered and approved with Bot FrameworkUsers can try your bot from the directory via the web chat controlUsers can discover and add your bot to the channels on which it is configuredBot Framework Components

BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)Your conversation logicLogicWeb ServiceYour BotBot Builder SDK(Node.js / C#)LUISBuild with C# or Node.jsYou host your botDialogs to model a conversationCognitive Services:Natural Language Understanding (LUIS) to get user intentsFace DetectionYour Bot


10

Purpose:forward messages from your Bot to a userand will send user messages to your Bot

Single API across multiple client services: Skype, Email, Slack,

JSON schema

Use Emulator for local debug

Developer Dashboard for testing connection onlineBot Framework Connector Service


11

Bot Connector Basics

Interact naturally wherever yourusers are talking

BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)Register the bot

Configure desired channels

Publish in the Bot Directory.

All bots registered with Bot Framework are auto-configured to work with Skype and the Web.

Developer Portal


13

Node.js and C#Dialogs to model a conversationDialogs are reusableConversations are scalable to multiple machinesDialog state is persisted in Bot Connector (per-user, per-session, per- user+session)Types of Dialogs:Built-in promptsYes/No, String, Number, ChoicesNatural language understanding (LUIS)Form slot filling (branching, disambiguation, multi-turn)Profile (e.g., home address)

Bot Builder SDKs


14

Public Directory of Bot Framework BotsUsers can discover, try, and add bots to the conversation experiences on which the bot is configured (no app required)Bots are public at developer discretion; bots must be submitted for review and approved in order to appear in the directorySearchable

Bot Directory

BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)Make your bot smarter using various cloud based servicesVisionSpeechLanguage Understanding (LUIS)KnowledgeSearch

https://www.microsoft.com/cognitive-services/Intelligent Bots Have Microsoft Research at your finger tips

BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)Demos

BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)ResourcesFace Detection bot - https://github.com/ritazh/facedetect-botBotBuilder SDK - https://github.com/Microsoft/BotBuilderCognitive Services - https://www.microsoft.com/cognitive-services/Speech Driven app - https://github.com/ritazh/speech-to-text-demoAuth bot - https://github.com/catalystcode/node-authbotPayment bot - https://github.com/bnookala/node-paymentbot

BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)FaceDetect bot face recognitionHuma.ai LUIS, Bing Speech, CRISAuthbot Azure AD authentication and get latest emailPaypalbot payment

Singapore botfest, many agenciesSkype and text messageThe defense agency created a bot that helps National Service enlistees explore various phases of their National Service careerconnects new enlistees with other members of their team based on common interestshelps the enlistee to re-connect with their former teammates (via face detection) after leaving their 2-year training period

18

Rita ZhangOpen Source Engineer@ritazzhangQuestions

Bhargav NookalaOpen Source Engineer@bhargav

BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)End slide

19

Determine intent and detectsentitiesSeamless integration with Speech RecognitionLearns over timeUse pre-built, world class models from Bing and CortanaModels work across devices Language Understanding (LUIS)Intent: TextToSpeechContent: headlinesRead me the headlines

Play yesterdays Daily Show

Intent: PauseDeviceDateTime.duration: 5 MinutesPause for 5 minutes

Intent: PlayEpisodeContent: Daily ShowDateTime.date: T-1 Day

LUIS helps you understand what your users are saying (e.g. turn on the lights, news about flight delays) Determines intent and detects entitiesSeamless integration with Speech RecognitionLUIS learns over timeUses pre-built, world class models from Bing and CortanaModels work across devices (phone, tablet or other Internet connected device)

For media, this might work like this:read me the headlinesIntent: TextToSpeechContent: NewsHeadlinesplay last nights Tonight ShowIntent: PlayEpisodeContent: TonightShowDate: T-1Daypause for 5 minutesIntent: PauseDeviceDevice: MediaPlayerDateTime.duration: 5M

2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.2/22/17 3:24 AM20

Building blocks for building bots

Software

Transcript of Building blocks for building bots