AliMe Bot Platform - LMS | Lab for Media Search at...

AliMe Bot Platform- Alibaba’s Personal � Intelligent � Assistant �

Alibaba Group � – Intelligent Innovation Division

——Haiqing Chen(Hunter)

Outline

n AliMe Bot Platform Introduction

n Intelligent � Interaction � Technical � Practice

n Look � Forward � to � The � Future

AliMe Bot Platform Introduction

Customer Service

Self � Support Outsource Cloud �

Service

Customer Service

Self � Support

Outsource

Cloud � Service

ChatbotPlatform

Platform

3rd

party AlibabaMerchantEnterprise

n Upgrading � of � service: � From � manpower � to � intelligence � plus � manpowern Upgrading � of � field: From � Ali � inside � to � merchants and � enterprises


AliMe bot � platform is a complete set of intelligent human-computerinteraction solution.

• Face to three domains solutions:• AliMe Assistant: Platform solution for Alibaba business• TiMi: Platform solution forMerchants of Taobao• Bee Bot: Platform solution for enterprises, ISVetc.

• Supporting two types of infrastructures:• SaaS: Front to end solution, a whole chatbot system includingchat UI• PaaS: SupplyAI interfaces capability for developers to help build their

systems

ConfigurationPlatform Chatbot QA Platform Smart Knowledge

Base AI Boost

Chatbot Application Platform

AliMe Assistant Platform

Different Domains of AliMe AssistantsTiMi

Alibaba Merchant

Taobao Tmall

SaaS

Enterprise

Super AliMeAssistant

Bee Bot Platform

WangXiang Travel

Second-hand

Other

B2B Logistics

Youku Other

SaaS PaaS

QianniuPlatform

DingTalk

AliCloud IOT

SaaS

Bot Framework

Algorithm Component Platform Data Platform

Oversea

SaaS or PaaS



AliMe Assistant

• Covered Areas:• Customer Service• Shopping Guide• General Assistant

Application• Chit Chat• Activity Operation• ……


TiMi

• Features:• General ontology

model for all categories: logistics field

• Industry ontology model: mobile phone, clothing etc.

• General QA model


Bee Bot Platform

• Features:• This is a whole solution

for 3rd party, i.e.enterprise, ISV etc.

• Smart knowledgebaseto fullfill knowledges

Outline




Intelligent � Interaction � Technical � Practice

AliMeAssistant TiMi Bee Bot

Platform

PaaS Service Layer

Dialog Management System

Matching Model Bot Framework

Algorithm Component Platform

ConfigurationPlatform

Knowledge � Base AI Boost

AliMe Bot Platform Application � Architecture


Intent Routing

CustomerService

ShoppingGuide

LogisticsField

ChitChat

others

ImageRecognition

Muti-turn Interaction

ASRSemantic � RecognitionUser Profile

Muti-model Interaction Recommendation

Knowledge(QA pairs, Ontology, Knowledge Graph)

Data AnalyticsData MiningML/DL Training

Routing Layer

Service Layer

Model Layer

Data Layer

AliMe Bot Platform Algorithm Model � Architecture


Query+Context

Intention Recognition

Dialog Management System

QA Bot Task Bot Chat Bot

KowledgeGraph IR Slot Filling Bot

FrameworkDRL IR+LSTM


Intention Identification Process

QuerySegmentation,POS Tagging,

NER

IntentionClassification

DialogManagement

SemanticRepresentation

IntentionAttribute

Extraction

DomainModel

ContextModel


Intention Identification Classification

• Traditionalmachine Learningmethods :• Supervised multiply classification• Supervised multiply binary classification• E.g. Bayes, KNN, SVM, Logistic Regression etc.

• Deep Learning:• Combined user profile or behavior to build deep learningprediction

model• E.g. CNN, DNN, LSTM etc.


Enrich Question with Profile, Behavior and Context

Question User Behavior

User � Profile

Query+ Context

DL Model


Intention � Identification � with Deep Learning

N V

Question(BOW/RNN/CNN)

Behavior,Profile, Context

L

Embedding

Softmax

N V

Question(BOW/RNN/CNN)

Behavior,Profile, Context

0/1

Embedding

0/1 0/1 0/1 Logistic Regression (LR)

All labels L1 L2 L3 ……

Multiple Classification(DNN 2-channel inputs)

Multiple Binary Classification(DNN 2-channel inputs)


Clas

s2vec

Intent rule

Intetionmodel

Senrepresenta

tion

• Cosine• WMD

Intent � decision & � feature � generation

Similaritymodel

Data

Domain2:top up

Intentrelation

Domain2:weatherreport

Domain1:book airlinetickets

SM

graph

Topicmodel LDA

aiml

• Lda2vec

• skip-thoughts

• Maxent• CNN

BotFramework

B

A

DC

Algorithmcomponent

feature � generation







S2S model RNN • SentenceRNN• ContextRNN

Intelligent � Interaction � Technical � Practice -QA Bot

An � Excerpt � of � a Knowledge � Graph

Pros

Supports Contextual Reasoning

Minimized maintaining � costs for � knowledge � items

Improved accuracy (10%+) and user experience

Cons

Loss of recall at initial stage

Intelligent � Interaction � Technical � Practice - QA Bot

Preprocessing1. � Anaphora � resolution

2. � Word segmentation

3. � Error correction

Recall1. � Open � search

AnswerProcessing

1. � Answer rendering

2. � Logging

Computation1. � Similarity

2. � Sentiment analysis

3. � Attribute identification

Indexing1. � Inverted Indexing

StructuredKnowledge

Term Weighting

Question

Answer

Retrieval � Model

Intelligent � Interaction � Technical � Practice -Task Bot

Task-Oriented � QA

DomainData

Input DMS

Slots � Filling

ContextModel

Output

Input：Extract the slots of query

Output：DMS feedbacks the result

Processes：

•Get the attributes of the intent tree to fill

•Check the state of the intent tree

•Get the result by � the � state


AliMe.ai

• Features:• Customchat flow• Customentities and

slot values• Support the 3rd party

interfaces to beinvokedwith specificprotocol


RL for task-botØrelated-work from MSR

References � Williams � J � D, � Zweig � G. � End-to-end � LSTM-based � dialog � control � optimized � with � supervised � and � reinforcement � learning[J]. � 2016.


DRL for task-botØrelated-work【From Cambrige】

References Wen T H, Vandyke D, Mrksic N, et al. A Network-based End-to-End Trainable Task-oriented Dialogue System[J]. 2016.

task � generation

action � generation

dialogue � managerment

belief-tracking intentrepresentation context

slot-extraction pattern-extration

word � seg ner

task generation

reinforcementlearning

datapreprocess

Intelligent � Interaction � Technical � Practice - Chit Chat

Hybrid Approach: Retrieval Model + Generation

• Retrieval Model and Generation Model：

• Retireval Model：More platform, but only � output � answers � in � a �

pre-established � knowledge � base

• Generation Model： Generate � answers � out � of � the � box, � but � the �

generated � answers � sometimes � can � be � inconsistent � or � meanless

• Basic � ideas：

• Hybrid Approach: IR Rerank + Generation


Seq2Seq � Rerank � Module

Outline




Look � Forward � to � The � Future

p Model � construction � in � different � domains � and � scenes

p The � accumulation � of � domain � data � and � knowledge � is � very � important

p The � direction � of � continuous � exploration � in � the � field � of � Technology: � generative � model, � reinforcement � learning, � transfer � learning, � machine � reading, � emotion � and � so � on

p Construction � and � application � of � small � data � model

Thank You!

Intelligent � Interaction � Technical � Practice - QA Bot


Experiment


Hybrid Approach: IR Rerank + Generation

AliMe Bot Platform - LMS | Lab for Media Search at...

Documents

Transcript of AliMe Bot Platform - LMS | Lab for Media Search at...