Natural human to machine communication / Alon eirew

18
Natural Human to Machine Communication By Alon Eirew

Transcript of Natural human to machine communication / Alon eirew

Page 1: Natural human to machine communication / Alon eirew

Natural Human to Machine Communication

By Alon Eirew

Page 2: Natural human to machine communication / Alon eirew

Who Am I

• Software Architect at Intel Multimodal Language Understanding Group• Smart Home, Robotics, Smart Car, Mobile & Wearables

• Lead Developer at Intel New Devices Group• Personal Assistant for Mobile and Wearables

• Lead Developer Telmap (Navigation App)• Navigation Assistant for Mobile

https://il.linkedin.com/in/aloneirew

Page 3: Natural human to machine communication / Alon eirew

3

Agenda

What is Natural Communication?

Why is it interesting?

Overview the components and see some examples

Page 4: Natural human to machine communication / Alon eirew

4

Human to Human

Page 5: Natural human to machine communication / Alon eirew

Natural Communication Feature

Page 6: Natural human to machine communication / Alon eirew
Page 7: Natural human to machine communication / Alon eirew

Change the way we are communicating with our smart devices

Page 8: Natural human to machine communication / Alon eirew

8

Technologies that can’t support conventional input sources

Page 9: Natural human to machine communication / Alon eirew

The Era Of Conversational UI

How can I help you?

Page 10: Natural human to machine communication / Alon eirew

10

Building Blocks

Hardware

Recognition

Understanding

1

2

3

Page 11: Natural human to machine communication / Alon eirew

11

Hardware and Recognition Modules

Automatic Sound Recognition

Speech To Text

Computer VisionObject RecognitionPerson RecognitionGesture Recognition

Page 12: Natural human to machine communication / Alon eirew

Intent Extraction (NLU)

12

Intent extraction is the process of understanding commands, requests or promises conveyed in

human language.

Please turn off the lights in the living

room at 4PM

Location: living room

Time: 4PM

Object: lights

Action: turn off

Subject: (you)

Page 13: Natural human to machine communication / Alon eirew

World Knowledge

13

Page 14: Natural human to machine communication / Alon eirew

User Knowledge

14

Page 15: Natural human to machine communication / Alon eirew

15

Knowledge Use Cases

Page 16: Natural human to machine communication / Alon eirew

Semantic Fusion

Object [1] [N-Best]

Label Prob Location

Chair 0.7x1, y1, z1

Stool 0.4

Table 0.2

Label Prob Location

Table 0.8 x4, y4, z4

Stool 0.3 x4, y4, z4

Object 0.2 x4, y4, z4

Object [2] [N-Best]

Gesture List

Object Rec 1

Object Rec 2

End Point

Clean next to

this ???? ????

Word Prob

School Desk 0.7

Ship Deck 0.5

Sport Deko 0.2

ASR [N-Best]

Object [3] [N-Best]

Label Prob Location

Plant 0.7x7, y7, z7

Tree 0.4

Object 0.2

Page 17: Natural human to machine communication / Alon eirew

Dialog Features

17

Page 18: Natural human to machine communication / Alon eirew

Thank You!