2015/11/29

Marat Zhanikeev maratishe@gmail.com HCI研@別府

PDF: bit.do/151129

Towards Social Robotics on Smartphones with Simple XYZV Sensor Feedback

(dumb) Robotics Today

• most robots today are dumb• if you have all the necessary inputs, no room/need for AI, just use your inputs

• the case of HAL: weight shifting,bioelectricity, relative angles

M.Zhanikeev -- maratishe@gmail.com -- Towards Social Robotics on Smartphones with Simple XYZV Sensor Feedback -- bit.do/151129 2/192/19

From to Smart Robotics• objective 1: there is a need for truely smart/flexible robotics• objective 2: preferably within commodity reach (smartphones)

Increasing specialization

Increasing smart-ness

Most robots today

Social Robotics

M.Zhanikeev -- maratishe@gmail.com -- Towards Social Robotics on Smartphones with Simple XYZV Sensor Feedback -- bit.do/151129 3/193/19

Social Robotics Basics

• basic idea: start with a blankrobot and teach it a task◦ humans are trainers

• optimization: reasoning byreinforcement learning

• complexity: reducesearch space via humanfeedback

M.Zhanikeev -- maratishe@gmail.com -- Towards Social Robotics on Smartphones with Simple XYZV Sensor Feedback -- bit.do/151129 4/194/19

Social Robotics in Soft- vs Hard-ware• good parallel between soft- and hard-ware social robotics• for higher reliability, vision/recognition in hardware robots should besimplified◦ reliable feedback gestures, possibly even without vision

• context is natural to software but is also important in hardware (more later)

Generic Use

Teaching, Guidance Reasoning

Human Role

Hardware Robots

Wide range of behavior

Reinforcement Learning

YES. Vision, recognition Guide only

Software Automation

Any kind of context

Bayesian Classification

NO Not needed

Guide and decision - maker

Potential for simplification

M.Zhanikeev -- maratishe@gmail.com -- Towards Social Robotics on Smartphones with Simple XYZV Sensor Feedback -- bit.do/151129 5/195/19

Social Robotics in Context Management• example of software robot that processes folksonomy bigdata

Robot

(careless) Input

Human Human

{structure}

(pinpoint) Select

Browse (or use otherwise)

Some Knowledge

(folksonomies, knowledge bases, databases, indexes, ontologies, etc.)

(metromaps )

guide, trainer

sources, crowd

Recommendation Bot = Rebot

M.Zhanikeev -- maratishe@gmail.com -- Towards Social Robotics on Smartphones with Simple XYZV Sensor Feedback -- bit.do/151129 6/196/19

Social Robotics in Context (2)

0 5.2 10.4 15.6 20.8 26Time/Learning Sequence

0

4

8

12

16

20

Rebo

t Ord

er v

s Use

r Sel

ectio

n

Linear fit (circle bullets)Rebot: New TrainingRebot: Order of Selection in Rebot List

0 5.2 10.4 15.6 20.8 26Time/Learning Sequence

0

4

8

12

16

20

Rebo

t Ord

er o

f Use

r Sel

ectio

n

• starts off really dumn, but gets better with time

• diff between human and robot decisions decreases

• when diff is mostly zero, human trainer can go home

M.Zhanikeev -- maratishe@gmail.com -- Towards Social Robotics on Smartphones with Simple XYZV Sensor Feedback -- bit.do/151129 7/197/19

Social Robotics and Blackswans• a useful "show me only the blackswans" application

• useful in : disaster scenarios, literature reviews, etc.• crucial to remove human component as some point

Human judgment

Auto judgement

Folksonomy

M.Zhanikeev -- maratishe@gmail.com -- Towards Social Robotics on Smartphones with Simple XYZV Sensor Feedback -- bit.do/151129 8/198/19

Social Robotics and Blackswans (2)

Metromap Classifier

Human

Check Metromap

Fuzzy?

Cold? Hot?

Robot (Automatic Classification)

Bad

Input

No Yes

No

No

M.Zhanikeev -- maratishe@gmail.com -- Towards Social Robotics on Smartphones with Simple XYZV Sensor Feedback -- bit.do/151129 9/199/19

Simple XYZV Channel

M.Zhanikeev -- maratishe@gmail.com -- Towards Social Robotics on Smartphones with Simple XYZV Sensor Feedback -- bit.do/151129 10/1910/19

Model of Robotic Decision Making• on smartphones, easy access to XYZ gravity and V acceleration = XYZV input

• perfect if the same channel can be used for sensor input and human feedback

Model

Human

Robot

Actions

Constraints, logic

Same -channel interaction

Context management

M.Zhanikeev -- maratishe@gmail.com -- Towards Social Robotics on Smartphones with Simple XYZV Sensor Feedback -- bit.do/151129 11/1911/19

The Single-Channel Objective

Number of distinct data inputs

Reliability

Single-channel

Acceptable level

1

Dumb curve Smart curve

M.Zhanikeev -- maratishe@gmail.com -- Towards Social Robotics on Smartphones with Simple XYZV Sensor Feedback -- bit.do/151129 12/1912/19

Experiment (1)

• attach smartphone to one ankle• detect up-the-stairs steps• use human feedback to learn whichdetections are wrong

M.Zhanikeev -- maratishe@gmail.com -- Towards Social Robotics on Smartphones with Simple XYZV Sensor Feedback -- bit.do/151129 13/1913/19

Experiment (1) Feedback

1. smartphone notifies detected steps viavibration

2. human punishes smartphone for baddetections (negative feedback)

• stroke/hit your smarphone when it is wrong/bad

• jerk/shake a leg to loosen it up

• both are natural interactions for humans

• positive feedback is harder (so far)

M.Zhanikeev -- maratishe@gmail.com -- Towards Social Robotics on Smartphones with Simple XYZV Sensor Feedback -- bit.do/151129 14/1914/19

Experiment (1) : Resultsnegative mirror = moving average“Stroke” feedback x

yzv

“Jerk” feedback

Event detected

StepStroke

StepJerk

M.Zhanikeev -- maratishe@gmail.com -- Towards Social Robotics on Smartphones with Simple XYZV Sensor Feedback -- bit.do/151129 15/1915/19

Experiment (1) : Other Results

• about 100 experimental sessions

• 60% are FPs, 30% are FNs• within the FPs, in 90% of them stroke/jerk feedback was detected

◦ 3s timeout, detect unusual XYZV signal right after vibration◦ vibration itself is too small to register in XYZV

M.Zhanikeev -- maratishe@gmail.com -- Towards Social Robotics on Smartphones with Simple XYZV Sensor Feedback -- bit.do/151129 16/1916/19

Experiment (2) Context

• current work in progress

• idea: establish context first, then detect steps• experiment: walk with a smartphone in yourpocket

M.Zhanikeev -- maratishe@gmail.com -- Towards Social Robotics on Smartphones with Simple XYZV Sensor Feedback -- bit.do/151129 17/1917/19

Experiment (2) Context (2)xyzv

flat walking• horizontal/flatwalking

xyzvwalking up the stairs

• walking up thestairs

M.Zhanikeev -- maratishe@gmail.com -- Towards Social Robotics on Smartphones with Simple XYZV Sensor Feedback -- bit.do/151129 18/1918/19

That’s all, thank you ...

M.Zhanikeev -- maratishe@gmail.com -- Towards Social Robotics on Smartphones with Simple XYZV Sensor Feedback -- bit.do/151129 19/1919/19