Post on 15-Feb-2017
How to say it to your Smart Glasses?
Martin Neumann, innospiring GmbH iot conference Berlin 2015
about martin inventor
bloggerauthor
teacher
entrepreneur
legacy
chemist
what to expect?
• entertainment • new and relevant information • inspiration
no wannabe products, serious prototypes, hover boards or alike
the age of
SMARTCar
CityTV
Dust
HomeMeter
Bike
Robots
jash?*
*just another smart hype?
you are here
market - status quo
B2B or B2C?
• no AR end consumer product* or market yet • a lot of demo applications • main issues are price (~700 €), availability of
applications and content, use cases, familiarity, social acceptance
*besides universe2go
key features?!
key features?!
Something VERY important is missing here…
user interaction!
user interaction?voice
gesture
joystick ringbracelet
smartphonewatch
head/eye cursor
with wearables
user interaction examplesGoogle Glass
speech recognition
Microsoft HoloLens
gesture & speech recognitionEpson Moverio
head cursor interaction & touch
interaction litmus tests• natural - self explaining, easy to learn • reliable - no false positives, no false negatives • affirmative - appropriate feedback • context sensitive - no misinterpretation • non-intrusive - not bulky, not nerdy
-> ISO 9241
voice interaction
• very flexible • natural • hands free • easy to implement (microphone)
voice interaction
• very flexible • natural • hands free • easy to implement (microphone)
So, what’s the problem?
low reliability!
• flexible • limited natural • sometimes cool
gesture interaction
• flexible • limited natural • sometimes cool
So, what’s the problem?
gesture interaction
learning curve, not hands free, socially awkward,
ambiguous, hard to implement
gesture interaction
but wait - Kinect is super successful, isn’t it?
gesture interactionHaving sold 8 million units in its first 60 days
on the market, Kinect has claimed the Guinness World Record of being the "fastest
selling consumer electronics device".
24 million units of Kinect had been shipped by February 2013.
gesture interactionHaving sold 8 million units in its first 60 days
on the market, Kinect has claimed the Guinness World Record of being the "fastest
selling consumer electronics device".
24 million units of Kinect had been shipped by February 2013.
*Atkinson, Claire (October 17, 2010). "Microsoft gears up for big launch of Kinect controller-free device for Xbox - NYPOST.com". New York Post. Retrieved October 20, 2010.
- „hard“ bundling Kinect with Xbox One failed - compare: in the last 7 years 420 million gaming units were sold - Microsoft had an advertising budget of US$500 million for the launch of Kinect*.
BUT
gesture interaction
Myo Armband
gaining traction
Leap Demo Myo+Sphero
Demo
Gesture Control in
Cars
Leap with Rift
head cursor interaction
universe2go Demo
Webseite Demo
• use well known technology (smartphone) • very reliable • easy to learn (guided UI) • easy to implement
wearable interactionsmartphone
• use well known technology (smartphone) • very reliable • easy to learn (guided UI) • easy to implement
So, what’s the problem?
wearable interaction
then, why use smart glasses at all?
smartphone
• cool? form factor • hands free • can by made quite small and/or fashionable
wearable interactionbracelet
• cool? form factorSo, what’s the problem?
wearable interaction
did you recognize the flaw?
bracelet
• cool? form factor
wearable interactionring
Ring Demo
• cool? form factor
So, what’s the problem?
wearable interaction
still to be explored, learning curve, still bulky
ringRing
Demo
• very reliable • well known • easy to implement
wearable interactionjoystick
• very reliable • well known • easy to implement
So, what’s the problem?
wearable interaction
super nerdy %-D
joystick
• well known device - socially accepted • technology gains market share • reliable • touch easy to implement
wearable interactionsmart watch
• well known device - socially accepted • technology gains market share • reliable • touch easy to implement
So, what’s the problem?
wearable interaction
how?
smart watch
multimodal interaction
voice
gesture
joystick
ring
braceletsmartphone
watch
head/eye cursor
something missing?
CONTEXTsocial point of view
timelocation
mepersonalization
interaction
the fun part
pointingLightsaber
Demo• view direction +
point direction = point of interest
• + very natural (even for two years old) • + reliable • + easy to implement Pebble U2G
Demo
tiltingSpace Demo
• + quite natural for navigational task • + reliable • + easy to implement
proximity
• relative location in reference to a known point • proximity = context • + very natural • - low precision
the perfect triple
direction location
action
„no interaction is an island“
absolute (geo) relative (indoor) proximity
view point move
touch gesture voice
technical challenges
direction location
action
absolute (geo) relative (indoor)proximity
viewpointmove
touchgesture voice
technical challenges
• absolute horizontal direction unreliable (compass) • indoor location unreliable (radio interference) • speech recognition either needs high computing
power or fast connection • touch: how to sense a finger click with least effort;
Myo - too bulky; Ring - too bulky; smart watch - needs second hand (what about an Myo-Pebble Smart Strap? :-)
applications (c)nature / traveling
city / architecturemuseum
smart home
gaming
social
smart glasses user interaction will be very
application specific!
smart home exampletask context action context
marker
switch light pov & point „click“ or speech -
configure device proximity long „click“ -
configure scenario
location, time, device status
double long „click“
favourite: phrase or dedicated
button
activate scenario location, time „double click“
and speechfavourite: phrase
or dedicated button
nature example
task context action
show feature info pov & point -
show trail info proximity „click“
city exampletask context action
show feature info location & pov & point -
show poi location & pov „click"
show directions location head curser input
museum exampletask context action
show feature info location & pov & point -
show poi location & pov „click"
show directions location head curser input
social example
task context action
AR selfie location & pov & point
„click“ and select via head cursor input
gaming example
may the force be with you :-)
to be sure…
• pointing is not limited to pointing to physical objects - you can also point to virtual objects e.g. a virtual keyboard for text input or virtual buttons
• universe2go
• AR: Ori Inbar - Smart Glasses Report 2015; Augmented Reality World Expo AWE 2015 (search Youtube)
• Smart Glasses: Microsoft Hololens, Epson Moverio, Google Glass, Magic Leap, Vuzix, ODG R7, Meta, Cast AR, Seer, Reckon, Optinvent, Atheer, Innovega, Sony SmartEyeglass, Lumus
• Wearables: Myo, Cicret, Ring, Nod Ring
• Smart Watch: Pebble (SDK)
• Gesture Control: Leap Motion, Thalmic Labs - Myo, CNET - Gesture control in cars
• Space Control: Rfduino, Arduino, Eclipse, RXTX Comm Lib (do not use the original one with OS X 10.10), Serial Java Code, Starry Night, Celestia
• Smart Home: Oomi, Nest, Alljoyn
• IMU: Invensense MPU 9150; Razor IMU; Free IMU; Damien Douxchamps List; DCM Theory; Madgewick AHRS
• Misc: Sphero, Don’t click it, ISO 9241, Custom Lightsaber
resources
a brave jump into the wildSmart Glasses will be controlled by
the view direction of the glasses together with a wearable which is a
kind of bracelet, ring or smart watch, which can detect „finger click“, pointing direction and tilting. In conjunction with time, location,
proximity and personalization this will generate an easy and reliable
user experience.
Martin Neumann
innospiring GmbH mobil: +49 160 97710054 martin.neumann@innospiring.de www.innospiring.de Twitter: @innospiring