UX Fellows Gesture Research | gotomedia | SXSW | IEEE
-
Upload
kelly-goto -
Category
Technology
-
view
1.482 -
download
1
Transcript of UX Fellows Gesture Research | gotomedia | SXSW | IEEE
#IEEEGoto
Kelly Goto@go2girl #IEEEGoto
Thumbs Up to Natural Gesture Interfaces? Can We Create a Global Gesture Language?
Tuesday, August 12, 14
#IEEEGotoGestures their Origins and DistributionsMorris, Collette, Marsh, O’Shaughnessypregnancyandbaby.comparamount.com
Tuesday, August 12, 14
#IEEEGoto
Tuesday, August 12, 14
#IEEEGoto
Gestuno » International Sign Language
http://www.handspeak.com
Global Gestures
A native language Gestuno is not. But it is a barrier breaking vocabulary that allows a select few to bridge an international language gap... the need to bridge that international barrier has aroused a sort of lingua franca rooted deep in the trade industry of modern Europe.
http://www.deaflibrary.org
“Government” in different cultures:
Tuesday, August 12, 14
#IEEEGoto
Global Gestures
Unlike human-to-human gesturing, human-machine gesturing does not have a foundation of thousands of years of history to serve as a context.
- Silicon Labs
Tuesday, August 12, 14
#IEEEGoto
Freehand Gestures Are Entering Our Lives
Tuesday, August 12, 14
#IEEEGoto
Freehand Gestures Are Entering Our Lives
Tuesday, August 12, 14
#IEEEGoto
Freehand Gestures Are Entering Our Lives
Tuesday, August 12, 14
#IEEEGoto
9,000
8,000
7,000
6,000
5,000
4,000
3,000
2,000
1,000
$Million
2012 2013 2014 2015 2016 2017 2018
Gesture Recogni<on Touchless Sensing
Global Touchless Sensing & Gesture Recognition Market Revenue, $million, (2013-2018)
• Expected to reach $15.02 billion by 2018.
• Market value in 2012 was approximately $2 billion.
• Consumer electronics applica<on contributes to more than 99% of the global gesture recogni<on market.
Global Market Revenue Projections
Tuesday, August 12, 14
#IEEEGoto
Global Market Revenue Projections
#IEEEGoto
Tuesday, August 12, 14
#IEEEGoto
Pointing Gestures
Tuesday, August 12, 14
#IEEEGoto
The UX Fellows Gesture Study
#IEEEGoto#IEEEGoto
Tuesday, August 12, 14
#IEEEGoto
We Compared Gestures in 18 Countries.
Tuesday, August 12, 14
#IEEEGoto
The UX Fellows Gesture Study
Which natural gestures would people from various cultures spontaneously use to control consumer electronic devices?
• Country differences?
• Country clusters?
• Underlying metaphors?
• Acceptance?
Tuesday, August 12, 14
#IEEEGoto
Setting 15 min
Warm-up: everyday gestures
#IEEEGoto
Tuesday, August 12, 14
#IEEEGoto
Warmup: Everyday ‘Natural’ Gestures
C: I like!
B: I guess I ate something wrong
A: I think I need to see a dentist
What are these people trying to tell us?
D: That meal is really tasty
D: That meal is really tasty
Tuesday, August 12, 14
#IEEEGoto
Examined Commands
Switch on device
Volume up
Channel up
Context info for current program
Channel down
Share current program
Volume down
Mute volume
Pause current program
Homepage
EPG
P+
P-‐
Backward navigation in menus
Confirm on-screen dialog (OK)
Deny on-screen dialog (cancel)
Pause video-on-demand (VOD)
Restart VOD
Fast forward VOD
Rewind VOD
Skip chapter VOD
Previous chapter VOD
Stop VOD
Switch off device
Tuesday, August 12, 14
#IEEEGoto
Participant Sample
Persons who are able to imagine a gesture-control scenario for a TV.
• Regular usage of a recent HD flat TV (max. 5-6 years of age) and other CE devices and services
• High interest in consumer electronics
• Regular usage of a touch device
• No Xbox Kinect and gesture-enabled TV users
• Potential buyers of international "premium" CE
brands
(no national, retailer or "no name" brands)
Tuesday, August 12, 14
#IEEEGoto
Participant Sample
sample
soci
o-ec
onom
icba
ckgr
ound
country
Tuesday, August 12, 14
#IEEEGoto
Participant Sample
51%49%
18 countries x 20 subjects = 360 participants
16
65 years
32
min
max
mean
Tuesday, August 12, 14
#IEEEGoto
The Results
#IEEEGoto#IEEEGoto
Tuesday, August 12, 14
#IEEEGoto
Difficulty of TV-Related Gestures
Self-‐reported difficulty, 5 = very difficult, 1 = very easy, n = 360 tech-‐savvy CE consumers in 18 countries
easy
diff
icul
t
Tuesday, August 12, 14
#IEEEGoto
Commonality of Gestures
no. of countries with the same main gesture
High Medium Low
Tuesday, August 12, 14
#IEEEGoto
Commands with High Commonality
Volume down
Volume up
Pause VOD
Pause current program
Rewind VOD
Fast forward VOD
Confirm on-screen dialog (OK)
Tuesday, August 12, 14
#IEEEGoto
Commands with High Commonality
Volume down
Volume up
Pause VOD
Pause current program
Rewind VOD
Fast forward VOD
Confirm on-screen dialog (OK)
Commonality 18 of 18
Difficulty (1-‐5) 1.8
Q: How do you turn down the volume?
Tuesday, August 12, 14
#IEEEGoto
Commands with High Commonality
Volume down
Volume up
Pause VOD
Pause current program
Rewind VOD
Fast forward VOD
Confirm on-screen dialog (OK)
Commonality 18 of 18
Difficulty (1-‐5) 1.8
Q: How do you turn up the volume?
Tuesday, August 12, 14
#IEEEGoto
Commands with High Commonality
Volume down
Volume up
Pause VOD
Pause current program
Rewind VOD
Fast forward VOD
Confirm on-screen dialog (OK)
Commonality 17 of 18
Difficulty (1-‐5) 1.5
Q: How do you pause viewing a film?
NL
Tuesday, August 12, 14
#IEEEGoto
Commands with High Commonality
Volume down
Volume up
Pause VOD
Pause current program
Rewind VOD
Fast forward VOD
Confirm on-screen dialog (OK)
Commonality 17 of 18
Difficulty (1-‐5) 2.0
Q: How would you pause the current TV program?
NL
Tuesday, August 12, 14
#IEEEGoto
Commands with High Commonality
Volume down
Volume up
Pause VOD
Pause current program
Rewind VOD
Fast forward VOD
Confirm on-screen dialog (OK)
Commonality 15 of 18
Difficulty (1-‐5) 1.7
Q: How do you rewind? (with an image being shown)
AU, UK
NL
Tuesday, August 12, 14
#IEEEGoto
Commands with High Commonality
Volume down
Volume up
Pause VOD
Pause current program
Rewind VOD
Fast forward VOD
Confirm on-screen dialog (OK)
Commonality 15 of 18
Difficulty (1-‐5) 1.9
Q: You are watching a boring scene, how do you fast-‐forward (with an image being shown)?
AU, UK
NL
Tuesday, August 12, 14
#IEEEGoto
Commands with High Commonality
Volume down
Volume up
Pause VOD
Pause current program
Rewind VOD
Fast forward VOD
Confirm on-screen dialog (OK)
Commonality 15 of 18
Difficulty (1-‐5) 1.9
Q:: You have chosen a film from an online video library. The TV asks you "do you really want to buy the film?" How do you answer "yes"?
CN, IT, KR
Tuesday, August 12, 14
#IEEEGoto
Commands with Medium Commonality
Channel up
Channel down
Stop VOD
Mute volume
Backward navigation in menus
Deny on-screen dialog (cancel)
Skip chapter VOD
Tuesday, August 12, 14
#IEEEGoto
Commands with Medium Commonality
Channel up
Channel down
Stop VOD
Mute volume
Backwards navigation in menus
Deny on-screen dialog
(cancel)
Skip chapter VOD
Commonality 13 of 18
Difficulty (1-‐5) 1.8
Q: You are zapping through the TV programs: How do you change from channel 15 to channel 16 for example?
P+
AU, ES, KR, US
CA
Tuesday, August 12, 14
#IEEEGoto
Commands with Medium Commonality
Channel up
Channel down
Stop VOD
Mute volume
Backwards navigation in menus
Deny on-screen dialog
(cancel)
Skip chapter VOD
Commonality 12 of 18
Difficulty (1-‐5) 1.6
Q: You are zapping through the TV programs: How do you change from channel 16 to channel 15 for example?
AU, ES, KR, US
CA
P-‐
Tuesday, August 12, 14
#IEEEGoto
Commands with Medium Commonality
Channel up
Channel down
Stop VOD
Mute volume
Backwards navigation in menus
Deny on-screen dialog
(cancel)
Skip chapter VOD
Commonality 13 of 18
Difficulty (1-‐5) 2.1
Q: How would you stop watching the film altogether (because you want to do something different)?
CN, KR
FI, RU
NL
Tuesday, August 12, 14
#IEEEGoto
Commands with Medium Commonality
Channel up
Channel down
Stop VOD
Mute volume
Backwards navigation in menus
Deny on-screen dialog
(cancel)
Skip chapter VOD
Commonality 11 of 18
Difficulty (1-‐5) 2.0
Q: How do you set the volume to "mute"?
AU, CN, MX FI, RU
BE
UK
Tuesday, August 12, 14
#IEEEGoto
Commands with Medium Commonality
Channel up
Channel down
Stop VOD
Mute volume
Backwards navigation in menus
Deny on-screen dialog
(cancel)
Skip chapter VOD
Commonality 10 of 18
Difficulty (1-‐5) 2.3
Q: Imagine that you were looking for a film in an online video library. You have gone to the comedies sec<on and have chosen the descrip<on of a single movie. Now you want to return to the previous set of op<ons (all of the comedies). Which gesture would you use for that?
AU, BE, CA
AR, RU
ES, MX, US
Tuesday, August 12, 14
#IEEEGoto
Commands with Medium Commonality
Channel up
Channel down
Stop VOD
Mute volume
Backwards navigation in menus
Deny on-screen dialog
(cancel)
Skip chapter VOD
Commonality 9 of 18
Difficulty (1-‐5) 1.8
Q: How do you express that you don’t want to buy it?
AU, CN, RU, TR
KR
AR, BE, IT, MX
Tuesday, August 12, 14
#IEEEGoto
Commands with Medium Commonality
Channel up
Channel down
Stop VOD
Mute volume
Backwards navigation in menus
Deny on-screen dialog
(cancel)
Skip chapter VOD
Commonality 9 of 18
Difficulty (1-‐5) 2.2
Q: How would you skip to the next chapter of the film?
BE, CN, ES, FI, FR, IN, MX, UK
NL
Tuesday, August 12, 14
#IEEEGoto
Commands with Low Commonality
Switch on device
Switch off device
Previous chapter VOD
Restart VOD
Share current program
EPGContext info for currentprogramHomepage
Tuesday, August 12, 14
#IEEEGoto
Commands with Low Commonality
Commonality 8 of 18
Difficulty (1-‐5) 2.0
Q: How would you switch on the TV?
Switch on device
Switch off device
Previous chapter VOD
Restart VOD
Share current program
EPGContext info for currentprogramHomepage
BE, FR, US
KR
DE, IT
TR
CNCA
FI
Tuesday, August 12, 14
#IEEEGoto
Commands with Low Commonality
Commonality 7 of 18
Difficulty (1-‐5) 2.9
Q: How would you switch off the TV? Note: No common gesture in Italy
Switch on device
Switch off device
Previous chapter VOD
Restart VOD
Share current program
EPGContext info for currentprogramHomepage
ES, MX, NLIN, KR
RU
TR
CA
AR
Tuesday, August 12, 14
#IEEEGoto
Commands with Low Commonality
Commonality 8 of 18
Difficulty (1-‐5) 2.0
Q: How would you return to the previous chapter of the film?
Switch on device
Switch off device
Previous chapter VOD
Restart VOD
Share current program
EPGContext info for currentprogramHomepage
BE, CN, ES, FI, FR, MX, NL
IN, US
UK
Tuesday, August 12, 14
#IEEEGoto
Commands with Low Commonality
Commonality 7 of 18
Difficulty (1-‐5) 2.0
Q: How do you start it (film) again? Note: No common gesture in UK
Switch on device
Switch off device
Previous chapter VOD
Restart VOD
Share current program
EPGContext info for currentprogramHomepage
AU, ES, KR, MX
BE
NL RU
US
AR, DE
Tuesday, August 12, 14
#IEEEGoto
Commands with Low Commonality
Commonality 6 of 18
Difficulty (1-‐5) 3.2
Q: How would you share with your friends what you are currently watching or listening to? Note: No common gesture in UK
Switch on device
Switch off device
Previous chapter VOD
Restart VOD
Share current program
EPGContext info for currentprogramHomepage
CN, FR, IN
FI
AR, TR
MX
NLKR
RU
US
Tuesday, August 12, 14
#IEEEGoto
Commands with Low Commonality
Commonality 5 of 18
Difficulty (1-‐5) 2.8
Q: You'd like to see an overview of tonight's TV programs of the main channels (EPG). How do you ac<vate this func<on with a hand gesture? Note: No common gesture in Italy
Switch on device
Switch off device
Previous chapter VOD
Restart VOD
Share current program
EPGContext info for currentprogramHomepage
AU, US CN, FI, IN, KR, RU
BE, UK
TR
NL
MX
Tuesday, August 12, 14
#IEEEGoto
Commands with Low Commonality
Commonality 4 of 18
Difficulty (1-‐5) 2.8
Q: How would you get more informa<on for the running TV program?
Switch on device
Switch off device
Previous chapter VOD
Restart VOD
Share current program
EPGContext info for currentprogramHomepage
FR, KR, RUAU, FI
CN, IN, TR
AR
DE
BE
ES
US
UK
Tuesday, August 12, 14
#IEEEGoto
Commands with Low Commonality
Commonality 4 of 18
Difficulty (1-‐5) 2.9
Q: Now you want to go to the ini<al set of op<ons -‐ that means the first op<ons you see when you switch on the device.Note: No common gesture in Italy
Switch on device
Switch off device
Previous chapter VOD
Restart VOD
Share current program
EPGContext info for currentprogramHomepage
AR, AU, MX
DENL, UK
FI
US
BE, IN
CA, ES, TR
Tuesday, August 12, 14
#IEEEGoto
Some General Insights
#IEEEGoto
Tuesday, August 12, 14
#IEEEGoto
Direct vs Abstract Gestures
We observed two kinds of natural gestures.
● "Direct" gestures
¡ strongly influenced by touch devices
● Abstract gestures
¡ basis in language, iconography and metaphors
¡ associated with more complex tasks
¡ lower intercultural commonality
Tuesday, August 12, 14
#IEEEGoto
1.0
1.5
2.0
2.5
3.0
TR FR MX AR IN FI CN RU NL DE AU US BE ES CA IT KR UK
Everyday gestures CE gestures
difficulty
min
max 5.0
Average Gesture Difficulty by Country
…
Tuesday, August 12, 14
#IEEEGoto
Average Gesture Difficulty by Country
1.0
1.5
2.0
2.5
3.0
FR CN TR IN AR RU MX US IT DE AU FI CA BE ES NL UK KR
CE gestures
…
difficulty
min
max 5.0
Tuesday, August 12, 14
#IEEEGoto
General Acceptance
§ Our 360 tech-savvy participants responded very positively to the idea of semantic gesture input for TVs. "fun to use", "convenient", "coolness factor”
§ Participants would like to use gestures to control other devices as well (e.g. audio).
Tuesday, August 12, 14
#IEEEGoto
General Acceptance
Can CE gestures be deducted from everyday gestures?
USAInfluenced by touch screens — most thought gestures should be customizable.
CanadaPersonal hobbies eg: gestures inspired by skiing, kick-boxing etc. Sign language
Russia.Not a lot of everyday gestures used. Replicating alphabets, creating new gestures and pointing used.
No.
Yes.Mexico Smart phones and touch screen devices eg: sliding the menu up from the bottom, swiping gestures etc.
Spain | Mexico | France | NetherlandsEveryday gestures and touch screen gestures
Australia | Germany | ItalyEveryday gestures
Argentina | China | KoreaTouch screen devices
IndiaAnalog gestures, touch screen gestures and icon based gestures
BegiumEveryday gestures, gestures while using electronic devices and touch screen gestures
FinlandEveryday gestures and touch screen gestures
Tuesday, August 12, 14
#IEEEGoto
Private vs Public Display
§ Considerations
§ Ergonomics
§ “Gorilla Arm”§ Recognition
privatepublic
Tuesday, August 12, 14
#IEEEGoto
Operation
Many participants felt that voice and sound could be a good support for gesture input.
waking up the device with a sound
Can the TV hear me?
alterna4ve op4on based on context and need
Tuesday, August 12, 14
#IEEEGoto
Operation
FR CN TR IN AR RU MX US IT DE AU FI CA BE ES NL UK KR
Seman<c
Poin<ng
Many users would prefer switching between gesture types.
FR CN TR IN AR RU MX US IT DE AU FI CA BE ES NL UK KR Σ
Seman<c l l l l l l l l l l l l l l l 15
Poin<ng
FR CN TR IN AR RU MX US IT DE AU FI CA BE ES NL UK KR Σ
Seman<c l l l l l l l l l l l l l l l 15
Poin<ng l l l l l l l l l 9
associated with more advanced commands alterna4ve op4on based on context and need
Tuesday, August 12, 14
#IEEEGoto
Typical Concerns of Users
§ Learning and retention of gestures
(but: would be easy if based on implicit
behavior repertory)
§ Sensitivity and accuracy of devices
§ Interference by other gestures in the
room
§ Compatibility with lean-back disposition
Tuesday, August 12, 14
#IEEEGoto
Conclusions
1. For basic CE commands a common language for natural gestures already exists across cultures.
2. More complex commands could be realized by either
a. customizing these gestures for each individual market
b. customizing individually
c. proposing a set of semantic gestures that need to be learned (the study provides some suggestions)
d. switching to pointing gestures in combination with a GUI
3. It is necessary to consider cultural differences at a country level
Tuesday, August 12, 14
#IEEEGoto
Future of communication
Do we want to create a better remote control, or do we want to create a 2-way language of communication?
- go2girl
Tuesday, August 12, 14
#IEEEGoto
Future of communication
LG Magic Remote
Tuesday, August 12, 14
#IEEEGoto
The problem: no standard gestures currently exist.
Many new devices in the world of ‘touchless tech’ are using completely different gestures for the same common functions.
Future of communication
Tuesday, August 12, 14
#IEEEGoto
Can we utilize this methodology to start a conversation and bring the deaf community, the large technology giants and standards together to develop a universal gesture language?
source: autocarhire.com
Future of communication
Tuesday, August 12, 14
#IEEEGoto
Future of communication
Tuesday, August 12, 14
#IEEEGoto
The governance crisis point
Quantity & complexity of information
Ability to deal with quantity & complexity
of information
Zap Think LLC
TIME
(innovation) COMPLEXITY vs. COPING (desire)
Tuesday, August 12, 14
#IEEEGoto
Remember: People don’t advance as quickly as technology.
BEYOND USABLE
Tuesday, August 12, 14
#IEEEGoto
CONTEXT
Tuesday, August 12, 14
#IEEEGotosource: wikipedia
the future?
Tuesday, August 12, 14
#IEEEGoto
Find all results on www.uxfellows.com
Drop us a line: [email protected]
Michael Woermann [email protected]
www.3bridgesaustin.com
Tuesday, August 12, 14
#IEEEGoto
● UX design + research consulting
● San Francisco, CA
● Global mobile focus
#IEEEGoto
Tuesday, August 12, 14