NUI Research
Transcript of NUI Research
![Page 1: NUI Research](https://reader030.fdocuments.net/reader030/viewer/2022032904/6241f709c15df74fa761a600/html5/thumbnails/1.jpg)
Turning Data into Insight
NUI Research
![Page 2: NUI Research](https://reader030.fdocuments.net/reader030/viewer/2022032904/6241f709c15df74fa761a600/html5/thumbnails/2.jpg)
Motivations
The spread use of ICT in government, healthcare, education and basic
citizen needs.
Many software consider accessibility for disability users, but no low
literacy accessibility.
Above items, are bringing “technological discrimination”.
Adult persons (> 40 years old) , often have digital low literacy (developing
countries is common), often called digital immigrants.
Urban vs. Rural areas, literacy.
![Page 3: NUI Research](https://reader030.fdocuments.net/reader030/viewer/2022032904/6241f709c15df74fa761a600/html5/thumbnails/3.jpg)
Turning Data into Insight
“I don’t want to make the most efficient use of the computer;
I want to make the most efficient use of the human.
How do you optimize the human user?”
Bill Buxton
NUI for Healthcare
Ask Appointments and Self-care at home using NUI state of the arttechnologies. No computer experience required
NUI for Education
Learning Management Systems using NUI state of the arttechnologies. No learning curve required
Unlimited opportunities
![Page 4: NUI Research](https://reader030.fdocuments.net/reader030/viewer/2022032904/6241f709c15df74fa761a600/html5/thumbnails/4.jpg)
Input Ecology
Keyboard
Good for
Accuracy
Con
Limited
Mouse
Good for
Moving across the
screen efficiently
Con
Indirect
Stylus
Good for
Freeform input
Con
Low accuracy
Need large controls
Single point
Good for
Direct object selection
Con
Not for quick input
Low accuracy
Need of large controls
Mis-hits from fingernails
of perceived hit area
Multi point
Good for
Manipulating objects
Social
Con
Not for input
Low accuracy
Need large controls
Support of hands and
screen
![Page 5: NUI Research](https://reader030.fdocuments.net/reader030/viewer/2022032904/6241f709c15df74fa761a600/html5/thumbnails/5.jpg)
Essentially, what is "natural" depends on what
is already familiar to the user and what is also
appropriate at the moment the user is
interacting with the system
Appropriate User
Interface Bill Buxton
![Page 6: NUI Research](https://reader030.fdocuments.net/reader030/viewer/2022032904/6241f709c15df74fa761a600/html5/thumbnails/6.jpg)
CLI GUI NUI
![Page 7: NUI Research](https://reader030.fdocuments.net/reader030/viewer/2022032904/6241f709c15df74fa761a600/html5/thumbnails/7.jpg)
![Page 8: NUI Research](https://reader030.fdocuments.net/reader030/viewer/2022032904/6241f709c15df74fa761a600/html5/thumbnails/8.jpg)
![Page 9: NUI Research](https://reader030.fdocuments.net/reader030/viewer/2022032904/6241f709c15df74fa761a600/html5/thumbnails/9.jpg)
![Page 10: NUI Research](https://reader030.fdocuments.net/reader030/viewer/2022032904/6241f709c15df74fa761a600/html5/thumbnails/10.jpg)
![Page 11: NUI Research](https://reader030.fdocuments.net/reader030/viewer/2022032904/6241f709c15df74fa761a600/html5/thumbnails/11.jpg)
![Page 12: NUI Research](https://reader030.fdocuments.net/reader030/viewer/2022032904/6241f709c15df74fa761a600/html5/thumbnails/12.jpg)
![Page 13: NUI Research](https://reader030.fdocuments.net/reader030/viewer/2022032904/6241f709c15df74fa761a600/html5/thumbnails/13.jpg)
NUI-based technologies such
as speech, touch, contextual
and environmental awareness,
immersive 3D experiences,
Multi - Channel and anticipatory
computing -- all with the goal of
a computer that can see, listen,
learn, talk and act smartly on
our behalf.
Steve Ballmer.
![Page 14: NUI Research](https://reader030.fdocuments.net/reader030/viewer/2022032904/6241f709c15df74fa761a600/html5/thumbnails/14.jpg)
![Page 15: NUI Research](https://reader030.fdocuments.net/reader030/viewer/2022032904/6241f709c15df74fa761a600/html5/thumbnails/15.jpg)
![Page 16: NUI Research](https://reader030.fdocuments.net/reader030/viewer/2022032904/6241f709c15df74fa761a600/html5/thumbnails/16.jpg)
We should
ignore
THE PERIODIC TABLE… for a moment
And focus ON human being!
![Page 17: NUI Research](https://reader030.fdocuments.net/reader030/viewer/2022032904/6241f709c15df74fa761a600/html5/thumbnails/17.jpg)
NNatural
Learning Curve
Design Symbol
Element Name
FactMultimodality
Metric
MetricCognitive load
![Page 18: NUI Research](https://reader030.fdocuments.net/reader030/viewer/2022032904/6241f709c15df74fa761a600/html5/thumbnails/18.jpg)
UUser
Skills acquired
Design Symbol
Element Name
FactLiteracy
Metric
MetricSubjective Satisfaction
![Page 19: NUI Research](https://reader030.fdocuments.net/reader030/viewer/2022032904/6241f709c15df74fa761a600/html5/thumbnails/19.jpg)
IInterface
Guided / Situated Interaction
Design Symbol
Element Name
FactCommon
Metric
MetricVisual Cues / Feedback
![Page 20: NUI Research](https://reader030.fdocuments.net/reader030/viewer/2022032904/6241f709c15df74fa761a600/html5/thumbnails/20.jpg)
DDepth
DepthImagePixel[]
AAudio
Waveform
IRInfrared
byte[]
CColor
byte[]
TTouch and
Traditional Inputs
Points and Chars
![Page 21: NUI Research](https://reader030.fdocuments.net/reader030/viewer/2022032904/6241f709c15df74fa761a600/html5/thumbnails/21.jpg)
IFInteractions
InteractionFrame
(UserInfo[])
MKTraditional
InputMouse/Keyboard
FTFace Tracking
FaceTrackFrame
SSpeech
Recognition Events
STSkeleton
TrackingSkeleton[]
T
![Page 22: NUI Research](https://reader030.fdocuments.net/reader030/viewer/2022032904/6241f709c15df74fa761a600/html5/thumbnails/22.jpg)
![Page 23: NUI Research](https://reader030.fdocuments.net/reader030/viewer/2022032904/6241f709c15df74fa761a600/html5/thumbnails/23.jpg)
NUI interface design helps users among different ages anddiverse computer experience to ask a medical consultationreducing the learning curve.
![Page 24: NUI Research](https://reader030.fdocuments.net/reader030/viewer/2022032904/6241f709c15df74fa761a600/html5/thumbnails/24.jpg)
Aspect User 1 User 2 User 3
Age 67 58 29
Literacy Basic High School Accounting Professional
Times in front of the pc troughlife time.
0 5 All most 7/24
Last time in front of the pc. Never Six months ago Two hours ago
Living Area Rural Urban Urban
Users
![Page 25: NUI Research](https://reader030.fdocuments.net/reader030/viewer/2022032904/6241f709c15df74fa761a600/html5/thumbnails/25.jpg)
![Page 26: NUI Research](https://reader030.fdocuments.net/reader030/viewer/2022032904/6241f709c15df74fa761a600/html5/thumbnails/26.jpg)
![Page 27: NUI Research](https://reader030.fdocuments.net/reader030/viewer/2022032904/6241f709c15df74fa761a600/html5/thumbnails/27.jpg)
![Page 28: NUI Research](https://reader030.fdocuments.net/reader030/viewer/2022032904/6241f709c15df74fa761a600/html5/thumbnails/28.jpg)
![Page 29: NUI Research](https://reader030.fdocuments.net/reader030/viewer/2022032904/6241f709c15df74fa761a600/html5/thumbnails/29.jpg)
![Page 30: NUI Research](https://reader030.fdocuments.net/reader030/viewer/2022032904/6241f709c15df74fa761a600/html5/thumbnails/30.jpg)
Multimodal interaction through dashboard. (a) User give voice commands to navigate
through dashboard, as well as (b) touch the screen, (c) swipe his hands on the air or (d)
use traditional mouse to do a task
![Page 31: NUI Research](https://reader030.fdocuments.net/reader030/viewer/2022032904/6241f709c15df74fa761a600/html5/thumbnails/31.jpg)
Application created for help patients to ask appointments in a natural way. (a) The login
can be done using multimodal interaction as well as (b) review the bio signals graph, (c)
and assign an appointment on a selected day considering (d) the care professional
agenda.
![Page 32: NUI Research](https://reader030.fdocuments.net/reader030/viewer/2022032904/6241f709c15df74fa761a600/html5/thumbnails/32.jpg)
![Page 33: NUI Research](https://reader030.fdocuments.net/reader030/viewer/2022032904/6241f709c15df74fa761a600/html5/thumbnails/33.jpg)
17
14
3.2
54
3
0
2
4
6
8
10
12
14
16
18
User 1 User 2 User 3
MIN
UTES
First Time
GUI Design NUI Design
14
10
1.2
32.2
1.6
0
2
4
6
8
10
12
14
16
User 1 User 2 User 3
MIN
UTES
Second Time
GUI Design NUI Design
![Page 34: NUI Research](https://reader030.fdocuments.net/reader030/viewer/2022032904/6241f709c15df74fa761a600/html5/thumbnails/34.jpg)
Debating the future
of NUI is useless
![Page 35: NUI Research](https://reader030.fdocuments.net/reader030/viewer/2022032904/6241f709c15df74fa761a600/html5/thumbnails/35.jpg)
Creating the future
is more interesting
![Page 36: NUI Research](https://reader030.fdocuments.net/reader030/viewer/2022032904/6241f709c15df74fa761a600/html5/thumbnails/36.jpg)
The biggest change of technology is not software
or hardware, they are becoming faster and
cheaper, but the true change is about who is doing
what, where, with whom, for how much and how.
Bill Buxton
![Page 37: NUI Research](https://reader030.fdocuments.net/reader030/viewer/2022032904/6241f709c15df74fa761a600/html5/thumbnails/37.jpg)
What we build is about People,
It is not about technology.