8 Wearable Sensor System for Human Dynamics Analysis Wearable
Wearable Experience: New Educational Media for Knowledge Intensive Training
-
Upload
mikhail-fominykh -
Category
Education
-
view
492 -
download
2
Transcript of Wearable Experience: New Educational Media for Knowledge Intensive Training
Wearable Experiences for Knowledge Intensive Training
Mikhai l Fominykh, PhD
Europlan UK Molde Univers i ty Co l lege
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 1
June 28-30, 2016 • Vancouver, BC, Canada
Wearable Experience: New Educational Media for Knowledge Intensive Training
Invited speech
29/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 2
June 28-30, 2016 • Vancouver, BC, Canada
#WEKIT#edmedia#edmediaconf
About me
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 3
Project manager in workplace training with
Augmented Reality and Wearables
Associate Professor and researcher in
Emergency Management training
Adjunct Professor and teacherin Virtual Reality
Software developer in psychological treatment training
Software developer in welfare personnel training
Software developer in cooperation training for medical students
Background:Experiential learning
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 4
Mikhail Fominykh, Peter Leong, and Brenda Cartwright (2016/2017) A Study on Experiential Learning in a 3D Virtual Environment within a Professional Counseling Distance Course, in progress.
Background:Virtual Reality Recording
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 5
Real time class R0
R1
R1
R2
Recording
Replay
Recording
Mikhail Morozov, Alexey Gerasimov, Mikhail Fominykh, and Andrey Smorkalov (2013) Asynchronous Immersive Classes in a 3D Virtual World: Extended Description of vAcademia, DOI: 10.1007/978-3-642-38803-3_5
Background:Experiential learning
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 6
Background:Experiential learning
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 7
Photo by Mikhail Fominykh
Wearable ComputingAugmented Reality
Experience and KnowledgeWEKIT methodology
Information communication
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 9
Experience-communication
Images: http://www.unmuseum.org/, http://en.wikipedia.org/, http://digitalxtrememedia.com/; https://www.youtube.com/watch?v=7d59O6cfaM0, https://www.microsoft.com/en-us/research/project/holoportation/
https://www.youtube.com/watch?v=qAsu-RoVgWc
AR?
https://www.youtube.com/watch?v=7d59O6cfaM0
AR!
Object-centric
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 12
Action-centric
Performance Augmentationwith wearables
sensor-sensory loop:
super-real experience010000100001000010001010001111110
The guestenacts experiences
The stagerproduces narratives
WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING29/06/2016 13
Photo by Ravensbourne
Wearable Computing
Wearable computing pursues an interface ideal of a continuously worn, intelligent assistant that augments memory, intellect, creativity, communication, and physical senses and abilities*.
* Starner (2011) The challenges of wearable computing: Part 1, IEEE Micro, Volume 21, Issue 4, DOI: 10.1109/40.946681
Why Wearable Computing?
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 15
Better access and management of information
Portability during operation
Enable hands free or hands-limited use
Can run continuously; and attempt to sense the user’s current context (Rhodes, 2000)
Focuses the user’s attention and presents information in an unobtrusive, context-dependent manner (Kortuem et al.)
Wearability
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 16
Head mounted (headsets, glasses,
ear attachments)
Hand, wrist and arm mounted (gloves, watches, rings, bands)Chest, neck back
mounted (belts, bands, wests, shirts)
Leg, feet mounted (bands, shoes)
Photo by Mikhail Fominykh
Augmented Reality
Augmented Reality provides an enriched view onto the physical world, adding layers with contextually useful information, delivered visually or by stimulating other senses.
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 18
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 19
+80 /+100%
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 20https://www.youtube.com/watch?v=YJg02ivYzSs
Photo by Mikhail Fominykh
Reality is more rough
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 22
Working example
Kaj Helina, Timo Kuulaa, Jaakko Karjalainena, Iina Aaltonena, Fridolin Wildb, Juhani Viitaniemia, Antti Väätänena (2015) Usability of the ARgh! Augmented Reality system for on-the-job learning
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 23
Kaj Helina, Timo Kuulaa, Jaakko Karjalainena, Iina Aaltonena, Fridolin Wildb, Juhani Viitaniemia, Antti Väätänena (2015) Usability of the ARgh! Augmented Reality system for on-the-job learning
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 24
Working example
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 25
Working example
What is WEKIT?
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 26
Funding: EU Horizon 2020, ICT-20 2015: Technologies for better human learning and teachingBudget: EUR 2.753.143,75WEKIT Community https://wekit-community.org/WEKIT project website http://wekit.eu/
Experience and knowledge
Learning = converting
experience to knowledge
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 27
Mikhail Fominykh, Fridolin Wild, Carl Smith, Victor Alvarez and Mikhail Morozov: "An Overview of Capturing Live Experience with Virtual and Augmented Reality”, DOI: 10.3233/978-1-61499-530-2-298.
Experience and knowledgeLearning =
= converting experience to knowledge
immediate
experience
(aka ‘practice’)
information for a
master level
(aka ‘theory’)
separated
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 28
Experience and knowledgeLearning =
= converting experience to knowledge
immediate
experience
(aka ‘practice’)
information for a
master level
(aka ‘theory’)
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 29
Experience and knowledgeLearning =
= converting experience to knowledge
immediate
experience
(aka ‘practice’)
information for a
master level
(aka ‘theory’)
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 30
Experience and knowledgeLearning =
= converting experience to knowledge
immediate
experience
(aka ‘practice’)
information for a
master level
(aka ‘theory’)
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 31
Experience and knowledgeLearning =
= converting experience to knowledge
immediate
experience
(aka ‘practice’)
information for a
master level
(aka ‘theory’)
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 32
Experience and knowledgeLearning =
= converting experience to knowledge
immediate
experience
(aka ‘practice’)
information for a
master level
(aka ‘theory’)
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 33
Experience and knowledgeLearning =
= converting experience to knowledge
immediate
experience
(aka ‘practice’)
information for a
master level
(aka ‘theory’)
Exp
erie
nc
ed
lea
rne
r
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 34
MethodologyWhat is Wearable Experience?
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 35
The Wearable Experience training methodology aims to provide an innovative learning method that is based on the idea of capturing the experience of an expert and enabling trainees to wear it while re-enacting, thus giving the trainee access to the tacit knowledge of the expert and enabling master-apprentice knowledge sharing.
Capture Re-enact
Evaluate
MethodologyWhat is Wearable Experience?
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 36
The Wearable Experience training methodology aims to provide an innovative learning method that is based on the idea of capturingthe experience of an expert and enabling trainees to wear it while re-enacting, thus giving the trainee access to the tacit knowledge of the expert and master-apprentice knowledge sharing.
Giuseppe Scavo, Fridolin Wild and Peter Scott(2015) The GhostHands UX: telementoring with hands-on augmented reality instruction, DOI: 10.3233/978-1-61499-530-2-236
Captureexample
MethodologyWhat is Wearable Experience?
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 37
The Wearable Experience training methodology aims to provide an innovative learning method that is based on the idea of capturing the experience of an expert and enabling trainees to wear it while re-enacting, thus giving the trainee access to the tacit knowledge of the expert and master-apprentice knowledge sharing.
Wild, Perey, Helin, Davies and Ryan (2014) TELLME’s Learning Experience Model in action on the shop floor of a textile weaving mill
Re-enact example
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 38
Jan Schneider, Dirk Börner, Peter van Rosmalen and Marcus Specht (2015) Presentation Trainer: Polishing your communication skills, DOI: 10.1007/978-3-319-24258-3_52
Evaluate example
Mikhail Fominykh, Fridolin Wild, Carl Smith, Victor Alvarez and Mikhail Morozov: "An Overview of Capturing Live Experience with Virtual and Augmented Reality”, DOI: 10.3233/978-1-61499-530-2-298.
Capture | Re-Enact
WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING28/06/2016 39
TechnologyWhat is Wearable Experience?
An experience that is facilitated by devices located on or in the body that can include:
tracking user activity, biometric and environmental data
matching and mapping user performance
augmenting user’s activity with another user’s data
adapting user’s activity and behavior
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 40
# 41
Maintenance workers in Aeronautics
# 42
Imaging and diagnostic workers in Healthcare IT
# 43
Sub-system integration, operation & maintenance in Space
Photo by European Space Agency
WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING29/06/2016 44
WEKIT methodologyPhase ACapturing
WEKIT methodology Phase BRe-enactment
WEKIT methodology Phase CEvaluation
GoalsCapture of Expertise
What are relevant aspects of expertise and how to capture them?
Enable enactment of expertiseHow to actively support the transfer of expertise to a learner?
Design of learning taskHow should learning activities be designed to support / complementexpertise transfer?
Requirement generationHow should technology be designed to foster all of the above?
48WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING28/06/2016
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 49
Domain limited (Fernand and Herbert, 1996)
Failure to recall surface features (Schmidt and Boshuizen, 1993)
Inaccurate prediction judgement and advice (Hinds, 1999)
Inflexibility and bias (Sternberg and Frensch, 1992)
Qualities of ExpertsSuperior Mental Knowledge (De Groot, 1978)
Perceptual abilities (Lesgold et al., 1988)
Qualitative analysis (Klein, 1998)
Better strategies (Lemaire & Siegler, 1995)
Cognitive effort (Patricia 2003)
Shortcomings of Experts
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 50
Top-down approachIdentify tangible and intangible aspects relevant in all tasks
Find abstract key factors and rating scheme
Assess tasks and mechanisms according to factors and schemes
WEKIT methodology
Bottom-up approachStart to analyse concrete tasks in industrial settings
Classify them according to common scheme identifying aspects of expertise and learning
Try to find mechanism suitable to support expertise transfer
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 51
Task Classifications
Transfer mechanism H
igh
pre
cisi
on
Hig
h s
pee
d
Dec
isio
n t
akin
gC
olla
bo
rati
ve
task
sH
igh
mem
ory
lo
ad
Per
cep
tual
tas
k
Ass
erti
ve t
ask
Du
al t
ask
Per
form
ance
Man
agin
g U
nce
rtai
nty
Hig
h S
pat
ial
Task
Slow motion
Zoom 3 13
Think aloud protocol 1 8
Object enrichment 2
Contextualization 3, 9 3
Self awareness of physical state 14
Directed focus 15
Case identification 16
Spatial Simulation 18
In Situ Real-time Feedback 4 17
Mobile control
Virtual Post its 5 5
Haptic hints5,
12
Virtual/Tangible Manipulation 63,
10, 11
Remote symmetrical tele-assistance
7 7
Literature1 Schraagen, 1993
2 Tang et al., 2003
3 Quarles et al., 2008
4 BMW AR system, 2010
5 Sabine et al., 2011
6 Lahanas V et al., 2014
7 Chinthammit et al., 2014
8 Lundgrén-Laine et al, 2010
9 Curtis et al., 1998
10 Seichter, 2004
11 Wang, Chen, Gong, & Hsieh, 2007
12 Khademi et al., 2012
13 Mulloni, Dünser &Schmalstieg 2010
14 Larkin, 1983
15 Jarodzka et al., 2013
16 Mulzoff, 1990
17 Kotranza, Lind, Pugh & Lok 2009
18 Martin-Gutierrez et al., 2010
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 52
Tasks
Cognitive
Perceptual
Developmental
Human
Machine
Environment
Space
Time
Target
Domains Dimensions
Top-down approach
What is WEKIT?
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 53
TEL community – driven project
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 54
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 55
jTEL summer schooler
jTEL summer schooler
jTEL summer schooler
jTEL summer schooler
jTEL summer schooler
jTEL summer schooler
jTEL summer schooler
28/06/2016 https://wekit-community.org/ideas/
28/06/2016
http://bit.ly/wekit-facebookhttp://bit.ly/wekit-twitterhttp://bit.ly/wekit-linkedinhttp://bit.ly/wekit-plushttp://bit.ly/wekit-youtube
WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 59
D i s c l a i m e rT h i s p r o j e c t h a s r e c e i v e d f u n d i n g f r o m t h e E u r o p e a n U n i o n ’s H o r i z o n 2 0 2 0 r e s e a r c h a n d i n n o v a t i o n p r o g r a m m e u n d e r g r a nt a g r e e m e n t N o 6 8 7 6 6 9 . ht t p : / / w e k i t . e u /
Q & A#WEKIT
Presented by Mikhail Fominykh
mikhail.fominykh@europlan -uk.eu
http://www.mikhailfominykh.com/http://www.linkedin.com/in/fominykh/ https://www.facebook.com/mikhail.fominykh/ http://slideshare.net/mfominykh/
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 60