Post on 01-Feb-2016
description
CROSSMODAL EFFECTS IN TACTILE AND VISUAL
SIGNALING
Can we get the message through?
Sensing
Communicating
Effective action depends on…
Presenting
Processing
Understanding the situation
Picture of thermometer in
HMMWV taken 31 May, ‘04
in central Baghdad.
Sensing
Sensing… with the senses covered…
Effective action depends on
Communicating
Communicating… with the senses in extreme context
TOC - ABCS
GPS
Full FBCB2 onLower Tactical
Internet
L-Band
CO/PLT Leaders/Sgts
CO/PLT Voice Nets
SQD Leader
TM Leaders
SQD Leader
TM Leaders
SQD Leader
Squad Voice Net
Squad Voice Net
Squad Voice Net
BFT (FBCB2)
GPSBattalion SA NetDBCS (T)
DBCS (P)
Who else needs to know?
• Gilson & Fenton (1974) A tactile display presenting aircraft control information was comparable to using a visual display and in some cases actually produced a reduction in tracking error, altitude variations decreased as well as speed variations.
• Zlontik (1988) The use of tactile signaling helped pilots and air traffic controllers overcome some of their challenges.
• Gilland & Schlegel (1994) reported that tactile communication applied to the head lowered performance on a concurrent task.
• Sklar & Sarter (1999) Showed tactile signals are able to be perceived concurrently with auditory or visual information.
• Cholewiak, Brill, & Schwab (2004) found that for the torso, a ring of eight loci of vibration is the most one can resolve with accuracy exceeding 90%.
• Hoop, Smith, Clegg & Heggestad (2005) Tactile signals aided in interruption management.
• Pettitt, Redden, & Carstens (2006) found that tactile presented arm and hand signals could be interpreted faster than standard visually presented signals when undergoing dynamic movement tasks.
Literature Review
Tactile Patterns emulating Standard Army Hand Signals* (FM 21-60)
Tactons*…
Merlo, J.L., Stafford S.C., Gilson, R. & Hancock, P.A. (2006). The effects of physiological stress on tactile communication. Proceedings of the Human Factors and Ergonomics Society 50th Annual Meeting, San Francisco, CA.
Merlo, J.L., Terrence, P.I. Stafford S.C., Gilson, R. & Hancock, P.A., Redden E.S., Krausman A., Pettitt, R., White, T.L., & Carstens, C.B. (2006). The effects of dynamic environments and physiological stress on tactile communication and signaling. Proceedings of the Society for Human Performance in Extreme Environments 4th Annual Meeting, San Francisco, CA.
Merlo, J.L., Terrence, P.I. Stafford S.C., Gilson, R. & Hancock, P.A., Redden, E.S., Krausman, A., Pettitt, R., White, T.L., & Carstens, C.B. (2006). Communicating through the use of vibrotactile displays for dismounted and mounted soldiers. Proceedings of the Twenty-fifth Army Science Conference, Orlando, FL.
Multi-Modal Benefits• Spence and Driver (2004) Attention and then perception is largely a multi-
sensory construction
• Stein and Meredith (1993) bi-modal and tri-modal neurons have stronger cellular response with two sensory modalities than with uni-modal stimulation
• Wallace, Meredith, Stein (1998) found that combinations of two different sensory stimuli have been shown to significantly enhance the response of Superior Colliculus neurons above those evoked by either uni-modal stimulus
• Allman and Meredith (2007) found uni-modal visual neurons had their visual
response modulated by concurrently presented auditory stimuli
• Current research has explored these conflicts of congruent and incongruent cross-modal paradigms in many of the possible permutations and combinations of very basic visual, auditory, kinesthetic and tactile stimuli (Soto-Faraco, Lyons, Gazzaniga, Spence, & Kingstone, 2002; Soto-Faraco, Morein-Zamir, & Kingstone, 2005; Soto-Faraco, Spence, & Kingstone, 2004a; Spence & Driver, 1997; Spence & Walton, 2005).
Experimental Method
• visual only (video presentation of the signals)
• tactile only (tactile presentation of the signals)
• both visual and tactile simultaneous and congruent (i.e. the same signals were presented both visually on the video and through the tactile system)
• both visual and tactile simultaneous and incongruent (i.e. the visually presented signal did not match the presented tactile signal).
Equipment
• an acoustic transducers
• displace 200-300 Hz sinusoidal vibrations onto the skin
• C2’s contactor is 7 mm, with a 1 mm gap separating it from the tactor aluminum housing
• 17 grams
• 8 tactors mounted on an elastic belt with acoustic padding
• 8 X 45 degrees = 360 degrees
• Tactor Control Box (TCB) Stores tactile messages and contains bluetooth receiver
• Total system under 2 lbs
Spatial layout of Tactile Display
1FRONT
3
2
RIGHT
54
BACK
LEFT7
6
8
N
NE
E
SE
SSW
W
NW
Arm/ Hand Signal
Tactile Pattern Visual Pattern
Attention
Sequenced activation of three forward tactors creating a linear motion back and forth across the front of the body
Halt Four tactors simultaneously actuated on the sides of the body
Rally Sequenced activation of all tactors creating a circular motion around the body
Move Out
Sequenced back-to-front activation of tactors creating movement from each side of the body which converges in the front
NBCSequenced activation on both sides simultaneously creating three distinct impulses on the sides of the body
Method
Attention
RALLY
RALLY
Equipment
• Samsung Q1 Ultra Mobile Intel Celeron M ULV (900 MHz) 7” WVGA (800 x 480) LCD
SoftwareHardware
• Sound dampening earmuffs reduction rating of 11.3 decibels at 250 Hz
• Windows XP, tablet• LabVIEW 8.2 National Instruments• Microsoft Office 2003 • SPSS 11.5• Windows Media Player
Experimental Procedure
20 Participants
• 10 Males
• 10 Females
• Mode age 19 years
Training session
• Learned both visual and tactile signals simultaneously
• Advanced only after mastery
• Lasted approximately 10 minutes
Experimental block• 10 trials visual only • 10 trials tactile only• 20 congruent trials• 20 incongruent trials 60 total trials per block
2 x Blocks = 120 trials
Signals
• Attention
• Halt
• Move Out
• NBC
• Rally
Screen Capture from Visual interface
Participants chose the appropriate signal as fast a possible after signal presentation. The tactile only presentation showed a video of the soldier standing at the rigid position attention.
0
500
1000
1500
2000
2500
3000
Concurrent Visual Only Tactile Only
Condition
Mea
n R
espo
nse
Tim
e (m
s)
Initial Results
Response Time x Condition
t(19)=-2.25, p≤.04 t(19)=-2.16, p≤.04
t(19)=-3.98, p≤.01
Oneway ANOVA
( F(1, 19)=473.445, p<.000, partial eta squared = .961, observed power, β= 1.000 )
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
100.00%
Concurrent Visual Tactile
Condition
Per
cen
t C
orr
ect
Accuracy x Condition
t(19)=2.00, p≤.06
t(19)=4.03, p≤.01
t(19)=1.27, p≤.22
Initial Results
0
500
1000
1500
2000
2500
3000
Concurrent and congruent Visual Only Tactile Only Response not matchingeither presentation
Condition
Me
an
Re
sp
on
se
Tim
e (
ms
)
Initial Results
0
5
10
15
20
25
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Partcipant
Fre
qu
ency
of
Mo
dal
ity
Sel
ecti
on
Tactile
Visual
0
5
10
15
20
25
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Partcipant
Fre
qu
ency
of
Mo
dal
ity
Sel
ecti
on
Tactile
VisualBlo
ck 2
Blo
ck 1
Discussion
• Data should provide valuable information on:– Congruency effects– Selected attention– Interference effects– Tactile signal efficiency– Modality interactions
• Future research for tactile signaling includes:– Spatial references– Temporal importance– Alternate inputs
Questions?