Interfacing with the Sense of Touch: The Somatosensory ... haptic collaboration.pdf · Interfacing...
Transcript of Interfacing with the Sense of Touch: The Somatosensory ... haptic collaboration.pdf · Interfacing...
http://www.lsr.ei.tum.de
A. Peer
Institute of Automatic Control EngineeringTechnische Universität München
Interfacing with the Sense of Touch:The Human Somatosensory System (Network)
and Dynamic Tactile Displays
Human‐Centered Robotic SystemsHaptic Human-Robot Interaction
Analysis of human-human interaction
Synthesis of interactive robotic systems
Intention recognition
Advanced teleoperation controllers
Single and multi-user scenarios
Joint decision makingAdaptation
Telepresence and Teleaction Systems
Brain and BodyComputer Interfaces Human Motor Control
fMRI and TMS studieswith Max-Planck Institute
Intention recognition
fromphysiological
signals
Embodiment of intentionsand personalization of actions
IntentionrecognitionIntention
recognitionEmbodiment
ofIntentions
Embodimentof
Intentions
Bilateralcontrol
Sharedcontrol
Supervisorycontrol
Autonomouscontrol
degree of autonomy
Automatic shifting ofrobot autonomy
Design and control of human-system interfaces
Modellingof human
motor behavior
Haptics= from the greek haphe, the sense of touchhaptesthai, to touch something
3
Haptics – The Sense of Touch
The 5 human senses:
sight hearing smell taste touch
4
The Sense of Touch
Somatosensation:• Tactility
Perception of vibration, shear forces, pressure
• Thermal perception
• Pain perception
Kinesthesia:• Perception of motion and forces
Proprioception:
• Perception of posture and position
5
What is a Tactile Display?
Tactile properties:
• Texture
• Roughness
• Vibration
• Geometry of small objects (distribution of pressure)
• Sliding shear forces
• Temperature
A tactile display is a physical device that can display tactile propertiesto a user.
6
Application Areas of Tactile Displays• Teleoperation (medical application: palpation in minimal invasive surgery)
• Automobile industry
• Military application (nagivation, orientation‐assistance, increase of situation awareness)
• Assistance of the blind (Braille systems, orientation and navigation assistance)
• Commercial applications (vibration alarms, games,
enhancement of graphic displays)
• Psychophysical studies
[www.immersion.com]
[www.top‐braille.com]
[iDrive, BMW AG] [Erp et al. 2004]
A Special Type: Dynamic Tactile Displays
7
Dynamic tactile display: generates tactile patterns on a stationary skin surface
[MIT tactile vest]
[http://www.ultracane.com]
[www.freedomscientific.com]
Historical Displays: The Optacon
Optacon (optical‐to‐tactile‐conversion) (Livill, Telesensory, 1962‐2000)
• 24‐by‐6 matrix of tiny metal rods, each independently vibrated by a piezoelectric reed (bimorph)
• rods are vibrated according to black parts of the image, thus forming a tactile image of the letter being viewed by a camera module passed over the text
• Typical reading rates: 30‐50 wpm
• Also reading of images possible
1990s: page scanners with optical letter
recognition appear on the market, which are
less expensive, have a better learning curve,
and allow faster reading
8
Historical Displays: TVSS
Tactile Vision Substitution System (TVSS) [White et al., 1970]:
• Aimed at converting visual information to patterns for the skin
• Consists of TV camera and an array of vibrators built into the back of a chair
9
[White et al.,1970]
Pictorial vs Coded Approach
Tactons are structured, abstract, tactile messages which can be used to communicate information non‐visually.
10
Pictorial versus coded approach [Geldard, 1974]Analogic versus synthetic system approach [Sherrick, 1975]
What is the analogy in the tactile domain?
Natural parameters: frequency, amplitude, duration, waveform, rhythm
[Brown et al., 2005]
Known coded approaches in other modalities:• Vision: icons, which are defined by a meaningful image• Audition: earcons, which are defined by melody, pitch and timbre
Historical Displays: Vibratese Language
• Three intensities, three durations, five loci
• Most frequently occurring letters were assigned to shortest signals
• Numbers get longest duration
• Vowels are represented by each single vibrator
• Vibratory characters for most commonly recurring words: the, of, and, in
11
[Geldard, 1957]
Achieved speed:• 38 words per minute
12
Agenda
The somatosensory system
Types of tactile displays
Psychophysics of the sense of touch
Spatio‐temporal patterns on the skin
Tactile illusions – effects of motion
Application areas and selected examples
13
Anatomy of the Skin
Dermal layers:
• Epidermis
• Dermis/ Korium
• Subkutis
Hairy/glabrous skin:
• Different receptors
[www.kidsnet.at]
Epidermis
Subkutis
Dermis/Korium
Form: discoid receptorsLocation: in Epidermis close to Dermis
Form: Egg‐shaped stack of flattened cells with intertwined nerve fiber
Location: in Dermis under EpidermisSize: 30 x 80 um
Form: branched nerve fibers encloused by a cylinder‐shaped capsule
Location: DermisSize: 0.1 x 0.5‐2mm
Form: capsule build of multiple layersLocation: are located deep in the skin, also in inner
organs and jointsSize: 1x2mm, largest skin structure
14
Tactile Receptors
[E. B. Goldstein, Wahrnehmungspsychologie, 1996]
Merkel‘s disks
Meissner corpuscle
Ruffini corpuscle
Pacini corpuscle
15
Skin Receptors in Glabrous and Hairy Skin
Glabrous skin:• Merkel‘s disks• Meissner‘s corpuscle• Ruffini endings• Pacini corpuscle• Free nerve endings
Hairy skin:• Merkel‘s disks• Ruffini endings• Pacini corpuscle• (Hair follikel)• (Free nerve endings)
[Kandel, Schwartz, Jessell, Principles of neural science, 2000]
Microneurography
Insertion of electrode under the skin and recording of a single nerve fiber
[www.rrk‐berlin.de, 2008]
16
Nerve Fiber Types
4 types:
• Fast/slow adapting fibers (FA, „fast adapting“ / SA, „slowly adapting“)
• Small and large receptive fields
[Deetjen, Speckmann, Physiologie, 1994]
17
Receptive fields
Ada
ptat
ion
Small,sharp borders Large, washy borders
fast, onlydynamicresponse
slow,static
response
time time
time time
Nerve Fiber Types and Tactile Receptors
Type of fiberSize of
receptive field
ReceptorOptimal stimulus
Function
Small Pressure Intensity detectors
Small Tap on skin Velocity detectors
Large
Stretching of the skin, movement of joints
Intensity detectors
Large Fast vibrationAcceleration detectors
18
Merkel‘s disks
Meissner corpuscle
Ruffini corpuscle
Pacini corpuscle
time
time
time
time
19
Haptic Information Processing
↓ Skin receptors detect stimulus
↓ Nerve fibers enter spinal cord
↓ Two lines of afferent nerve fibers:– Medial Lemniscus: thick fibers, sensation of touch
and perception of position of limbs (proprioception)
– Spinothalamic tract: thin fibers, temperature and pain perception
↓ Entry into Thalamus and interconnection of fibers by means of synapses
↓ Forwarding from Thalamus to primary somatosensory cortex
[E. B. Goldstein, Wahrnehmungspsychologie, 1996]
signal travels to brainin the spinal cord
signal to spinal cord
stimulus
Somatosensory Cortex
Humunculus
• High resolution requires
⇒ small receptive fields
⇒ high receptor density
• Body regions with high receptor density are represented by larger areas in the Cortex
20
21
Agenda
The somatosensory system
Types of tactile displays
Psychophysics of the sense of touch
Spatio‐temporal patterns on the skin
Tactile illusions – effect of motion
Application areas and selected examples
Types of Tactile Displays
22
Categorization according to stimulation type:
1. Mechanical stimulation
a) Skin indentation
b) Vibration
c) Lateral displacement
2. Electro‐tactile stimulation
3. Functional neuromuscular stimulation
23
Pneumatic Displays
Principle:• Energy in form of compressed air is
transformed into motion
Technological realisation:• Micro air jets• Air pockets• Inflatable air‐rings
Advantages:• Simple, clean, cheap• Non invasiv
Disadvantages:• Can lead to discomfort• Low bandwidth (< 10 Hz)
air pocketsInflatable pneumatic rings
data glove forposition sensing
air jets
[Shimoga 1993]
Pin‐Array Displays
Principle:
• Stimulation of FA I und SA
receptors by vertical motion of
pins
Technological realisation:
• shape‐memory‐alloy
• Piezo linear motors
• Elektroactive polymeres
Elektroactive polymere[Nanoarchitecture.net, 2008]Piezo linear actuator
Shape memory alloy[Burdea 1996]
[Forschungszentrum Karlsruhe]
piezoelectric actuator moved object
friction
actuatedrod
slow elongationsticking
fast shrinkingsliding
Advantages:
• Display of localized forces and contact geometries
Disadvantages:
• Can lead to pain
• High energy consumption
• Actuator density limited by technology
24
25
Vibro‐tactile Displays
Principle:• Stimulation of FA II receptors by means of vibration
Technological realisation:• Voice coils• Miniature loudspeaker• Piezoelectric actuators• Motor with excentric load
Advantages:• Can lead to discomfort or pain • Compact construction
Disadvantages:• Miniature loudspeaker: high weight• Voice coils, Piezos: noise Piezoelectric actuators
Lautsprecher
[Wikipedia: Tauchspule, 2008]
26
Shear Force Displays
Principle:
• Shear forces simulate sliding over object
Technological realisation:
• Bimorph piezo actuators
• Elektroactive polymeres
• Rotation motors
Elektroactive polymere[Nanoarchitecture.net, 2008]
Bimorph piezoactuator[Nanoarchitecture.net, 2008]
[Fritschi et al., 2008][Chen & Marcus, 1994]
[Hayward et al.]
27
Electro‐tactile Displays
Principle:• Stimulation of receptors by electric pulses
Technological realisation:• Mini‐electrodes mounted on the skin
Advantages:• Low energy consumption and weight• Small moving parts• Uniform contact with skin
Disadvantages:• Danger of discomfort• Danger of pain in case of two small electrodes and
too high currents• Variable skin conductance
electrodes
[Shimoga 1993]
[Kaczmarek et al. 1995]
Functional Neuromuscular Stimulation (FNS) ‐ Displays
Principle:• Stimulation of neuromuscular systems (not the skin)
Technological realisation:• Mounting of electrodes on muscles and nervous system
Advantages:• No limitation by means of technological realisation
Disadvantages:• Invasive, risky• High complexity• Can cause pain• High responsibility
PerceptionNeuronalResponse
Stimulus
Psychophysics
Neuro‐physiology
Neuronalcoding
[Steve Hsiao]
28
29
Agenda
The somatosensory system
Types of tactile displays
Psychophysics of the sense of touch
Spatio‐temporal patterns on the skin
Tactile illusions – effects of motion
Application areas and selected examples
30
What is Psychophysics?
psychophysics
Psychophysics is the subdiscipline of psychology which focuses on the relationship between stimulus and its perception by a human being
stimulus perception
Why do we need knowledge about human perception?
• To design tactile displays appropriately
• To design an adequate and useful haptic rendering
31
Origin of Phsychophysics
1860 Fechner‘s Elements of Psychophysics→ introduced first methods to measure
mental processes→ established first relationship between physical
and mental states
1879 Wilhelm Wundt founded first laboratory of experimental psychology in Leipzig
Gustav Theodor Fechner
32
Absolute und Difference Threshold
Absolute threshold:
Smallest energy of a stimulus , that can lead to a sensation
Difference threshold/limen (DL) or just noticeable difference (JND):
minimum amount by which a stimulus intensity must be changed in order to produce a just noticeable variation in sensory experience (JND=DL)
physical psychologicalDL1=JND1 =ΔΦ1
DL2=JND2 =ΔΦ2
DL3=JND3 =ΔΦ3
DL4=JND4=ΔΦ4
Φ0
Φ1
Φ2
Φ3
Φ4
Ψ0=0
Ψ1=1
Ψ2=2
Ψ3=3
Ψ4=4
33
Linear relationship between the differential threshold and the
stimulus intensity :
Weber‘s Law
Example: Discrimination of weights
found Weber fraction: c = 1/30
Heavy weights need to differ more than leight, so that they are perceived differently
[Gescheider, Psychophysics, 1985]
Adaptation Problem
• Absolute threshold increases
• Changes in subjective magnitude of supra‐threshold stimuli
sensation magnitude declines during the exposure
• Recovery time ranges from a few seconds to several minutes
• Adaptation within one tactile channel does not affect the sensitivity of others
Adaptation: decrement in sensory magnitude and in the detection threshold following more prolonged stimulation
34
35
Perception of Vibrations with Different Frequencies
perception thresholds
• Receptors react differently to different frequencies
• Isolation of receptors by cooling and masking
• Perception of vibration by most sensitive receptor
[E. B. Goldstein, Wahrnehmungspsychologie, 1996]
smallest perceivablevalue at ~200‐300 Hz
reference value: 1μm amplitude of vibrator*
*
Frequency [Hz]
SA I
SA II
RA I
PC
Perception of Vibrations with Different Frequencies
• Dependence on contactor size:
36
[Verrillo, 1963]
Discrimination of Amplitudes and Frequencies
37
• Range: 20 – 1000 Hz
• Weber fraction:
≈ 0.03
[Franzen & Nordmark, 1975]
DL of vibrations with different frequencies:
[Geldard, 1957]
DL of vibrations with different intensities:
• Range: 0‐55dB SL
• Weber fraction:
0.05 (0.4 dB) [Knudsen et al., 1928]
0.11 (0.9 dB) [Schiller, 1953]
0.3 (2.3 dB) [Sherrick et al., 1950]
Discrimination deteriorates as frequency increasesDiscrimination deteriorates as intensity increases
• Two stimuli with same amplitude, but different frequencies are felt differently
• Subjective magnitude grows more rapidly for body loci with lower sensitivity
Perceptual Interaction of Amplitude and Frequency
38
[Verillo et al., 1972][Verillo et al., 1969]
max. intensity (above vibrations become
unpleasant)
Perceptual Interaction of Amplitude and Frequency
3‐5 values of vibration rates
39
Varying vibration ratesConstant sensory magnitude: 36 dB
Varying vibration ratesVarying sensory magnitude: 20, 28, 36 dB
5‐8 values of vibration rates
[Sherrick 1985]
Detection of Vibrations with Different Durations
40
[Verrillo, 1965]
Short stimulus is more difficult to detect than a long one
Discrimination of Vibrations with Different Durations
• Stimulus < 0.1 s can be mistaken for a „poke“ or a „nudge“
• Stimulus > 2 s is too time consuming
• 25 discriminable steps from 0.1 s to 2 s
41
[Geldard, 1957]
Number of Distinguishable Stimuli
Psychophysical dimension
Range Number of JNDs under laboratory conditions
Number of distinguishable stimuli under practical conditions
Intensity 0 – 55 dB SL 15 [Geldard, 1957] 3 [Geldard, 1957]
Frequency 20 – 1000 Hz 3 – 8 [Sherrick 1985]
Duration > 0 25 (from 0.1 – 2s)[Geldard, 1957]
4 – 5 (from 0.1 – 2s)[Geldard, 1957]
42
Waveform
43
• Subtle differences in waveform cannot be perceived on the skin
• Few tactile waveforms can be used without training unlike in audio
• Limited bandwidth of devices unifies waveforms
Square wave played with TACTAIDSine wave played with TACTAID
[Brown et al., 2005]
Amplitude Modulation
• Sinusoid amplitude modulation leads to perceptually different waveforms
• Known relationship between amplitude modulation and roughness:
– modulated sinusoid feels rougher than non‐modulated
– perceived roughness increases with decreasing modulation frequency
– unmodulated sine wave feels less rough than any other stimulus
44
250 Hz sinusoid modulated with 30 Hz sinusoid
Rhythm: Compound, Hierarchical, andTransformational Tactons
Compound tactons:
• Tactons can be combined to create compound messages
45
Hierarchical tactons:
• Tactons can be combined in a hierarchical way: each tacton is a node in a tree and inherits properties from the levels above it
Transformational tactons:
• Transformational tactons have several properties, each represented by a different tactile parameter
e.g. tactons for files in computer:
− file type rhythm
− size frequency
− creation date body location[Brewster & Brown, 2004]
Localization of Vibratory Stimuli
46
Law of mobility [Vierordt, 1870]: the more „mobile“ the body site , te greater is the sensitivity either to the location of a touch or to the separation between touched locations.
[Cholewiak and Collins, 2003]
Anhaltspunkte (anchor points) [Boring, 1942]: local signs that can provide reference points against which the observer can measure the position of less well localized tactile stimuli
Localization of Vibratory Stimuli
47
[Cholewiak et al., 2004]
Neurological redundancy of abdominal anchor points
Ecological significance
Artificial Anchor Points
• Artificial anchor points can be generated by including an odd site (stimulus that has a different quality compared to the rest of stimuli)
48
[Cholewiak and Collins, 2003]
Effect of Age on Location Discrimination Capability
• With increasing age the density of receptors decreases drastically (e.g. Meissner corpuscles fall from 24 per mm2 in young people to 8 per mm2 in 70‐year old seniors)
• Age affects absolute discrimination threshold drastically
• Localization capability of vibrations is only little affected by age
49
[Cholewiak and Collins, 2003]
Number of Stimuli: Channel Capacity
Confusion matrix:
50
Encoded information:
Transmitted information:
Overall recognition rate: 73,19 %
Channel of capacity of the observer for absolute judgements of nonvisual unidimensional stimuli has asymptote near level of 2.5 bits (about 6 alternatives) [Miller, 1956]
[Cholewiak et al., 2004]
51
Agenda
The somatosensory system
Types of tactile displays
Psychophysics of the sense of touch
Spatio‐temporal patterns on the skin
Tactile illusions – effects of motion
Application areas and selected examples
52
Spatial Resolution of Stimuli
• Spatial resolution:
– Determined by two‐point limen
– Increases with warmth and decreases with age
[Kandel, Schwartz, Jessell, Principles of neural science, 2000]
Temporal Resolution and Order of StimuliAbsolute identification of temporal order of several sequentially presented stimuli to different
body sites requires very slow rates of presentation:
• Temporal resolution without order judgements: ~5ms
• Threshold for temporal order judgements: ~ 20ms for two stimuli
• Threshold increases significantly with more stimuli (about 500ms)
53
[Hirsch and Sherrick, 1961]
• Correctness of discrimination between pairs of successive vibratory patterns, requires to avoid as much as possible common elements in the patterns
• No single vibrator contributes to error production more than another
Discrimination of Patterns ‐ Communality
[Geldard and Sherrick, 1965]
Effects of Multiple Stimulation Sites
• Masking: reduced ability to detect a stimulus in the presence of a background or masking stimulus (simultaneous, forward, and backward masking)
• Enhancement: occurs when the presence of a brief stimulus causes a second stimulus to appear to be of greater intensity than when it is presented alone
• Summation: refers to the total or combined sensation magnitude of two stimuli occurring close together in time
• Suppression: occurs when the presense of one stimulus decreases the ability of the subject to detect a second stimulus when the two simuli are delivered to different places on the skin. Masking from a remote site.
55
[Verrillo and Gescheider, 1991]
Masking: Change Numbness
„Change numbness“ [Gallace et al., 2006]:
• Changes between two vibrotactile patterns is perceived significantly less when:
– a pause is made between patterns
– a masking patterns (e.g. all vibrators activated) is shown in between
56
[Gallace et al. 2006]
sensitvity of observer
Enhancement and Summation of Successive Stimuli
• Summation:
– 3 dB effect when all frequencies are within one mechanoreceptor system
– effect more pronounced across all time intervals when frequency of each stimulus activates different tactile systems
57
1st stimulus
2nd stimulus comparisonstimulus
Summation
Enhancement
• Enhancement:
− Enhancement effect is strong for short time intervals when the intensity of the first stimulus is raised 10 dB above the second
− Effect diminishes when time interval is lengthened
− Enhancement effects disappear when the first and second stimuli are presented in different tactile systems
[Verrillo and Gescheider, 1975]
Suppression: Threshold Elevation
Temporal factors have an effect on
threshold elevations:
• Threshold elevation is maximal if no difference in the arrival time exists between signals coming from test site and masking site (neural signalling speed 50‐70 m/s)
• If arrival time is compensated for, masking effect is independent of stimulus loci
Vibrotactile masking is controlled by differential time delays and not by marker placement
58
maskerlocated at I
[Gilson, 1969]
59
Agenda
The somatosensory system
Types of tactile displays
Psychophysics of the sense of touch
Spatio‐temporal patterns on the skin
Tactile illusions – effects of motion
Application areas and selected examples
Tactile Illusions – Effects of Motion
Apparent motion, „Cutaneous phi“:
• „powerful ‚gouging‘ that moves from one stimulus site to the other at a rate depending on the distance between sites and the duration of the time between onsets“ [Sherrick & Rogers, 1966]
• Distances < 30 cm can be covered
• Transient pressure pulse at each
stimulus site:
(visual movement)
(tactile movement)
(tactile movement)
[Sherrick and Rogers, 1966]
Setting for optimal effect of illusion
Tactile Illusions – Effects of Motion
„The Cutaneous Rabbit“ – Sensory Saltation [Geldard and Sherrick, 1972]
• Train of taps is presented consecutively at different locations on the skin
• Perceived vibration „hops“ from one place to the other
– With constant timing, but reduced number of taps at each locus, hops get longer
– Increasing number of taps makes hops shorter
61
[Geldard, 1975]T ~ 25-200 ms
perceived Phantomvibrations vibration
actuators
2-35 cm
Tactile Illusions – Effects of Motion
„Funneling“ [Gardner and Spencer, 1972]
• Simultaneous activation of multiple vibrators: instead of one vibration at the location of the single vibrations, only one position in the middle is perceived, which is stronger in intensity than the single vibrations
• Works also with electrostimulation
62
T ~ 5ms
perceivedPhantomvibration
vibrationactuators
~ 2cm
Two-dimensional:
[Tachi 1985]Possible reasons:• Mechanical properties of skin• Interaction of receptors• Central information processing
63
Agenda
The somatosensory system
Types of tactile displays
Psychophysics of the sense of touch
Spatio‐temporal patterns on the skin
Tactile illusions – effects of motion
Application areas and selected examples
Application Area I: Enhancement of Desktop Interfaces
• Reduction of overload by visual information
• Increase of focus of attention: users can concentrate visual attention on primary task, while tactile feedback provides information from secondary task
• Enhancement of interaction with buttons, scrollbars, menus
• Improvement of pointing, targeting, and steering type interactions
64
[iFeel mouse of Logitech]
Eight task properties of a progress bar[Conn et al., 1995]
• Acceptance• Scope• Initiation• Progress• Hearbeat• Exception• Remainder• Completion
Tactons for Progress Information
Tactile progress bar:
• Amount of remaining time proportional to the time between two pulses
• Time between two pulses is scaled to the amount being downloaded
Reaction time to task completion:• Visual: 13.54 seconds (SD 5.2) • Tactile: 8.7 seconds (SD 5.6)
[Brewster and King, 2005]
Application Area II: Mobile and Wearable Devices
• Mobile information systems:
– Tactile patterns provide information on the caller, replace or enhance items on the display, aid in the navigation of the devices menus so that the user does not need to look at the screen
– Tactons can describe information such as the type of building, type of shop, number of stairs, location of rooms, …
– Information on stock market data
• Mobile orientation systems
– Information about orientation in space for pilots, microgravity environments
• Mobile navigation systems
– Directional information, compass‐like display of direction
• Mobile communication systems
– Tactile icons, tactile melodies
66
[www.immersion.com]
[Erp et al., 2004]
Transformational Tactons to Code Messages and Calls
• Type of message coded by 3 roughness grades: smooth, rough, very rough
• Type of call coded by 3 rhythms:
– voice call
– text message
– multimedia message
[Brown et al., 2005]
• Spatial disorientation is a serious threat to pilots
in aircrafts
• Somatogyral illusion may lead to graveyard spin
• Tactile displays can help to overcome this threat
• Two rendering modes:
− Inside‐out: instrument presents Earth‐fixed
orientation
− Outside‐in: signal rotates at the same angular
velocity and in the same direction with respect
to the observer as the observer rotates with
respect to Earth
Orientation‐Aid
68
24x2tactors
[Erp et al., 2006]
Vibrotactile In‐Vehicle Navigation System
Typical threats of in‐vehicle support systems:
• Overload of traditional sensory channels (vision, audition)
use free sensory channel
• Overload of the cognitive capacities that a driver has
at disposal
use intuitive information presentation that
minimizes information processing
Vibrotactile display for navigation tasks
• Eight vibrating elements mounted in a
driver‘s seat
• Coding of distance to next waypoint by rhythms
and direction of course change by location of
activated tactors
69
[Erp et al., 2004]
Waypoint Navigation at Sea and in the Air
• Eight vibrating elements around torso
• Direction coded by location of vibration
• Passing of waypoint by activation of all
vibrators
70
Pilot following instruction of the navigator
Pilot following instruction of the tactile navigation system
[Erp et al., 2004]
Application Area III: Visually Impaired Users
• Display of symbolic, graphical, and multidimensional data
– Tactons advance Braille like Earcons advance synthetic speech
– Representation of graphical information
– Visualization of multidimensional data by mapping to different dimensions of a stimulus (amplitude, frequency, locus, ...)
• Tactile waypoint navigation systems
• Blind walking canes
• Tactile game consoles
71
[http://www.ultracane.com][VTPlayer mouse]
Reading of Charts by Visually Impaired
• Combination of graphics tablet as absolute positioning device and VTPlayer mouse on which information is displayed dynamically
• Axes and bar representation with tangible tactile relief
• Buttons on stylus to trigger audio information on the bars
[VTPlayer mouse]
[Wall and Brewster, 2006]
Summary
• Dynamic tactile displays are not well researched yet
• Many open research questions:
– Psychophysics of touch:• Information carrying capacity of different stimuli
• Spatiotemporal masking effects
• Modelling of dynamic effects
– Technology: • Adaptation to psychophysical findings (actuator density, bandwidth, etc.)
• Ergonomic aspects
– Haptic rendering:• Integration of knowledge from psychophysics
– Multimodality: How do multiple modalities interfere with each other?
73
Selected Literature
• S. Wall, S. Brewster, Sensory substitution using tactile pin arrays: human factors, technology and application, Signal Processing 86, 3674‐3695, 2006
• S. A. Brewster, S. A. Wall, L. M. Brown, E. E. Hoggan, Tactile Displays, The Engineering Handbook on Smart Technology for Aging, Disability and Independence (John Wiley & Sons), ISBN 978‐0‐471‐71155‐1, 2008
• R.T. Verillo and G.A. Gescheider, Tactile Aids for visually impaired: ch1: Perception via the sense of touch. Ed. Summers, Whurr Publishers Ltd. London, 1992
• J.C. Craig, C. E. Sherrick, Dynamic tactile displays, Tactual perception, a sourcebook edited by William Schiff and Emerson Foulke, Cambridge University Press, 1982
74