Haptic Interface using Sensory Illusion
Tomohiro Amemiya
*NTT Communication Science Labs., [email protected]
NTT Communication Science Laboratories 2/39
IEEE VR 2008 Tutorial #2 © 2008 [email protected]
OutlineIntroduction
Haptic Interface using Sensory IllusionsKinesthetic force illusionVestibular illusionTactile illusions
Summary
NTT Communication Science Laboratories 3/39
IEEE VR 2008 Tutorial #2 © 2008 [email protected]
A. Kitaoka (2006),http://www.ritsumei.ac.jp/~akitaoka/
NTT Communication Science Laboratories 4/39
IEEE VR 2008 Tutorial #2 © 2008 [email protected]
E. Adelson (2002), http://web.mit.edu/persci/people/adelson/checkershadow_illusion.html
NTT Communication Science Laboratories 5/39
IEEE VR 2008 Tutorial #2 © 2008 [email protected]
Stimulation
Environment HumanPerception InformationSensation
=
Interface
Different!Measurement
Nonlinearity betweensensation and perception
==
Stimulation Perception InformationSensation
Sensory illusions
NTT Communication Science Laboratories 6/39
IEEE VR 2008 Tutorial #2 © 2008 [email protected]
Perceived World is not Identical to Physical World.
A series of still photos Motion pictures
Information compression by subtracting color
difference information
Almost sameas original picture
Subtracting frequencies difficult for humans to hear
Physical world Psychological world
Compression algorithm in mp3 format or Mini Disk
Compression algorithm on JPEG
Basis of animation
Illusion
Almost sameas original sound
NTT Communication Science Laboratories 7/39
IEEE VR 2008 Tutorial #2 © 2008 [email protected]
Human Interface using Sensory IllusionHuman science
To understand the distortion between the physical and psychological world
EngineeringTo utilize the distortion to build information displays
T. Amemiya, I. Kawabuchi, H. Ando, T. Maeda, "Double-Layer Slider-Crank Mechanism to Generate Pulling or Pushing Sensation without an External Ground", In Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2007), 2101-2106, Oct 2007.T. Amemiya, H. Ando, T. Maeda, ""Lead-Me Interface" for a Pulling Sensation from Hand-held Devices", ACM Transactions on Applied Perception 5(4), 2008.
Buru-Navi
[ Laval Virtual “Grand Prix” Award (2007) ]
[ Laval Virtual VR interface Award (2007) ]
NTT Communication Science Laboratories 9/39
IEEE VR 2008 Tutorial #2 © 2008 [email protected]
???
Physical constraints∥
Both the user and device must be connected toan external ground in order to generate force
Haptics in Mobile Devices??
Relatively heavySpatially constrained
NTT Communication Science Laboratories 10/39
IEEE VR 2008 Tutorial #2 © 2008 [email protected]
Related Work: Non-grounded Force Displays
Prior non-grounded force displays:Unable to generate constant forceUnable to generate translational force
Gyro Effect(Yano et al., 2003)
Angular Momentum(Tanaka et al., 2001)
Mechanical Brake(Ando et al., 2002)
NTT Communication Science Laboratories 11/39
IEEE VR 2008 Tutorial #2 © 2008 [email protected]
My Approach Creating a Force Sensation
Periodic translational motion (=oscillation)
Not creating a “physical” force
Exploiting characteristics of human perception
To create a constant and translational force by non-grounded displays
NTT Communication Science Laboratories 12/39
IEEE VR 2008 Tutorial #2 © 2008 [email protected]
Psychophysical quantity is notidentical to physical quantity.
1. Exploiting Characteristics of Human Perception
If the asymmetric oscillation is designed well, a directed force sensation can be induced.
1.0
0.0
0.2
0.4
0.8
0.6 ∫+
=Tt
td 0)( ττα
∫+
≠Tt
td 0)( τταϕ o
x = α(t): accelerationT: time cycle
y = ϕ(x): sigmoid curve of perception
Psychophysical quantity
Physical quantity
NTT Communication Science Laboratories 13/39
IEEE VR 2008 Tutorial #2 © 2008 [email protected]
2. Slider-crank Mechanism to Generate Asymmetric Oscillation
The weaker acceleration is not noticed.The washout-effect of the weaker acceleration allows humans to be tricked into perceiving this as uni-directional force.
Stimuli
Perception
Illusion
NTT Communication Science Laboratories 14/39
IEEE VR 2008 Tutorial #2 © 2008 [email protected]
“Buru-Navi” PrototypeCrank wheel
Swinging arm
Guide roller
Connecting rod
Weight
Motor
Swinging armMotor pinionCrank wheel(with crown gears)
NTT Communication Science Laboratories 15/39
IEEE VR 2008 Tutorial #2 © 2008 [email protected]
http://www.dailymotion.com/video/x1t8yl_buru-navi
http://labolib3.aecl.ntt.co.jp/member_servlet_home/asx/Large/2006-08.asx
Introduction movie
NTT Communication Science Laboratories 16/39
IEEE VR 2008 Tutorial #2 © 2008 [email protected]
Asymmetric oscillation(test stimuli)
Evaluation
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-200
-100
0
100
Time [s]
Acc
eler
atio
n [m
/s2 ] 5 cycle/sec
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-200
-100
0
100
Time [s]
Acc
eler
atio
n [m
/s2 ] 5 cycle/secSymmetric oscillation
(control stimuli)
xy
xy
NTT Communication Science Laboratories 17/39
IEEE VR 2008 Tutorial #2 © 2008 [email protected]
Frequency [cycles/sec]
Symmetric oscillation(comparison)
0102030405060708090
100
3 5 7 9 11 13 15
Proportion of “Being pulled” to x-direction [%]
Asymmetric oscillation
Asymmetric Oscillation Generates a Pushed/Pulled Sensation
x
x
NTT Communication Science Laboratories 18/39
IEEE VR 2008 Tutorial #2 © 2008 [email protected]
Another Approach to Generate an Asymmetric Oscillation
Spring-cam mechanismResonant motion by linear actuators
Cam Follower
Extension Spring
CamSlider Linear Guide
xy
Phase 1 Phase 2
T. Amemiya, H. Ando, T. Maeda, "Hand-held Force Display with Spring-Cam Mechanism for Generating Asymmetric Acceleration", In Proc. of World Haptics Conference 2007, pp. 572-573, 2007.
NTT Communication Science Laboratories 19/39
IEEE VR 2008 Tutorial #2 © 2008 [email protected]
Example of Applications (1): Accessibility
Collaborative research withKyoto City Fire DepartmentKyoto Prefectural School for the Visually Impaired
Haptic direction indicatorBuru-Navi+GPS+Electric compass
NTT Communication Science Laboratories 20/39
IEEE VR 2008 Tutorial #2 © 2008 [email protected]
Example of Applications (2): Entertainment
Force feedback “tray” for novice waiters
Buru-NaviRotation mechanism
Position and posture identification system
Video processing
NTT Communication Science Laboratories 21/39
IEEE VR 2008 Tutorial #2 © 2008 [email protected]
Tray with Force Display
Pochette
stepper motor a belt with the belt pulley
Buru-Navi
Tray
NTT Communication Science Laboratories 22/39
IEEE VR 2008 Tutorial #2 © 2008 [email protected]
Singapore Science Center (2008)
T. Amemiya, T. Maeda, H. Ando, "Location-free Haptic Interaction for Large-Area Social Applications", Personal and Ubiquitous Computing, 2008.
Laval, France (2007)
T. Maeda, H. Ando, T. Amemiya, N. Nagaya, M. Sugimoto, M. Inami, "Shaking The World: Galvanic Vestibular Stimulation As A Novel Sensation Interface", In Proc. of ACM SIGGRAPH 2005 Emerging Technologies, 2005.
Human Interface Using GVS
NTT Communication Science Laboratories 24/39
IEEE VR 2008 Tutorial #2 © 2008 [email protected]
Illusion in Balance Sensation
Motion Platforms
Balance can be also controlled by thermal or galvanic vestibular stimulations.
Vestibular
semicircular canal
Otolith
auditory ossicle
eustachian tubeeardrumexternalauditorymeatus
cochlea
vestibular nerve
cochlear nerve
???Mobile/Wearable
NTT Communication Science Laboratories 25/39
IEEE VR 2008 Tutorial #2 © 2008 [email protected]
GVS (Galvanic Vestibular Stimulation)
If it is strong enough, it not only throws you off balance but alters the course of your movement.
When a weak DC current is delivered to the mastoid behind your ear, your body responds by shifting your balance toward the anode.
NTT Communication Science Laboratories 26/39
IEEE VR 2008 Tutorial #2 © 2008 [email protected]
Radio-controlled Humans
T. Maeda, H. Ando, T. Amemiya, N. Nagaya, M. Sugimoto, M. Inami, "Shaking The World: Galvanic Vestibular Stimulation As A Novel Sensation Interface", In Proc. of ACM SIGGRAPH 2005 Emerging Technologies, 2005.
Over 3,000 people have experienced this! (no accidents so far..)
NTT Communication Science Laboratories 27/39
IEEE VR 2008 Tutorial #2 © 2008 [email protected]
Remote-controlled Riders
H. Ando, T. Amemiya, J. Watanabe, M. Inami, T. Maeda, "The Evaluation of Nail-Mounted Tactile Display", In Proc. of World Haptics Conference 2005, 2005.(poster presentation)
Nail-mounted Tactile Display
NTT Communication Science Laboratories 29/39
IEEE VR 2008 Tutorial #2 © 2008 [email protected]
Conventional tactile displays
Tactile display
Human
ground
Nail-mounted tactile display
Vibration actuator
Force sensor
B/W sensor
+
+
Voice-coil motor
Finger-color sensor
Reflection sensor
Human
Tactile display
ground???
NTT Communication Science Laboratories 30/39
IEEE VR 2008 Tutorial #2 © 2008 [email protected]
Vibration during Active Touch
NTT Communication Science Laboratories 31/39
IEEE VR 2008 Tutorial #2 © 2008 [email protected]
J. Watanabe, S. Hayashi, H. Kajimoto, S. Tachi, S. Nishida, Tactile motion aftereffects produced by appropriate presentation for mechanoreceptors, Experimental Brain Research 180 (3), pp. 577-582, 2007.
Tactile Motion Aftereffect
NTT Communication Science Laboratories 33/39
IEEE VR 2008 Tutorial #2 © 2008 [email protected]
Adaptation (moving stimulus) Test stimulus
Vision
Illusory motion
Tactile
Motion Aftereffect
Lerner (2002)Hollins (1994), Lerner (2002)
Previous Studies:Clear MAEs like in vision were NOT observed.
???
NTT Communication Science Laboratories 34/39
IEEE VR 2008 Tutorial #2 © 2008 [email protected]
Adaptation
Interval
Test Stimulus
Judgment
(400-ms stimulus+600-ms rest)x10
2 sec
One of nine ISOIs
“Upward” or “Downward” ?
Four subjectsThree adaptation conditions (No、Upward、Downward)
ISOI=100 ms
Experimental Procedure
30 Hz 200ms
Time
ISOI
ISOI
NTT Communication Science Laboratories 35/39
IEEE VR 2008 Tutorial #2 © 2008 [email protected]
△ No Adaptation× Upward Adaptation○ Downward Adaptation
Experimental Result (one subject)
NTT Communication Science Laboratories 36/39
IEEE VR 2008 Tutorial #2 © 2008 [email protected]
Mechanoreceptors under the skinMeissnerRA(Rapid Adaptation)TransientRespond to vibration 30 Hz
Skin surface
Epi-dermis
Dermis
Endodermis
Merkel SA(Slow Adaptation)SustainedRespond to static distortion
this reportAdapt: RA →test : RA=reproductive MAE
earlier studiesAdapt: RA→test :SA=No MAE
Hollins (1994)Lerner (2002)
Lerner (2002)
What are the differences between earlier studies and our experiments?
NTT Communication Science Laboratories 37/39
IEEE VR 2008 Tutorial #2 © 2008 [email protected]
SummarySensory illusions can be tools for designing human interfaces.
Kinesthetic illusionsVestibular illusionsTactile illusions
In overcoming certain difficulties, sensory illusions are very effective.
Virtuous circle between human science and engineering is essential for human interfaces using sensory illusions.
NTT Communication Science Laboratories 38/39
IEEE VR 2008 Tutorial #2 © 2008 [email protected]
Members & AcknowledgementTomohiro Amemiya, Ph.D.
(NTT CS Labs.)Taro Maeda, Ph.D.
(Osaka Univ.)
Junji Watanabe, Ph.D.(PREST, JST)
Hideyuki Ando, Ph.D.(NTT CS Labs.)
NTT Communication Science Laboratories 39/39
IEEE VR 2008 Tutorial #2 © 2008 [email protected]
ReferenceT. Amemiya, H. Ando, T. Maeda, ""Lead-Me Interface" for a Pulling Sensation from Hand-held Devices", ACM Transactions on Applied Perception, Vol. 5, No. 4, 2008.T. Amemiya, T. Maeda, H. Ando, "Location-free Haptic Interaction for Large-Area Social Applications", Personal and Ubiquitous Computing, Springer, 2008.T. Amemiya, I. Kawabuchi, H. Ando, T. Maeda, "Double-Layer Slider-Crank Mechanism to Generate Pulling or Pushing Sensation without an External Ground", Proc. of IEEE/RSJ International Conference on Intelligent Robotsand Systems (IROS 2007), pp. 2101-2106, 2007.H. Ando, T. Amemiya, J. Watanabe, M. Inami, T. Maeda, "The Evaluation of Nail-Mounted Tactile Display", In Proc. of World Haptics Conference 2005, 2005.Maeda, H. Ando, T. Amemiya, N. Nagaya, M. Sugimoto, M. Inami, "Shaking The World: Galvanic Vestibular Stimulation As A Novel Sensation Interface", Proc. of ACM SIGGRAPH 2005 Emerging Technologies, 2005.J. Watanabe, S. Hayashi, H. Kajimoto, S. Tachi, S. Nishida, Tactile motion aftereffects produced by appropriate presentation for mechanoreceptors, Experimental Brain Research 180 (3), pp. 577-582, 2007.
Top Related