Haptics Technologieselsaddik/abedweb/teaching/elg5124/pdf/Chapter... · Haptics and Virtual reality...
-
Upload
duonghuong -
Category
Documents
-
view
229 -
download
2
Transcript of Haptics Technologieselsaddik/abedweb/teaching/elg5124/pdf/Chapter... · Haptics and Virtual reality...
22/01/2013
1
Haptics Technologies:Bringing Touch to Multimedia
C2: Haptics Applications
Outline Haptic Evolution: from Psychophysics to Multimedia
Haptics for Medical Applications Surgical Simulations
Stroke-based Rehabilitation
Support of the Visually Impaired
Tele-robotics and Tele-operation Tele-Surgery
Military Applications
Media Haptic Broadcasting
E-Commerce
Video Games
Arts and Design
Mobile Haptics
Haptics and Virtual reality
Education and Learning
Haptics for Security
Closing Remarks
Haptic Evolution: from Psychophysics to Multimedia
Experimental psychology Beginning of 20th century
Understanding human touch perception and manipulation
Robotics era Seventies and eighties of last century
Create devices with a dexterity inspired by human abilities
Tele-operation era Late eighties
extension of sensing and manipulation to a remote location
Haptics Early nineties
Tactile and kinaesthetic interfaces and computer haptics
Multimedia Haptics
22/01/2013
2
Haptic Evolution: from Psychophysics to Multimedia
The application spectrum is quite vast, and its trend of expansion is anticipated to increase
Haptics for Medical Applications
Already a strong presence in the area of medical training Professionals guide students through a simulated
surgical process Example: Trainee feels the surgeon performing a precision cut on
a simulated human organ
Statistically, surgeons have higher success rates the more often they perform a procedure
Allows surgeons to practice a procedure with less fatigue and stress
If geographically separated, realistic haptic rendering is near impossible!
Realistic haptic models of human tissue and organs
Surgical Simulations
May be used to train surgeons at remote locations In the future, tele-haptic applications will be used to perform surgical
procedures on patients at remote locations. e.g., Arctic regions, the space station
Examples: Needle-based procedures use needles, catheters, guide-wires, and small
bore instruments for teaching straightforward procedures
Minimally invasive surgery (called laparoscopic surgery)
Robot-assisted surgery (da Vinci Surgical System)
Simulation environments
Suturing video 1
Suturing video 2
22/01/2013
3
Surgical Simulations
Virtual eye
Surgical tool
Phantom Haptic Device
Eye Surgery Training Simulation
X. Shen et al., “Haptic-Enabled Telementoring Surgery Simulation", IEEE Multimedia, 2008
Stroke-based Rehabilitation
Involves applying forces to the injured/disabled organ (such as the finger, arm, ankle) to help it recover its strength and range of motion
helps to measure performance and to tailor performance-based exercises for each patient
Enables home located rehab and remote monitoring
Stroke-based Rehabilitation Game-like tasks to address certain parameters of hand movement:
Based on Jebsen Test for Hand Function (JTHF)
Offers continuous evaluation of the patient’s improvement.
Example exercise: Arranging eight cubes according to a color pattern.
22/01/2013
4
Stroke-based Rehabilitation Based on Augmented Reality (AR) technology
Post-Stroke Interactive and Entertaining Rehabilitation with ReActive objects (SIERRA)
Reactive Objects
react with vibration to inform the subject of being interacting with virtual objects
Components: A real cup
A vibrotactile actuators (pager motors)
An actuator controller
a bluetooth module
a 9V battery
Features: Inexpensive
light
portable
The actuators are arranged to cover a whole hand when grabbing.
Stroke-based Rehabilitation
Rehab test 1
Rehab test 2
Rehab test 3
Rehab test 4
Rehab test 5
Rehab test 6 (foot Stroke)
22/01/2013
5
Support of the Visually Impaired
Haptics enhances the perception of blind or visually impaired people by converting visual or sound information into a haptic modality
Example applications: European Union GRAB project
the exploration of chart data,
a city map explorer, and
a simple adventure game
TACTICS system interpret complex scientific data
Chromo-Haptic Sensor-Tactor (CHST) To feel colors
VIbroTActiLe (VITAL) system Tactile feedback as a navigation tool for the visually impaired
Inner fabric to protect the skin
Support of the Visually Impaired
How it works? Touch by hand, cell phone, mouse
For hand Hand tracking using depth image
For touch-enabled cell phone
For mouseVirtual touch
pad
22/01/2013
6
Tele-robotics and Tele-operation
Collaborative Design Designers at various locations
collaborate on the same project using a HVE
Haptics offers the participants additional information regarding the project
Objects may be moved, grasped, assembled, disassembled
Requires position sampling and collision detection for N users
Geographically separated users –network performance is key
Tele-robotics and Tele-operation
To remotely handle dangerous or radioactive substances
Applications: underwater exploration,
assembly and manufacturing,
micro-assembly
Tele-Surgery
Training surgeon on a simulated human organ
Tele-surgery
Train surgeons at remote locations
Perform surgical procedures on patients at remote locations
Brain surgery
Reachin Display[Reachin Technologies]
22/01/2013
7
Military Applications
Battlefield conditions, such as the presence of artillery fire or smoke, might mask sound and vision modalities
Haptics become an efficient communication channel
Media
Started in 1959, when tactile stimulation was used to enhance the movie “The Tingler” by attaching vibrating devices to the theatre seats
Provide higher immersion in multimedia contents
two content scenarios: Authored content
Viewers are able to interact with and influence the presentation of pre-recorded content
Real-time content The acquisition and display of haptic content occurs in real time
Haptic Broadcasting
Broadcasting multimedia streams with haptic channel Examples:
Home shopping environment Viewers can touch the 3D model through a haptic device
A fiducial marker position and orientation are calculated
Haptic Video Annotate video stream with haptic stimuli
Users feel what they see and hear
Haptic YouTube video Annotated with tactile feedback using XML notation
Used a haptic enabled browser and an arm band device
22/01/2013
8
E-Commerce
Haptic virtual environments gives new meaning to the phrase “Try before you buy”
Consumers could experience aspects of the product not available using current E-Commerce technology Example: “How heavy is this new PDA?” “How soft is this sweater?” –
haptic interfaces can answer these questions
Browser plug-in or Java application offers haptic and graphical display of a 3d model of the object on a vendor website
Requires that accurate models of new products can be acquired quickly and cheaply
Video Games
Gaming experience comprises four aspects: physical, mental, social, and emotional.
Force feedback technology enhances the physical aspects of the gaming experience
Greater immersive experience to gamers
Haptic information is recorded and played-back in synchronization with audio-visual data
BUT: Multiplayer games require high-bandwidth, low-latency networks
Logitech MOMO racing wheel[Logitech]
Logitech Rumblepad[Logitech]
Logitech Force 3D joystick[Logitech]
Gaming
S. Andrews et al., “HaptiCast: A physically-based 3D game with haptic feedback”, in Proc. of Future Play 2006.
When applied to video games, it gives players a higher sense of immersion/realism.
22/01/2013
9
Arts and Design
Arts & Culture Due to a “hands-off” policy,
museums do not permit visitors to handle sculptures and artifacts
Haptic equipped kiosks could be used to permit visitors to feel and manipulate these objects virtually
Requires realistic models of museum artifacts to be acquired so that users may truly explore them Example: tracing the
contours of Michelangelo’s “David”
Digital 3D model of ‘David’[The Digital Michelangelo Project]
Mobile Haptics
Mobile devices utilize gesture to allow easier and more intuitive user interaction
Haptics would enhance such intuitivism
Tactile haptic feedback is becoming common in smart phones
Example: Samsung haptic mobile phone
Haptic SCH-W420
Mobile Haptics
The KissMe system A neck piece to display kisses
Mobile device connected to neck piece via Bluetooth
Users can send kiss messages to each other
22/01/2013
10
Mobile Haptics
The HugMe system 2.5D video camera
A haptic Jacket (mobile device)
A haptic device
Local person Remote person
Depth cameraForce feedback device
Haptics and Virtual reality
Integrating haptic interactions with existing virtual environments: Linden Lab's multiuser online virtual world
Second Life
Education and Learning
Learners learn better a sensorimotor skill task using haptic modality
Example: Haptic Multimedia learning tool: Teach handwriting of alphabets (Japanese, Arabic, etc)
Three modes of operation: Full guidance
Partial guidance
No guidance (testing)
InteraSense (too)
22/01/2013
11
Haptics for Security
Haptic Biometrics More “advanced” handwritten signature
Environment: Virtual Check Application.
Visual stimuli: Virtual pen and a virtual check.
Haptic stimuli: Force and frictional feedback
Haptics for Security
Graphical password with haptics Utilizes the physical attributes captured during human-computer
interaction (pressure and velocity)
Overcomes shoulder surfing attacks
Continuous verification and authentication
Closing Remarks
The incorporation of haptics into virtual environments is still in its infancy!
A wide range of human activities, such as communication, education, entertainment, commerce, and science, would forever change if we learned how to capture, manipulate, and create haptic sensory stimuli that are nearly indistinguishable from reality
Business models and frameworks are needed to make haptic devices practical, inexpensive, and widely accessible (as easy as mouse)
22/01/2013
12
متشکرم
谢谢!ありがとう!