Physical I/O devices Part 2: haptic output and other non-visual … · 2012-10-21 · • Haptic...
Transcript of Physical I/O devices Part 2: haptic output and other non-visual … · 2012-10-21 · • Haptic...
Department of Signal Processing
Physical I/O devicesPart 2: haptic output and other
non-visual displaysSGN-5406 Virtual Reality 2012
Atanas Boev
based on material byStanislav Stankovic and Ismo Rakkolainen
1
SGN-5406 Virtual Reality 2012
Department of Signal Processing
Outline2
SGN-5406 Virtual Reality 2012
Physical I/O devices
Part 1: Input (haptic sensors)
Passive Active
Nofeedback
Movementsupport
Intentionalfeedback Tactile Kinesthetic
VestibularHaptic
Olfactory
Part 2: Output (non-visual displays)
End-effector
Forcefeedback
3D “fly” mouseWiimotePS MoveKinectVR Gloves3D ProbesAccelerometersEtc…
KeyboardMouseJoystickTouch screensSteering wheel3D “desk” mouse(SpaceNavigator)Etc…
Combination ofinput+output
Surgery simulatorNovint FalconCyberGrasp glove
Virtual keyboard +force feedback
Department of Signal Processing
DISPLAYS IN GENERALOutput
3
SGN-5406 Virtual Reality 2012
Department of Signal Processing
Output devices
• Present the artificial computer generated stimuli to the user.
• Provide the information about the state of VR environmentto the user.
• Provide feedback about the results of user’s actions.
• Work with human sensory organs:• Visual system• Auditory apparatus• Haptic, (tactile, kinesthetic)• Rarely with other senses – olfactory (smell), taste, etc.
Department of Signal Processing
Displays
Display = Output Device
Video display:• Computer Screen• Projector• HMD – Head Mounted Display• 3D Screens
Audio display:• Speakers• Headphones
Haptic display:• Tactile displays• Force Feedback Devices
Department of Signal Processing
HAPTIC DISPLAYSDisplays
6
SGN-5406 Virtual Reality 2012
Department of Signal Processing
Skin senses (reminder)
• Tactile – sense of touch• Kinesthesis – sense of joint position and effort
• E.g. carrying heavy load affects both tactile (pressureon hands) and kinesthetic (muscle effort) senses.
• The tip of the finger• The most sensitive• Even 20 nm movement
• Termoception – sense of hot and cold (15-45°C, 0.001°/s)
SGN-5406 Virtual Reality 2012
Department of Signal Processing
Types of haptic displays
• Tactile - provides artificial stimuli (touch,vibration) to receptors in our skin
• Kinesthetic – provides artificial sense offorce• End-effector – limits the natural movement, thus
providing feeling of solid objects / passive resistance• Force feedback – applies force, gives sense of active
movement / impact
SGN-5406 Virtual Reality 2012
Department of Signal Processing
Multimodality
• Haptic output needs to be correlated with video andaudio stimuli.
• Vibrations of gamepad paired with events in thegame (character getting hit).
• Vibration of a mobile phone while phone is ringing.
• Vibration of the device when a virtual key is pressedon the touch screen.
Department of Signal Processing
Haptic display controller
SGN-5406 Virtual Reality 2012
Department of Signal Processing
TACTILEOutput devices
11
SGN-5406 Virtual Reality 2012
Department of Signal Processing
Actuators
• Actuators, devices by which artificial stimuli ispresented to human skin.
• Actuators present some force to a region skin.
• Types of actuators:• Blader actuators – pneumatic, hydraulic• Vibrator actuators – electromechanical• Pin actuators – electromechanical• Piezoelectric devices.• Electro-active polymers.• Etc.
Department of Signal Processing
Bladder Actuators
• Pockets that can be expanded and contracted:• By controlling the flow of air (pneumatic)• By controlling the flow of liquid (hydraulic)
• Strategic placement of the pockets creates thesensation of pressure on different areas of theparticipant’s hand and body
Department of Signal Processing
Vibrator Actuator
• Most often found in mass market products:• Game controllers• Mobile phones• Portable consoles• Data Gloves• Simulator seats
• Easier to control then other types of devices.• Very robust and easy to implement.• Can’t convey the sense of surface texture or shape of the object.• Offer a limited range of effects.
• Low-frequency speakers (subwoofers) can also be used as a vibratorydisplay.
Department of Signal Processing
Case study - Haptic Compass
• Belt with 12 vibration devices.• At any given moment the device facing north vibrates.• Augmentation of human senses. Humans do not have an explicit compass
sense. This device gives acute awareness of sense of direction.
• See• Udo Wächter, University of Osnabrück• http://feelspace.cogsci.uni-osnabrueck.de/
Department of Signal Processing
Pin Actuators
• Small pin arrays placed on each finger.
• Height of pins controlled electronically.
• Textures are detected by pressure variationsacross the fingertip over time.
• Good for displays for blind people
Department of Signal Processing
Case study - interactive terrain
Northrop Grumman’s TerrainTable.
Array of 4600 pins push up touch-sensitive silicone screen.
Overhead projector projects 2D map.
Military applications.
Department of Signal Processing
Temperature Actuators
• Can very rapidly present temperature fluctuations,typically to finger tips.
• Danger of tissue damage.• Limits must be adjusted to the safe range of temperatures tolerated by
human skin.
• Not the ”real” temperature of the simulated objects.
• Use scenarios rare, not many practical implementations.
Department of Signal Processing
Senseg E-Sense
• Tactile feedback on touch screens.
• Screen is covered by a grid ofelectrically activated elements.
• Tixel a tactile pixel.• Tixels generate a controlled electric field which
extends several millimeters above the surface.• Exploits electro-sensory phenomenon.
See: http://senseg.com/technology/system-architecture
Department of Signal Processing
Senseg E-Sense
• Ultra-low electrical current ispassed into the insulatedelectrode – the tixel.
• Electrical charge creates a smallattractive Coulomb force to fingerskin.
• By modulating this attractiveforce, any number of touchsensations can be generatedfrom vibrations, clicks, texturedsurfaces, etc.
Department of Signal Processing
Aurborne tactile display
• SIGGRAPH 2008 EmergingTechnologies: Airborne UltrasoundTactile Display• Mid-air tactile sensations by means of
airstreams generated by ultrasonicactuators
• SIGGRAPH 2009 Emerging Tech:Touchable Holography• See: http://www.youtube.com/watch?v=Y-
P1zZAcPuw
SGN-5406 Virtual Reality 2012
Department of Signal Processing
KINESTHESICOutput devices
22
SGN-5406 Virtual Reality 2012
Department of Signal Processing
End-effector Devices
• Force Feedback – End-effector Devices.
• End-effector devices are a special class of forcefeedback devices.
• Users limbs, hands, arms, legs are in contact withmachinery.
• Input and output device.• Movements on hands serve as input.• Device provides feedback through the active force.
• Generally linked to mechanical tracking sensors.
Department of Signal Processing
Telesurgery devices
• Stanford Research Institute,SRI International.
• M7 Surgical Robot.
• Developed for NASA.
• Can perform operations inZero gravity.
• Compensates for unwantedmovements in Zero-G.
Department of Signal Processing
End-effector Devices
• UNC Nanomanipulator.
• Microscop image on screen.
• Mechanical arm with 6DoF.
• Forcefeed back.
• See:http://cismm.cs.unc.edu/tag/nanomanipulator/
Department of Signal Processing
Force Feedback Data Glove
CyberForce system by CyberGloveSystems.
Mechanical system that exerts forceon hand and arm.
Sense of weight and inertia whilepicking up a "heavy" virtual object
Feel resistance of a simulatedobjects.
http://www.cyberglovesystems.com/products/cyberforce/overview
Department of Signal Processing
Components of End-effectorDisplays
Mechanical trackers.
Force generating device.
Technologies:• Electronic motors – one for each DoF,• Hydraulics,• Pneumatics
Department of Signal Processing
Features of End-effector Devices
• Often operates also as an input device, potentially providingresistance to input controls.
• Mechanical movement sensors are generally incorporated directlyinto the system. Mechanical tracking is generally very fast andaccurate.
• Typically operate with respect to single point in the virtual world.
• Number of DOFs: 1-6.
• Tactile displays e.g. vibrator actuators can be mounted within theend-effector.
• Can also be constructed to enhance the user’s force.
Department of Signal Processing
Full Body Devices
End-effector devices which affect user’swhole body.
Two mayor types:• Exoskeletons
(wearable robotics)• Surround platforms
Department of Signal Processing
Exoskeletons
• Exoskeletons – Meaning external skeleton.
• Can work with whole body or just with some parts.
• Exoskeletons are not just I/O device.• Applications:
• Military:• Enhance the power of user’s body or limbs. Increase the load a person can
carry or prologue the time a person can endure the load.• Medical:
• Restore mobility in paralyzed limbs.• Telepresence:
• Operate machinery at a distance, in environments dangerous for humans.
Department of Signal Processing
Exoskeleton Examples
• Novint XIO game controller.
• Partial exoskeleton.
• Arm exoskeleton.
• Mass market product.
Department of Signal Processing
Exoskeleton Examples
• Hybrid Assistive Limb.
• Cyberdine Inc. And Tsukuba Uni.Japan.
• HAL detects bioelectrical impulsessent from brain to limbs, using a set ofelectrodes on the skin.
• Possible applications:• Rehabilitation,• Physical training,• Help for disabled people,• Rescue services,• Entertainment (as input device)
Department of Signal Processing
VESTIBULAROutput devices
33
SGN-5406 Virtual Reality 2012
Department of Signal Processing
Vestibular Displays
• Work with human vestibular apparatus (sense of equilibrium).
• Work by physically moving the user.
• Artificial sense of equilibrium, acceleration and orientation that user isexpected to feel during a real motion.
• Strong relationship between the vestibular and visual systems of humans.
• Prevents (or causes) simulator sickness.
• Makes better immersion.
• Typically used in specialized simulators (flight simulators).
Department of Signal Processing
Vestibular displays
• D-BOX Motion Code movie seat.
• Moving seat platform – 4 to 6DoFdepending on model.
• D-BOX Motion controller device.
• Movie theaters and amusement parks.
• http://www.d-box.com
Department of Signal Processing
Galvanic Vestibular Stimulation
• Vestibular apparatus works on the sameprinciple as human auditory system.
• Works on the same principle as Cochlearimplant.
• Sending specific electric messages to anerve in the ear that transmits balanceinformation.
• Can alter one’s perception of balance.• See also: lecture VR4.3
• Side effect are still not known.
http://www.youtube.com/watch?v=ZlNfBrXYYTc
Department of Signal Processing
Locomotion Platforms
• User needs to remain within certain limited physical space.• User needs to have an impression of physical movement in
virtual space.• Simulate the physical movement of the user in VR environment,
while keeping user in the same physical position.
• Examples:• Omni-directional treadmills.• CirculaFloor.
Department of Signal Processing
Omni-directional treadmill
• Two treadmills placed on top ofeach other.
• Movement along X and Y axes.
• Platform provides the sensationof moving ground.
• Suspension system to hold theuser in place.
Department of Signal Processing
CirculaFloor
• Several movable tiles.
• With each step a newsegment moves infront ofuser’s feet.
• Movement of the segmentcounters the movement ofuser while maintaining theillusion of motion.
See: http://youtu.be/rYsvB2y2Ero
Department of Signal Processing
Haptic Rendering
• Rendering – process of generating of artificialstimuli, based on the current state of VRenvironment.
• Needs to be fast, up to 1000 Hz.
1. 3D objects & properties loaded from the database.2. Collision detection. Only colliding objects are
passed on.3. Compute collision forces, smoothing, mapping.4. Haptic texturing (vibrations, temperature, etc.).5. Present output to the user through haptic display.
Department of Signal Processing
OLFACTORYOutput devices
41
SGN-5406 Virtual Reality 2012
Department of Signal Processing
Olfactory displays
SGN-5406 Virtual Reality 2012
Digital signals in software code trigger the generator to emitprecise amounts of the appropriate aroma.
Department of Signal Processing
Olfactory displays
• Artificial smell simulation• Mostly at research, prototype phase, some niche products
• Film: Scent of Mystery (1960), with accompanying smells• Sensorama 1960, Smellizer 1984
• Enhanced cosmetics, perfume, food advertising, aromatherapy
• Aromas can be used to enhance the experience and trigger fear,excitement and other emotions
• Multi-sensory magazines (Esquire cover, smell etc.)
• Companies• http://www.structuralgraphics.com/sensory-effects/• http://www.aerome.com/• http://www.aromajet.com/• http://www.digiscents.com/• http://www.trisenx.com/intro.html
SGN-5406 Virtual Reality 2012
Department of Signal Processing
Case study – ScentScape
• ScentScape™ specifications:• Provides 20 basic scents per
cartridge• Scent cartridges last 200 hours or
more in heavy use, depending onpersonal settings
• ScentScape™ scent cartridges willbe available in standard andmedia-specific versions
• Separate "volume control" to adjustoverall smell strength for personalpreference
SGN-5406 Virtual Reality 2012
See: http://www.scentsciences.com/products/scent_scape.html