PC-based Telerehabilitation System with Force Feedback
Transcript of PC-based Telerehabilitation System with Force Feedback
Input Devices
Universidade de Aveiro Departamento de Electrónica, Telecomunicações e Informática
Virtual and Augmented Reality 2017/18 Beatriz Sousa Santos
What is Virtual Reality?
“A high-end user interface that involves real-time simulation and interaction through multiple sensorial channels.” (vision, sound, touch, smell, taste) (Burdea and Coiffet., 2003)
2
Input Devices: Trackers, Navigation and gestures interfaces
3
Input devices • Trackers:
– Magnetic (AC, DC) – Optical – Ultrasonic – Inertial, – Mechanical – Hybrid ...
• Navigation and manipulation interfaces:
– Tracker-based – Trackballs – 3D mice, ...
• Gesture interfaces:
– Gloves – Depth cameras ...
4
Virtual objects have 6 degrees of freedom (D.O.Fs):
-three translations;
-three rotations.
- Roll – rotation around the longitudinal axis (Burdea and Coiffet., 2003)
Trackers measure the motion of “objects” (e.g. user head) in a fixed system of coordinates.
Tracker is a special purpose H/W to measure the real-time change in a 3D object position and orientation
5
Example: 3D magnetic sensor in a HMD
Without the head tracker - the image - the sound cannot change to match the head posture Required tracking accuracy: Image > sound
(Burdea and Coiffet., 2003)
6
Body Tracking:
• Head
• Hand and fingers
• Torso
• Feet
• A group of people, ...
Indirect tracking: Using physical objects (props and platforms)
Technologies:
• Electromagnetic
• Optical
• Ultrasonic
• Inertial
• Mechanical
• Hybrid ...
What is usually tracked? How?
7
Tracker characteristics:
• Measurement rate – Readings/sec
• Sensing latency
• Sensor noise and drift
• Measurement accuracy (errors)
• Measurement repeatability
• Tethered or wireless
• Work envelope
• Sensing degradation
8
Tracker performance parameters:
• Accuracy
• Jitter
• Drift
• Latency
• Tracker update rate
9
Tracker performance parameters should be analyzed to match a solution for sensorial channel and budget of an application
Tracker characteristics
Real object position
Accuracy
Tracker position
measurements
10
(Burdea and Coiffet., 2003)
Accuracy: Difference between the object’s actual 3D position and that reported by the measurement
Tracker characteristics
Real object position
Accuracy
Resolution
Tracker position
measurements
11
(Burdea and Coiffet., 2003)
Resolution:
“the smallest amount of the quantity being measured that the instrument will detect.”
(used by Ascension)
(Polhemus uses a different definition )
Real object fixed
position
Signal noise
Time
Tracker data
12
(Burdea and Coiffet., 2003)
Jitter: Change in tracker output when the tracked object is stationary
Real object fixed
position
Sensor drift
Time
Tracker data
13
Drift: Steady increase in tracker error with time
(Burdea and Coiffet., 2003)
Real object
position
Sensor latency
Time
Tracker data
14
Latency: Time delay between action and result: time between the change in object position/orientation and the time the sensor detects this change
(Burdea and Coiffet., 2003)
15 (Burdea and Coiffet., 2003)
Tracker update rate: Number of measurements that the tracker reports every second If the same tracker electronics is used to measure several objects, the sampling rate suffers due to multiplexing
Dedicated electronics to each tracked object
Same electronics to each tracked object
16
Most used trackers:
• Magnetic
• Ultrasonic
• Optical
• Inertial
• …
Magnetic Trackers A magnetic tracker is a non-contact position measurement device that uses a magnetic field produced by a stationary TRANSMITTER to determine the real-time position of a moving RECEIVER element may be AC DC
17
Magnetic Trackers
• Use low-frequency magnetic fields to measure position
• Fields are produced by a fixed source
• Size of source grows with the tracker work envelope
• The receiver is attached to the tracked object and has three
perpendicular antennas • Distance is inferred from the voltages induced in the antennas – needs
calibration…
18
19
(Burdea and Coiffet., 2003)
Magnetic tracker with the old Data Glove
Magnetic tracker accuracy degradation
23
(Burdea and Coiffet., 2003)
Comparison of AC and DC magnetic trackers
• DC trackers are immune to non-ferromagnetic metals
(brass, aluminum and stainless steel)
• Both DC and AC trackers are affected by the presence of ferromagnetic metals
(mild steel and ferrite)
• Both are affected by copper
• AC trackers have better resolution and accuracy
• AC trackers have slightly shorter range
24
A “standard” for motion tracking for years:
25
Polhemus
APPLICATIONS: High Accuracy Head Tracking EEG Localization Training and Simulation Eye Tracking Neuroscience Biomechanics
(proprietary AC electromagnetic technology)
Widely used and expensive:
26
Ascension
Tracking, guiding, and localizing medical instruments within a patient's body
3D measurement and analysis of human movement for biomechanical purposes: sports, performance, and design
Track head and objects for matching computer-generated imagery with head direction; weapon aiming and interactive instruction
Tracking of head, hands and 3D pointers for interaction with large scale and immersive displays
Example of a magnetic tracker
28
Aurora Adds customizable real-time spatial measurement technology to simulation systems to deliver realistic surgical navigation or weapons targeting simulation in all six degrees of freedom (6DOF).
29
Example of “Low cost” magnetic tracker:
30
• True six degree-of-freedom magnetic motion tracking
• Precise sensor for 1mm and 1 degree tracking
• No line of sight to controllers required
• Low-power magnetic field
• low power consumption
• Low latency feedback
Razer Hydra
31
Razer Hydra Technical Specifications - Per controller Thumb-ergonomic analog stick for fluid control - 4 Hyperesponse action buttons - Rapid-fire trigger and bumper for faster in-game response - Non-slip satin grip surface - True six degree-of-freedom magnetic motion tracking - Lightweight, anti-tangle braided cable Base station - Low-power magnetic field, low power consumption - Ultra precise sensor for 1mm and 1 degree tracking - No line of sight to controllers required - Low latency feedback - Approximate Size : 120 mm / 4.92” (Length) x 120 mm / 4.92” (Width) x 100 mm / 3.94” (Height) - Approximate Weight : 800 g / 1.76 lbs
Ultrasonic Trackers
A non-contact position measurement device that uses an ultrasonic signal produced by a stationary transmitter to determine the real-time position/ orientation of a moving receiver.
32
(Burdea and Coiffet., 2003)
Ultrasonic Trackers
• Use low-frequency ultrasound to measure position
• Ultrasound produced by a fixed triangular source (speakers)
• Number of sources grows with the tracker work envelope
• The receiver is triangular and attached to the tracked object and has
three microphones
• Distance is inferred from the sound time of flight
• Sensitive to air temperature and other noise sources
• Requires “direct line of sight”
• Slower than magnetic trackers (max 50 updates/sec)
33
Ultrasonic tracker (Logitech)
(Burdea and Coiffet., 2003)
http://www.vrdepot.com/vrteclg.htm
6 DOFs Update rate - 50 / sec Latency – 30 ms
34
Optical Trackers A non-contact position measurement device that uses optical sensing to determine the real-time position/ orientation of an object
35
(Burdea and Coiffet., 2003)
outside-looking-in inside-looking-out
Outside-looking-in
LaserBIRD optical tracker
36
Sub-degree and sub-millimeter accuracy Measurement rate of 240 meas/sec Tracking response: 7.17 ms
https://www.ascension-tech.com/products/
Outside-looking in Vicon MX https://www.vicon.com/
• Uses 4 Mpixel cameras with own 120 LED
array (infrared, or visible red). Accuracy 0.02 of a pixel
• Camera has real-time onboard image processing (masking and thresholding)
• Resolution 2352x1728 @ 160 fps
• 8 cameras are connected to a MX net unit which then communicates with the PC
• User wears reflective markers (small spheres)
(Burdea and Coiffet., 2003)
• The best accuracy is close to the work envelope • Very large tracking surface and
resistance to visual occlusions (line of sight)
38
Inside-out optical tracker advantages/ disadvantages
(Burdea and Coiffet., 2003)
39
HTC Vive “Lighthouses”
https://www.vive.com/eu/
Hybrid Ultrasonic/Inertial Trackers
• No interference from metallic objects
• No interference from magnetic fields
• Large-volume tracking
• “Source-less” orientation tracking
• Full-room tracking
40
But…
• Accelerometer errors a lead to decreased accuracy
• Errors grow geometrically in time!
• Gyroscope errors compound position errors
• Needs independent position estimation to reduce “drift”
41
44
https://www.virtalis.com/tracking-systems/
Hybrid Ultrasonic/Inertial Tracker
6 DOFs Utrasonic – position Inertial - orientation Sub-millimiter accuracy Head + hand units Used for: - Training - Assembly/ disassembly and design review - …
Mechanical Trackers A mechanical tracker consists of a serial or parallel kinematic structure composed of links interconnected by sensorized joints.
46
(Burdea and Coiffet., 2003)
Mechanical tracker - Push 1280 stereo display (Fakespace Inc)
Item is no longer available
Mechanical Trackers
Pros
• Use sensors imbedded in exoskeletons to measure position
• Have extremely low latencies
• Are immune to interference from magnetic fields and large metal objects
Cons
• Limit the user’s freedom of motion
• Can be heavy if worn on the body
47
Example of an exoskeleton (mechanical tracker)
48
http://www.youtube.com/watch?v=uJza6G-7tD4
Painting a virtual wall: example of a virtual rehabilitation task for a patient recovering from stroke or traumatic brain injury
49
Other devices can be used to track
(at low cost)
https://www.leapmotion.com/
https://developer.microsoft.com/en-us/windows/kinect
(Jerald., 2016)
Navigation and Gesture Input Devices
• Navigation interfaces allow relative position control of virtual objects
(including a virtual camera)
• Gesture interfaces allow dexterous control of virtual objects and interaction
through gesture recognition.
50
Navigation and manipulation Input Devices
• Trackballs
• 3D mice
• 3D probes
• Wiimote, …
• Perform relative position/velocity control of virtual objects
• Allow “fly-by” application by controlling a virtual camera
51
The Cubic Mouse (research, not commercial)
52
http://www.youtube.com/watch?v=1WuH7ezv_Gs http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.84.6127&rep=rep1&type=pdf
- Allows to intuitively specify 3D- coordinates
- The rods represent the X, Y, and Z axes of a given coordinate system
53
3D mouse Space controller
http://www.spacecontrol.us/3d-maeuse-spacecontroller-funktionsprinzip.html http://www.3dconnexion.com/products/what-is-a-3d-mouse.html http://www.youtube.com/3DMaus
• Sends relative coordinates
• Allows
- to move 3D objects in a more intuitive way than a 2D mouse
- allows rotation around each axis
• Price ~300 USD
3D-Spheric-Mouse
http://www.engadget.com/gallery/axsotic-3d-mouse-takes-a-spherical-stab-at-an-age-old-problem/#slide=616049
- Has no mechanical sensors that can generate unwanted behaviors
- The specially developed polymer
springbodies create a smoothly consistent workflow
Gesture Input Devices
• There are/ have been various sensing gloves such as:
- Fakespace Pinch Glove (switches)
- Immersion CyberGlove (stain gauges),
- Gloveone …
• Have larger work envelope than trackballs/3-D probes
• Most need calibration for user’s hand
55
5DT Data Glove
CyberGlove
Pinch Glove
Item is no longer available
Gloveone
http://www.cyberglovesystems.com/sdk-videos/2015/9/17/vhsdk-sample-app-video
~
The CyberGlove
http://www.cyberglovesystems.com/
57
Other devices can be used to detect gestures
https://www.leapmotion.com/
https://developer.microsoft.com/en-us/windows/kinect http://kinectvr.com/
58
Speech recognition is also an interesting possibility:
• Frees hands
• Allows multimodal input
• Specialized software
• Issues: recognition, ambient noise, training, false positives
An input device “providing an infinit VE”: a treadmill for VR
59 https://www.youtube.com/watch?v=oWIDqebGUqE
May have applications, behond gaming: promote physical exercise, train people, …
60
Virtusphere (“the VR hamster ball”)
https://www.youtube.com/watch?v=2e5Qvac3BB8
62
Conclusion
When choosing a device, consider:
• Cost
• Generality
• DOFs
• Ergonomics / human factors
• Typical scenarios of use
• Output devices
• Interaction techniques
Input + output CyberTouch Glove
63
http://www.cyberglovesystems.com/cybertouch/#photos
64
Input + output GloveOne
https://www.neurodigital.es/gloveone/
https://www.kickstarter.com/projects/gloveone/gloveone-feel-virtual-reality
Main bibliography
- Jerald, J., The VR Book: Human-Centered Design for Virtual Reality, ACM and Morgan & Claypool, 2016
- G. Burdea and P. Coiffet, Virtual Reality Technology, 2nd ed. Jonh Wiley and Sons, 2003
- Craig, A., Sherman, W., Will, J., Developing Virtual Reality Applications: Foundations of Effective Design, Morgan Kaufmann, 2009
- J. Vince, Introduction to Virtual Reality, Springer, 2004
65