MMVR Workshop presentation1...Computers and Advanced Technology in Education, September 2008, Crete,...
Transcript of MMVR Workshop presentation1...Computers and Advanced Technology in Education, September 2008, Crete,...
H3D API An open source API for dexterous skills
simulators
Daniel Evestedt
SenseGraphics
OutlineWhat is haptics?◦ Concepts◦ Hardware devices◦ Software
H3D◦ HAPI◦ H3DAPI◦ Toolkits
Case studies◦ Simodont ‐Moog dental simulator
◦ Cork University hospital ‐ spinal anesthesia simulator
SenseGraphics technology
SenseGraphics H3DAPI Software development platform (Open Source)◦ SenseGraphics H3DAPI is designed for users intending to write software applications in “multi‐sensory”environments.
SenseGraphics Immersive Workstations◦ Work with your data in Immersive environment.
What is haptics?
Tactile vs kineastetic force feedback
Exploratory procedures
Lateral motion (texture)
Pressure (hardness)
Static contact (temperature)
Unsupported holding (weight)
Enclosure (global shape)
Contour following (local shape)
What is haptics?
3‐DOF ‐ 6‐DOF
Haptic devices Display systems
H3D platform
H3DAPI◦ X3D‐based scene‐graph API◦ Graphics, haptics, sound◦ Scripting through Python◦ C++, OpenGL
HAPI◦ Haptics SDK◦ C++
◦ Cross‐platform (Windows, Linux, OSX)◦ Device independent
www.h3d.orgw.h3d.o
HAPI
C++
Haptics API
Cross‐platform
Device independent
Modular and easily extendable
HAPI ‐ components
Force effects
Shapes andsurfaces Device handling
11
Haptics renderers
HAPI – program flow
Syncronisation point
Get device state
Render shapes and effects
Send force, torque to device
Shapes, effects
Contact info,Device info
Main thread
HAPI thread (1000Hz)
Code example (HAPI)int main(int argc, char* argv[]) {
AnyHapticsDevice hd;
HAPISurfaceObject *my_surface = new FrictionSurface();HapticPrimitive *my_haptic_sphere =
new HapticPrimitive( new Collision::Sphere( Vec3( 0, 0, 0 ),0.05 ),
my_surface );
hd.setHapticsRenderer( new GodObjectRenderer() );if( hd.initDevice() != HAPIHapticsDevice::SUCCESS ) {
cerr << hd.getLastErrorMsg() << endl;return 0;
}hd.enableDevice();hd.addShape( my_haptic_sphere );hd.transferObjects();
string temp_string;cerr << "Press ENTER to exit" << endl;getline( cin, temp_string );
hd.disableDevice();hd.releaseDevice();
}
H3D API
H3DAPI
H3DAPI Overview
X3D Python
HAPI
15
C++Scene Graph nodes
Field network
OpenGL
H3D API
X3D – scene definition
Python – scripting, scene updates, interaction
C++ ‐ nodes, scene building blocks
Simple X3D example (H3DAPI)
<Shape><Appearance><Material diffuseColor=“1 0 0”/><FrictionalSurface stiffness=“0.3” />
</Appearance><Sphere radius=“0.1” />
</Shape>
H3D API Components
3D graphics Shaders
Particle system
Haptics
AnimationScripting
Sound
More info
www.H3D.org
Wiki
Training
Consultancy
www.sensegraphics.org
Documentation
www.web3d.org
Toolkits
MedX3D (volume rendering)
VHTK (volume haptics toolkit)
UI (3D user interface)
RigidBodyPhysics(under development)
MedX3D
Web3D Consortium – Medical working group◦ Create ISO standard for representation of human anatomy
◦ First step: volume visualization
MedX3D toolkit for H3D API◦ First implementation of standard draft
◦ GPU based raycasting and slice based 3D texture rendering
Render styles
ComposedVolumeStyle
Compose several styles into one
Opacity-Phong-Edge enhancementOpacity-Phong
SegmentedVolumeData
Render different parts of data with different rendering styles.
Blood vessels: PhongHead: Opacity-Boundary
Cerebral ventricular system: ToneMappedHead: Opacity
VHTK – Volume Haptics Toolkit
Surface Haptics◦ interact with surfaces
Volume Haptics◦ haptic algorithms using volumetric data
◦ interaction with volumetric data
Applications◦ surgery simulation on authentic data
◦ scientific visualization of volumetric data
Dichloroethane Electro Potential
Femur Bone Drilling Dynamics in Volume Haptics
RigidBodyPhysicsRigid bodies◦ Physics properties (mass, friction, etc)
◦ Geometric shape
◦ Joints
◦ Haptic interaction
◦ Currently ODE and Nvidia PhysX
UI toolkit
3D/haptic UI nodes
Menus
Sliders
Buttons
Case study 1
Simodont ‐ dental simulator
32
1/8/2009
Moog
HapticMASTER
Conventional Dental Training Equipment
33
1/8/2009 34
SIMODONT
Visual Display
Dental Mirror
Haptic Burr
Courseware
Audio
Foot PedalFoot pedal
35
Visual DisplaySharp, true size collocated visual display
Full resolution, full stereo image projection and mirror technology
361/8/2009
Dental tools
Simulator and Courseware Interaction
37
1/8/2009
39
1/8/2009
Courseware
Case study 2
Spinal anesthesia simulator
Cork University Hospital and Interaction Design Centre, 2007
Spinal anesthesia – Can we improve how it is taught?
Cork University Hospital and Interaction Design Centre, 2007
Determinant study – to understand the problem and the domainLearning related:
◦ Recognition of certain characteristic sensations as spinal anesthesia is performed
◦ Ability to visualise the relevant anatomy
Teaching related:◦ An explicit knowledge program for the procedure
◦ A case‐based learning program
◦ A valid, reliable competence assessment procedure
38
1/7/2009
Courseware
Cork University Hospital and Interaction Design Centre, 2007
Modeling of sensations
Medical experts were recruited to rate sensations associated with spinal anaesthesia
Cork University Hospital and Interaction Design Centre, 2007
Using the initial model
The haptic arm controls the needlesEach layer of tissue have different resistanceThe anatomy can be rotated for enhanced learning
Cork University Hospital and Interaction Design Centre, 2007
Human tissue modeling
A human tisssue model was developed to reproduce the actual sensations of performing spinal anaesthesiaThe model has been verified with medical experts and is believed to be very accurate
Cork University Hospital and Interaction Design Centre, 2007
Visualisations
Real‐time 2D and 3D visualisations are usedThe learner can place the needles and then view how the needle was placed and from that create a mental model of the procedure
Cork University Hospital and Interaction Design Centre, 2007
Simulator‐supported assessment procedure
Thanks for listening!
See us in the exhibition hall for questions and demos
Referenceswww.h3d.org (developer community website, API downloads)
www.sensegraphics.com (the company behind H3D API)
www.web3d.org (X3D specification)
http://www.xj3d.org/extensions/volume.html (MedX3D Volume Rendering X3D component)
http://www.web3d.org/x3d/workgroups/medical/ (X3D Medical working group)
http://www.sensegraphics.com/index.php?option=com_content&task=view&id=24&Itemid=36(list of projects based on H3D API)
http://www.moog.com/simulationandtesting/ (Moog, the company behind the dental simulator)
http://www.medcap.eu/ (Spinal Anaesthesia simulator project page)
http://webstaff.itn.liu.se/~karlu/ (VHTK publications)
www.volvis.org (sample medical datasets)
http://dev.forsslundsystems.se (open source dental surgery simulator)
ReferencesCork Spinal Anaesthesia Simulator publicationsLövquist, E, Kulcsár, Z., Fernström, M., Aboulafia, A., Shorten. G., (2008) The Design of a Haptic Simulator for Teaching and Assessing Spinal Anaesthesia. In proceedings of the 5th Intuition Conference, 6‐8 October, Turin, Italy
Lövquist, E., Aboulafia, A., Breen, D., Shorten, G., Zhang, D., Albert, D., (2008) Designing a simulation‐supported adaptive assessment system for spinal anaesthesia. Proceedings of the 11th International Conference on Computers and Advanced Technology in Education, September 2008, Crete, Greece.
Exploratory proceduresSusan J. Lederman and Roberta L. Klatzky. Hand movements: A window into haptic object recognition. Cognitive Psychology, 19(3):343‐368, July 1987,
VHTK (Volume Haptics Toolkit)Karljohan Lundin Palmerius, "Direct Volume Haptics for Visualization", Ph.D. Thesis, May 2007