Haptic Virtual Reality Training Simulator for Sacral...
Transcript of Haptic Virtual Reality Training Simulator for Sacral...
![Page 1: Haptic Virtual Reality Training Simulator for Sacral ...nearlab.polimi.it/wp-content/uploads/2019/01/Presentazione_Leonar… · Virtual reality and Haptic simulator: Virtual reality:](https://reader034.fdocuments.net/reader034/viewer/2022050503/5f94e3e67502815a89201a23/html5/thumbnails/1.jpg)
Titolo presentazione
sottotitolo
Milano, XX mese 20XX
Haptic Virtual Reality Training Simulator for Sacral Neuromodulation Surgery:
a Feasibility Study
Advisor: Prof.ssa Elena De Momi
Co-Advisor: Prof. Cristian Luciano,
University of Illinois
Candidate:
Leonardo Crespi
![Page 2: Haptic Virtual Reality Training Simulator for Sacral ...nearlab.polimi.it/wp-content/uploads/2019/01/Presentazione_Leonar… · Virtual reality and Haptic simulator: Virtual reality:](https://reader034.fdocuments.net/reader034/viewer/2022050503/5f94e3e67502815a89201a23/html5/thumbnails/2.jpg)
Leonardo Crespi
Index
2
Background
State of the Art
Approach
Implementation
Test
ResultsAnd
discussion
Conclusions
SacralNeuromodulation
Surgical training
Proposedsolution
Hardware And
Software
![Page 3: Haptic Virtual Reality Training Simulator for Sacral ...nearlab.polimi.it/wp-content/uploads/2019/01/Presentazione_Leonar… · Virtual reality and Haptic simulator: Virtual reality:](https://reader034.fdocuments.net/reader034/viewer/2022050503/5f94e3e67502815a89201a23/html5/thumbnails/3.jpg)
Leonardo Crespi
• Urological condition defined by urinary
urgency with or without incontinence, urinary
frequency and/or nocturia
• >200M people affected worldwide, mainly
older people
• Not life threatening, mainly affects quality of
life
• Three levels of therapy:
1. Behavioral therapy
2. Drug therapy
3. Sacral Neuromodulation
Overactive Bladder
3
N. Eapen at al, 2016, “Review of the epidemiology of overactive bladder” https://www.sciencepicture.com/
![Page 4: Haptic Virtual Reality Training Simulator for Sacral ...nearlab.polimi.it/wp-content/uploads/2019/01/Presentazione_Leonar… · Virtual reality and Haptic simulator: Virtual reality:](https://reader034.fdocuments.net/reader034/viewer/2022050503/5f94e3e67502815a89201a23/html5/thumbnails/4.jpg)
Leonardo Crespi
• Surgical procedure, ≈3%
people affected by OAB are
treated with SNM
• Electrical pulses on the sacral
nerve through an implanted
device
• High costs (≈20000€ per
patient after 5 years)
• Better overall results than other
therapies
Sacral Neuromodulation (1/2)
https://www.xvivo.net/
4
Arlandis et al, 2011,“Cost-Effectiveness of Sacral Neuromodulation Compared to Botulinum Neurotoxin A or Continued Medical Management in Refractory Overactive Bladder”
Implanted device
![Page 5: Haptic Virtual Reality Training Simulator for Sacral ...nearlab.polimi.it/wp-content/uploads/2019/01/Presentazione_Leonar… · Virtual reality and Haptic simulator: Virtual reality:](https://reader034.fdocuments.net/reader034/viewer/2022050503/5f94e3e67502815a89201a23/html5/thumbnails/5.jpg)
Leonardo Crespi 5
Sacral Neuromodulation (2/2)
https://www.bidneedham.org/departments/urology/urology-our-specialities/sacral-neurostimulation
KE Matzel et al, 2017, ‘’Standardized electrode placement technique’’Surgical Procedure
Fluoroscopic guidance
• 2D imaging, A-P and Sagittal
• S3 foramen and landmarks identification
Percutaneous access
• S3 foramen
• Skin and fusion plane are the references
• Checked placement
• Guide with radio
marker
• Curved stylet with 4
electrodes
![Page 6: Haptic Virtual Reality Training Simulator for Sacral ...nearlab.polimi.it/wp-content/uploads/2019/01/Presentazione_Leonar… · Virtual reality and Haptic simulator: Virtual reality:](https://reader034.fdocuments.net/reader034/viewer/2022050503/5f94e3e67502815a89201a23/html5/thumbnails/6.jpg)
Leonardo Crespi
Current training systems
6
http://syndaver.com/gallery/
https://simbionix.com/simulators/uro-mentor/
Biological bench models• Real tissues and textures• Useful for simple
procedures• Non reusable• Different anatomy• Ethical concerns
Cadavers• Realistic• Difficult to be found
Virtual Reality• Adaptable• Reusable• Measurable
performance• Very expensive• No haptic feedback
Non biological bench models• High/ low fidelity• Very expensive in the first case,
very inaccurate in the second• Non reusable• Not adaptable
![Page 7: Haptic Virtual Reality Training Simulator for Sacral ...nearlab.polimi.it/wp-content/uploads/2019/01/Presentazione_Leonar… · Virtual reality and Haptic simulator: Virtual reality:](https://reader034.fdocuments.net/reader034/viewer/2022050503/5f94e3e67502815a89201a23/html5/thumbnails/7.jpg)
Leonardo Crespi
Virtual reality and Haptic simulator:
Virtual reality: anatomical accuracy,
adaptability, safety
Haptics: realism through tactile
feedback
Proposed Solution (1/2)
Three steps simulation:1. Percutaneus access2. Guide insertion3. Electrode lead insertion
7
Requirements of an ideal training
system:
• Anatomical accuracy
• Realism
• Adaptability
• User friendliness
• Measurability of the performance
• Safety
• Commercial viability and relevancy
![Page 8: Haptic Virtual Reality Training Simulator for Sacral ...nearlab.polimi.it/wp-content/uploads/2019/01/Presentazione_Leonar… · Virtual reality and Haptic simulator: Virtual reality:](https://reader034.fdocuments.net/reader034/viewer/2022050503/5f94e3e67502815a89201a23/html5/thumbnails/8.jpg)
Leonardo Crespi
Proposed Solution (2/2)
8
![Page 9: Haptic Virtual Reality Training Simulator for Sacral ...nearlab.polimi.it/wp-content/uploads/2019/01/Presentazione_Leonar… · Virtual reality and Haptic simulator: Virtual reality:](https://reader034.fdocuments.net/reader034/viewer/2022050503/5f94e3e67502815a89201a23/html5/thumbnails/9.jpg)
Leonardo Crespi
Implementation
CT scan
9
3D Models VR Environment
Haptic device
![Page 10: Haptic Virtual Reality Training Simulator for Sacral ...nearlab.polimi.it/wp-content/uploads/2019/01/Presentazione_Leonar… · Virtual reality and Haptic simulator: Virtual reality:](https://reader034.fdocuments.net/reader034/viewer/2022050503/5f94e3e67502815a89201a23/html5/thumbnails/10.jpg)
Leonardo Crespi
Hardware
Haptic device
Processor Graphic Card RAM
PC used at UIC Intel i5-2500K, 3.3GHz NVIDIA Quadro K600 DDR3, 8 GB
PC used at PoliMi Intel i7-6800K, 3.4GHz NVIDIA Titan Xp DDR3, 16 GB
10
Model Omni Touch 3D Stylus™
Positional feedback 6 DOF
Force feedback 3 DOF (Position only)
Workspace dimensions(width x heigth x depth [cm])
26.5x23.6x8.9
Nominal Max force [N] 3.3
Nominal position resolution [mm]
0.084https://it.3dsystems.com/haptics-devices/touch
![Page 11: Haptic Virtual Reality Training Simulator for Sacral ...nearlab.polimi.it/wp-content/uploads/2019/01/Presentazione_Leonar… · Virtual reality and Haptic simulator: Virtual reality:](https://reader034.fdocuments.net/reader034/viewer/2022050503/5f94e3e67502815a89201a23/html5/thumbnails/11.jpg)
Leonardo Crespi
Anatomical Models
• Contrast enhancement• Segmentation of the CT scan• 3D modeling to refine the meshes
11
![Page 12: Haptic Virtual Reality Training Simulator for Sacral ...nearlab.polimi.it/wp-content/uploads/2019/01/Presentazione_Leonar… · Virtual reality and Haptic simulator: Virtual reality:](https://reader034.fdocuments.net/reader034/viewer/2022050503/5f94e3e67502815a89201a23/html5/thumbnails/12.jpg)
Leonardo Crespi
Software Development Kit
LACE
QuickHaptics VisualizationLibrary
Wykobi
Efficientmathematicalcomputation. v0.0.5
OpenGL basedmiddleware to create 2D and 3D graphics; v2.0.0
Haptic device management; v3.0
12
http://visualizationlibrary.org/docs/2.0/html/index.html
http://www.wykobi.com/https://www.3dsystems.com/haptics-devices/openhaptics
![Page 13: Haptic Virtual Reality Training Simulator for Sacral ...nearlab.polimi.it/wp-content/uploads/2019/01/Presentazione_Leonar… · Virtual reality and Haptic simulator: Virtual reality:](https://reader034.fdocuments.net/reader034/viewer/2022050503/5f94e3e67502815a89201a23/html5/thumbnails/13.jpg)
Leonardo Crespi
Application features (1/3)
C-Arm simulation
Antero-posterior view
13
Sagittal view
Raycasting , α-compositing
𝐶𝑓𝑏 =
𝑖=𝑓
𝑏
𝐶𝑖𝑎𝑖ෑ
𝑗=𝑓
𝑖−1
𝑇𝑗
𝑪𝒇𝒃 : composite color ([r g b]) of samples from pixel b (back) to
pixel f (front) taken along a ray to project the volum on a plane;𝒂: opacity; T = 𝟏 − 𝒂 , transparency;
![Page 14: Haptic Virtual Reality Training Simulator for Sacral ...nearlab.polimi.it/wp-content/uploads/2019/01/Presentazione_Leonar… · Virtual reality and Haptic simulator: Virtual reality:](https://reader034.fdocuments.net/reader034/viewer/2022050503/5f94e3e67502815a89201a23/html5/thumbnails/14.jpg)
Leonardo Crespi
Application features (2/3)
Animations to simulate the patient’s response to electrical stimulation
▪ Vector of N images used as a flip book to display the patient response, streamed back and forth from 0 to nat 60Hz
▪ n computed through a specifically designed function of distance (d) and current intensity (i)
▪ Current intensity (i) set by the user through the Graphic User Interface
𝑛 =
𝑁, 𝑑 = 0;
𝑎𝑖
𝑑, 0 < 𝑑 ≤ 𝐷;
0, 𝑑 > 𝐷;
N
n
Vector of images
14
Needleclose to the target
Needleless closeto the target
![Page 15: Haptic Virtual Reality Training Simulator for Sacral ...nearlab.polimi.it/wp-content/uploads/2019/01/Presentazione_Leonar… · Virtual reality and Haptic simulator: Virtual reality:](https://reader034.fdocuments.net/reader034/viewer/2022050503/5f94e3e67502815a89201a23/html5/thumbnails/15.jpg)
Leonardo Crespi
Application features (3/3)
Bendable electrode▪ LACE_Extrusion implementation▪ Catmull-Rom spline (Continuous piecewise
defined polynomial function 𝑃 𝑡 ) to create the shape, updated each graphic frame
▪ 4 control points set according to the proxy position
▪ Tension parameter (f) set to .5
Equation between two point 𝑃𝑖−1 and 𝑃𝑖
𝑃 𝑡 = 1 𝑡 𝑡2 𝑡3
0 1 0 0−𝑓 0 𝑓 02𝑓 𝑓 − 3 3 − 2𝑓 −𝑓−𝑓 2 − 𝑓 𝑓 − 2 𝑓
𝑃𝑖−2𝑃𝑖−1𝑃𝑖𝑃𝑖+1
𝑃𝑖−2
𝑃𝑖−1
𝑃𝑖
𝑃𝑖+1
𝑃(𝑡)
15
![Page 16: Haptic Virtual Reality Training Simulator for Sacral ...nearlab.polimi.it/wp-content/uploads/2019/01/Presentazione_Leonar… · Virtual reality and Haptic simulator: Virtual reality:](https://reader034.fdocuments.net/reader034/viewer/2022050503/5f94e3e67502815a89201a23/html5/thumbnails/16.jpg)
Leonardo Crespi
Haptic effects (1/2)
Shape related
▪ Stiffness, viscosityand popthrough
▪ Range 0 – 1 in QuickHaptics API
▪ Tuned with the help of urologists
Stiffness Viscosity Popthrough
Spine 1 0 Not defined
Skin 0.4 0.5 0.3
Liver 0.2 0.3 Not defined
Lungs 0.15 0.1 Not defined
Kidneys 0.35 0.2 Not defined
Values assigned to the anatomical models
16
Force (F) computed when an object is touched
𝐹 = 𝑠𝑡𝑖𝑓𝑓𝑛𝑒𝑠𝑠 ∗ Δ𝐷 + 𝑣𝑖𝑠𝑐𝑜𝑠𝑖𝑡𝑦 ∗ 𝑣
Δ𝐷: depth of penetration inside the object, computed as the difference between the proxy position and its projection on the surface of the object;v: proxy velocity
![Page 17: Haptic Virtual Reality Training Simulator for Sacral ...nearlab.polimi.it/wp-content/uploads/2019/01/Presentazione_Leonar… · Virtual reality and Haptic simulator: Virtual reality:](https://reader034.fdocuments.net/reader034/viewer/2022050503/5f94e3e67502815a89201a23/html5/thumbnails/17.jpg)
Leonardo Crespi
Haptic effects (2/2)
17
Event Related Fulcrum effectTriggered when the skin is entered with the needle to simulate a puncture
Line effectTriggered when the electrodeis inserted in the guide, to simulate the constraint and drive it to the correct spot
p = p’ p
p
p’
p’
r rr
r’r’ r’
F
F
𝒓: 𝑓𝑢𝑙𝑐𝑟𝑢𝑚 𝑝𝑜𝑖𝑛𝑡𝒑: 𝑝𝑟𝑜𝑥𝑦 𝑝𝑜𝑠𝑖𝑡𝑖𝑜𝑛;𝑸 = 𝒒𝒘,𝒒𝒙, 𝒒𝒚, 𝒒𝒛𝑝𝑟𝑜𝑥𝑦 𝑜𝑟𝑖𝑒𝑛𝑡𝑎𝑡𝑖𝑜𝑛;
𝒓′ = 𝒓 + 𝒒𝒙, 𝒒𝒚, 𝒒𝒛 ;𝒍: 𝑙𝑖𝑛𝑒 𝑝𝑎𝑠𝑠𝑖𝑛𝑔 𝑓𝑜𝑟 𝒓 𝑎𝑛𝑑 𝒓′;𝒑′: 𝑝𝑟𝑜𝑗𝑒𝑐𝑡𝑖𝑜𝑛 𝑜𝑓 𝒑 𝑜𝑛 𝒍;
𝑭 = 𝒌 𝒑′ − 𝒑 𝑓𝑜𝑟𝑐𝑒
l l l
Proxy Proxy
Proxy
![Page 18: Haptic Virtual Reality Training Simulator for Sacral ...nearlab.polimi.it/wp-content/uploads/2019/01/Presentazione_Leonar… · Virtual reality and Haptic simulator: Virtual reality:](https://reader034.fdocuments.net/reader034/viewer/2022050503/5f94e3e67502815a89201a23/html5/thumbnails/18.jpg)
Leonardo Crespi
Preliminary test to understand how a novice might get along with the application
• 15 Biomedical Engineering Master students
• 2 repetitions for each subject
• Quantitative data: measured by the application in terms of total time, fluoroscopy time, distance from the target and interactions with the periosteum; Wilcoxon signed rank test (α=0.05) for paired data to evaluate statistical significance of differences between the repetitions
• Qualitative data through the System Usability Scale, a ten item Likertattitude test that gives a score in range 0-100, to assess how welldesigned the system is from the user’s point of view
Test
18
![Page 19: Haptic Virtual Reality Training Simulator for Sacral ...nearlab.polimi.it/wp-content/uploads/2019/01/Presentazione_Leonar… · Virtual reality and Haptic simulator: Virtual reality:](https://reader034.fdocuments.net/reader034/viewer/2022050503/5f94e3e67502815a89201a23/html5/thumbnails/19.jpg)
Leonardo Crespi
Quantitative data
Results and Discussion (1/2)
First observation Second observation
P-value 0.01
19
P-value 0.02
P-value 0.99 P-value 0.91
![Page 20: Haptic Virtual Reality Training Simulator for Sacral ...nearlab.polimi.it/wp-content/uploads/2019/01/Presentazione_Leonar… · Virtual reality and Haptic simulator: Virtual reality:](https://reader034.fdocuments.net/reader034/viewer/2022050503/5f94e3e67502815a89201a23/html5/thumbnails/20.jpg)
Leonardo Crespi
Qualitative data
Results and Discussion (2/2)
𝑆𝑈𝑆𝑠𝑐𝑜𝑟𝑒 = 2.5
𝑖=1
5
[ 𝑄2𝑖−1 − 1 + 5 − 𝑄2𝑖 ]
• Each item has a score which statesthe level of agreement with the sentence, ranging from 1 (minimum) to 5 (maximum)
• 𝑄𝑖 is the score for the 𝑖𝑡ℎ item• The final SUS score is the average
of the subjects𝐹𝑖𝑛𝑎𝑙 𝑆𝑈𝑆𝑠𝑐𝑜𝑟𝑒 = 𝟔𝟎. 𝟔𝟕 ± 𝟏𝟓. 𝟑𝟓
Not at all33%
Somewhat40%
Fairly27%
Very0%
I thought there was too muchinconsistency in this system
Not at all0%
Somewhat27%
Fairly7%
Very33%
Extremely33%
I think i would need the support of a technical personto be able to use this system
20
Not at all0%
Somewhat13%
Fairly34%Very
40%
Extremely13%
I found the various functions in this system were well
intergrated
Not at all
Somewhat
Fairly
Very
Extremely
![Page 21: Haptic Virtual Reality Training Simulator for Sacral ...nearlab.polimi.it/wp-content/uploads/2019/01/Presentazione_Leonar… · Virtual reality and Haptic simulator: Virtual reality:](https://reader034.fdocuments.net/reader034/viewer/2022050503/5f94e3e67502815a89201a23/html5/thumbnails/21.jpg)
Leonardo Crespi
Solution features with the
respect to the requirements
Conclusions
• Anatomical accuracy• Realism• Safety • Measurability of the
performance• Commercial viability and
relevance
• User friendliness
• Adaptability
21
▪ Create a library of virtual patients▪ More extensive tests, involving
medical experts and directlycomparing this system with others available
▪ Information about the angle that the needle should be keept at during the percutaneus access given the anatomy
▪ Replacing the 3D monitor with an head mounted display
Future developments
![Page 22: Haptic Virtual Reality Training Simulator for Sacral ...nearlab.polimi.it/wp-content/uploads/2019/01/Presentazione_Leonar… · Virtual reality and Haptic simulator: Virtual reality:](https://reader034.fdocuments.net/reader034/viewer/2022050503/5f94e3e67502815a89201a23/html5/thumbnails/22.jpg)
Leonardo Crespi
Aknowledgements
22