Dept CISE, U of Florida Towards surgeon-authored VR ... · Towards surgeon-authored VR training:...
Transcript of Dept CISE, U of Florida Towards surgeon-authored VR ... · Towards surgeon-authored VR training:...
![Page 1: Dept CISE, U of Florida Towards surgeon-authored VR ... · Towards surgeon-authored VR training: the scene-development cycle NIH R01-EB018625 Grace Peng MMVR -- 2016 Saleh Dindar,](https://reader033.fdocuments.net/reader033/viewer/2022051906/5ff8db69d3cfbe392650b885/html5/thumbnails/1.jpg)
Towards surgeon-authored VR training: the scene-development cycle
NIH R01-EB018625 Grace Peng
MMVR -- 2016
Saleh Dindar, Thien Nguyen, Jӧrg Peters
Dept CISE, U of Florida
1
![Page 2: Dept CISE, U of Florida Towards surgeon-authored VR ... · Towards surgeon-authored VR training: the scene-development cycle NIH R01-EB018625 Grace Peng MMVR -- 2016 Saleh Dindar,](https://reader033.fdocuments.net/reader033/viewer/2022051906/5ff8db69d3cfbe392650b885/html5/thumbnails/2.jpg)
Outline● Authoring by Surgeon-Educators
● Distributing the work
● Surgeon-educator interface
● Modeling artist interface
2
![Page 3: Dept CISE, U of Florida Towards surgeon-authored VR ... · Towards surgeon-authored VR training: the scene-development cycle NIH R01-EB018625 Grace Peng MMVR -- 2016 Saleh Dindar,](https://reader033.fdocuments.net/reader033/viewer/2022051906/5ff8db69d3cfbe392650b885/html5/thumbnails/3.jpg)
Teaching Minimally Invasive Procedures
Practicing on animals
3
Box trainer
VR trainer
![Page 4: Dept CISE, U of Florida Towards surgeon-authored VR ... · Towards surgeon-authored VR training: the scene-development cycle NIH R01-EB018625 Grace Peng MMVR -- 2016 Saleh Dindar,](https://reader033.fdocuments.net/reader033/viewer/2022051906/5ff8db69d3cfbe392650b885/html5/thumbnails/4.jpg)
Advantages of Virtual Reality Simulators● Practice decision-making
● Objective measurement of
performance
● Low-cost, portable
● Customizable ?
4
![Page 5: Dept CISE, U of Florida Towards surgeon-authored VR ... · Towards surgeon-authored VR training: the scene-development cycle NIH R01-EB018625 Grace Peng MMVR -- 2016 Saleh Dindar,](https://reader033.fdocuments.net/reader033/viewer/2022051906/5ff8db69d3cfbe392650b885/html5/thumbnails/5.jpg)
Customizable? Authoring by Surgeons ➢ Rare procedures need more
training, but are low volume
➢ Anatomical variation
➢ Master surgeons teach their
unique approach
➢ Non standard anatomy
(tumor)
5
![Page 7: Dept CISE, U of Florida Towards surgeon-authored VR ... · Towards surgeon-authored VR training: the scene-development cycle NIH R01-EB018625 Grace Peng MMVR -- 2016 Saleh Dindar,](https://reader033.fdocuments.net/reader033/viewer/2022051906/5ff8db69d3cfbe392650b885/html5/thumbnails/7.jpg)
Toolkit for Illustration of procedures in Surgery (TIPS): Creation-Validation Cycle 7
![Page 9: Dept CISE, U of Florida Towards surgeon-authored VR ... · Towards surgeon-authored VR training: the scene-development cycle NIH R01-EB018625 Grace Peng MMVR -- 2016 Saleh Dindar,](https://reader033.fdocuments.net/reader033/viewer/2022051906/5ff8db69d3cfbe392650b885/html5/thumbnails/9.jpg)
Toolkit for Illustration of procedures in Surgery (TIPS): Creation-Validation Cycle 9
![Page 10: Dept CISE, U of Florida Towards surgeon-authored VR ... · Towards surgeon-authored VR training: the scene-development cycle NIH R01-EB018625 Grace Peng MMVR -- 2016 Saleh Dindar,](https://reader033.fdocuments.net/reader033/viewer/2022051906/5ff8db69d3cfbe392650b885/html5/thumbnails/10.jpg)
Modeling and testing a Stomach model using Blender2SOFA10
![Page 11: Dept CISE, U of Florida Towards surgeon-authored VR ... · Towards surgeon-authored VR training: the scene-development cycle NIH R01-EB018625 Grace Peng MMVR -- 2016 Saleh Dindar,](https://reader033.fdocuments.net/reader033/viewer/2022051906/5ff8db69d3cfbe392650b885/html5/thumbnails/11.jpg)
Toolkit for Illustration of procedures in Surgery (TIPS): Creation-Validation Cycle 11
![Page 12: Dept CISE, U of Florida Towards surgeon-authored VR ... · Towards surgeon-authored VR training: the scene-development cycle NIH R01-EB018625 Grace Peng MMVR -- 2016 Saleh Dindar,](https://reader033.fdocuments.net/reader033/viewer/2022051906/5ff8db69d3cfbe392650b885/html5/thumbnails/12.jpg)
➢ Surgeon-author level
○ Create and share teaching modules
○ Automatically instantiate a simulation
➢ Modeling-artist level
○ Design the anatomical structures
○ Specify physical behavior
○ Quickly test the designs
➢ Trainee level
○ VR simulation
○ Haptic interaction with the soft-tissue
➢ Developer level (sofa)
12
Authoring by Surgeons
![Page 13: Dept CISE, U of Florida Towards surgeon-authored VR ... · Towards surgeon-authored VR training: the scene-development cycle NIH R01-EB018625 Grace Peng MMVR -- 2016 Saleh Dindar,](https://reader033.fdocuments.net/reader033/viewer/2022051906/5ff8db69d3cfbe392650b885/html5/thumbnails/13.jpg)
Outline● Authoring by Surgeon-Educators
● Distributing the work
● Surgeon-educator interface
● Modeling artist interface
13
![Page 14: Dept CISE, U of Florida Towards surgeon-authored VR ... · Towards surgeon-authored VR training: the scene-development cycle NIH R01-EB018625 Grace Peng MMVR -- 2016 Saleh Dindar,](https://reader033.fdocuments.net/reader033/viewer/2022051906/5ff8db69d3cfbe392650b885/html5/thumbnails/14.jpg)
Old workflow14
Blender 3D modeling
SOFA simulation framework
![Page 15: Dept CISE, U of Florida Towards surgeon-authored VR ... · Towards surgeon-authored VR training: the scene-development cycle NIH R01-EB018625 Grace Peng MMVR -- 2016 Saleh Dindar,](https://reader033.fdocuments.net/reader033/viewer/2022051906/5ff8db69d3cfbe392650b885/html5/thumbnails/15.jpg)
NEW: Modeling-Artist Interface
➢ Blender interface:
model geometry(anatomy)
➢ Blender2SOFA plug-in:
define soft-tissue physics
➢ Test: auto-instantiated
simulation
15
![Page 16: Dept CISE, U of Florida Towards surgeon-authored VR ... · Towards surgeon-authored VR training: the scene-development cycle NIH R01-EB018625 Grace Peng MMVR -- 2016 Saleh Dindar,](https://reader033.fdocuments.net/reader033/viewer/2022051906/5ff8db69d3cfbe392650b885/html5/thumbnails/16.jpg)
Auto-generation of Visual, Physical and Collision Models16
PhysicalModel
CollisionModel
VisualModelGeometry
![Page 17: Dept CISE, U of Florida Towards surgeon-authored VR ... · Towards surgeon-authored VR training: the scene-development cycle NIH R01-EB018625 Grace Peng MMVR -- 2016 Saleh Dindar,](https://reader033.fdocuments.net/reader033/viewer/2022051906/5ff8db69d3cfbe392650b885/html5/thumbnails/17.jpg)
Specifying Physical behavior17
![Page 18: Dept CISE, U of Florida Towards surgeon-authored VR ... · Towards surgeon-authored VR training: the scene-development cycle NIH R01-EB018625 Grace Peng MMVR -- 2016 Saleh Dindar,](https://reader033.fdocuments.net/reader033/viewer/2022051906/5ff8db69d3cfbe392650b885/html5/thumbnails/18.jpg)
Volumetric Object Creation● Extrude in normal direction to
create thick shells
● Fill the space between two
models to create connecting
tissue
● Create volumetric models
embedding organs in fat
● Convert curves into volumetric
clamp & cut vessel models
18
![Page 19: Dept CISE, U of Florida Towards surgeon-authored VR ... · Towards surgeon-authored VR training: the scene-development cycle NIH R01-EB018625 Grace Peng MMVR -- 2016 Saleh Dindar,](https://reader033.fdocuments.net/reader033/viewer/2022051906/5ff8db69d3cfbe392650b885/html5/thumbnails/19.jpg)
● Specify Physical Behavior
○ Assign physical behaviors to geometry objects
○ Adjust parameters for physical behaviors
● Generate volumetric objects from surface geometry
● Generate visual, collision and physical models from
geometry
● Generate attachments between objects
● Generate fixed constraints on deformable objects
● Define surgical instruments with haptic force
● THANK YOU - QUESTIONS?
19
Blender2SOFA features
![Page 20: Dept CISE, U of Florida Towards surgeon-authored VR ... · Towards surgeon-authored VR training: the scene-development cycle NIH R01-EB018625 Grace Peng MMVR -- 2016 Saleh Dindar,](https://reader033.fdocuments.net/reader033/viewer/2022051906/5ff8db69d3cfbe392650b885/html5/thumbnails/20.jpg)
● Modifications to SOFA simulation software:
○ Source code: bitbucket.org/surflab/sofa
● Blender2SOFA plugin:
○ Source code: bitbucket.org/surflab/blender2sofa
● TIPS-Author
○ Published: tips-author.appspot.com
○ Source code: bitbucket.org/surflab/tips-author
20
Source Code
![Page 21: Dept CISE, U of Florida Towards surgeon-authored VR ... · Towards surgeon-authored VR training: the scene-development cycle NIH R01-EB018625 Grace Peng MMVR -- 2016 Saleh Dindar,](https://reader033.fdocuments.net/reader033/viewer/2022051906/5ff8db69d3cfbe392650b885/html5/thumbnails/21.jpg)
Surgeon-Educator Interface● Enable surgeons to deconstruct a surgical
procedure
○ Stage: a series of tasks to complete an objective
○ Task: a single action involving
■ Verb
■ Organ
■ Instrument
● Review lists of available organs and tools to
use
● Auto-instantiate the simulation from a
description
21
![Page 22: Dept CISE, U of Florida Towards surgeon-authored VR ... · Towards surgeon-authored VR training: the scene-development cycle NIH R01-EB018625 Grace Peng MMVR -- 2016 Saleh Dindar,](https://reader033.fdocuments.net/reader033/viewer/2022051906/5ff8db69d3cfbe392650b885/html5/thumbnails/22.jpg)
Outline● Authoring by Surgeon-Educators
● Distributing the work
● Surgeon-educator interface
● Modeling artist interface
22
![Page 23: Dept CISE, U of Florida Towards surgeon-authored VR ... · Towards surgeon-authored VR training: the scene-development cycle NIH R01-EB018625 Grace Peng MMVR -- 2016 Saleh Dindar,](https://reader033.fdocuments.net/reader033/viewer/2022051906/5ff8db69d3cfbe392650b885/html5/thumbnails/23.jpg)
Outline● Authoring by Surgeon-Educators
● Distributing the work
● Surgeon-author interface
● Modeling artist interface
23