Post on 05-Sep-2020
Augmented reality for medical applications
Henrique Galvan Debarba
09.06.2017 e-Health conference www.artanim.ch
Outline
• Artanim Foundation• Augmented Reality • Holographic AR at Artanim• Tracking and navigation for Holographic AR• Limitations of current consumer Holographic AR
09.06.2017 e-Health conference www.artanim.ch
Artanim
• Artanim is a non-profit foundation founded in 2011 and dedicated to the development and promotion of motion capture
09.06.2017 e-Health conference www.artanim.ch
R&D – Medicine• Joint biomechanics• 3D simulation and
surgical planning• Medical imaging,
orthopedics and movement science
• Sport medicine and rehabilitation
R&D – Virtual Reality• Motion capture
applications from different technologies
• AR/VR applications• 3D body scanning
solutions• Real-time motion
interaction and animation
Production• Films• Games• Commercials• Live performances• …
Education• Courses• Workshops• Introductions• …
Medical approach
09.06.2017 www.artanim.ch
3D simulation3D simulation3D reconstruction
Kinematics
Anatomy
KKiinemattiics
Anatomy
333DD recons
Kinematics
tionstruct
atics
Data acquisitionDatta acquiiisiitttiiion Clinical applications
CCCClllliiiinnniiiicccaaallll li ti
e-Health conference
Applications
09.06.2017 e-Health conference www.artanim.ch
Simulation of hip prosthesis and detection of conflicts
Study of knee ligaments behavior for applications in sports medicine and
rehabilitation
Study of knee ligaments behavior for
Simulation and detection of shoulder impingements in tennis players
Mobile AR - Pokémon GO
Augmented Reality
• Real environment augmented by computer-generated sensoryinput
09.06.2017 e-Health conference www.artanim.ch
AR projection - AccuVein
Holographic AR - Microsoft Hololens
The Hololens
09.06.2017 e-Health conference www.artanim.ch
Microsoft
HoloMed Project
• Development of an AR tool for surgery allowing thevisualization and analysis of the anatomy in real timeand in motion.
• Funding: La Tour Hospital
09.06.2017 e-Health conference www.artanim.ch
HoloMed Project
• Points of interest:• Pre-operative planning and visualization of bone resections, with
biomechanical movement models• Medical visualization, annotation and education• Rehabilitation and continued care (e.g. physiotherapy)• Diagnostics and sports medicine
09.06.2017 e-Health conference www.artanim.ch
HoloMed Project
• Long term goal:• Use in the operating room: reduce complexity/training/length of
procedures that use computer assisted navigation.
09.06.2017 e-Health conference www.artanim.ch
AR in the Operating Room
09.06.2017 e-Health conference www.artanim.ch
Holographic AR and Tracking
09.06.2017 e-Health conference www.artanim.ch
• Accuracy:• tracking is designed for
recreational use
• External navigations system• Registration of center of
projection with navigation system
for
Holographic AR and Tracking
• Prediction and data fusion to compensate for latency
09.06.2017 e-Health conference www.artanim.ch
Holographic AR limitations
• Renders at constant accommodation depth • Virtual image is presented for an unique eye accommodation, regardless
of the depth of the virtual object.
09.06.2017 e-Health conference www.artanim.ch
Avegant
Holographic AR limitations
• Field-of-view on Hololens
• Interpupilar distance and eye to screen displacement
09.06.2017 e-Health conference www.artanim.ch
Plopski, Alexander, et al. "Corneal-imaging calibration for optical see-through head-mounted displays." IEEE transactions on visualization and computer graphics 21.4 (2015): 481-490.
Questions?
09.06.2017 e-Health conference www.artanim.ch
Volume Visualization
09.06.2017e-Health conference www.artanim.ch000099999...000066666666....222222200000000117777eeeeee77777777777777-----HHHHHHHeeeeaaaaaalltth ccccccccooooonnnnnfffffeeerrrrrreeeeeeeeeeeenncccceeee wwwwwwwwwwwwwwwwwwwwwwww..aaaaaaaaaaarrrrrrrrrtttttttaaannnnniiiiiimmmmmmmm..ccccccchhhhhh
HOLOGRAPHIC AR
09.06.2017 e-Health conference www.artanim.ch
Augmented Reality
• Real environment augmented by computer-generated sensory input
09.06.2017 e-Health conference www.artanim.ch
Milgran 1994
The Case for AR in Medicine
• 1997 - A Survey of Augmented Reality
• What has changed?
09.06.2017 e-Health conference www.artanim.ch
Outline
• Artanim• Summary of activities• Medical approach• Medical applications
• Augmented Reality at Artanim• HoloMed project• Hololens and navigation systems
• How we work to use it
• Intermediate results
09.06.2017 e-Health conference www.artanim.ch
Ongoing development
• Adaptation of • Optimal Data superposition• Human-computer interaction (Hands free control)
09.06.2017 e-Health conference www.artanim.ch
Karlsruhe Institute for Technology Technische Universität München
Microsoft Hololens
HolomedDuration: September 2016 – September 2017Funding: La Tour HospitalDescription: Development of an AR tool for surgery allowing the visualization and analysis of the anatomy in real time and in motion
Navigation System
09.06.2017 e-Health conference www.artanim.ch
Overview
• Currently, AR involving moving real objects require external tracking.
• Consumer Holographic AR still has a long way to go:• intrinsic limitations for interaction in the reachable volume of space.
09.06.2017 e-Health conference www.artanim.ch