Augmented reality for medical applications...•Field-of-view on Hololens •Interpupilar distance...

Augmented reality for medical applications

Henrique Galvan Debarba

09.06.2017 e-Health conference www.artanim.ch

Outline

• Artanim Foundation• Augmented Reality • Holographic AR at Artanim• Tracking and navigation for Holographic AR• Limitations of current consumer Holographic AR


Artanim

• Artanim is a non-profit foundation founded in 2011 and dedicated to the development and promotion of motion capture


R&D – Medicine• Joint biomechanics• 3D simulation and

surgical planning• Medical imaging,

orthopedics and movement science

• Sport medicine and rehabilitation

R&D – Virtual Reality• Motion capture

applications from different technologies

• AR/VR applications• 3D body scanning

solutions• Real-time motion

interaction and animation

Production• Films• Games• Commercials• Live performances• …

Education• Courses• Workshops• Introductions• …

https://youtu.be/-LA0nhe4D5g

Medical approach

09.06.2017 www.artanim.ch

3D simulation3D simulation3D reconstruction

Kinematics

Anatomy

KKiinemattiics

Anatomy

333DD recons

Kinematics

tionstruct

atics

Data acquisitionDatta acquiiisiitttiiion Clinical applications

CCCClllliiiinnniiiicccaaallll li ti

e-Health conference

Applications


Simulation of hip prosthesis and detection of conflicts

Study of knee ligaments behavior for applications in sports medicine and

rehabilitation

Study of knee ligaments behavior for

Simulation and detection of shoulder impingements in tennis players

https://youtu.be/NLh1a8rzYlU

https://youtu.be/ZfWqfS8fCmc

https://youtu.be/Uh82hzf5Cjc

https://youtu.be/3FgvJEa2F_Y

https://youtu.be/whUWy_apBwk

Mobile AR - Pokémon GO

Augmented Reality

• Real environment augmented by computer-generated sensoryinput


AR projection - AccuVein

Holographic AR - Microsoft Hololens

The Hololens


Microsoft

HoloMed Project

• Development of an AR tool for surgery allowing thevisualization and analysis of the anatomy in real timeand in motion.

• Funding: La Tour Hospital


HoloMed Project

• Points of interest:• Pre-operative planning and visualization of bone resections, with

biomechanical movement models• Medical visualization, annotation and education• Rehabilitation and continued care (e.g. physiotherapy)• Diagnostics and sports medicine


HoloMed Project

• Long term goal:• Use in the operating room: reduce complexity/training/length of

procedures that use computer assisted navigation.


AR in the Operating Room


Holographic AR and Tracking


• Accuracy:• tracking is designed for

recreational use

• External navigations system• Registration of center of

projection with navigation system

for

Holographic AR and Tracking

• Prediction and data fusion to compensate for latency


https://youtu.be/8kbIca0w2x0

https://youtu.be/CUR3EXZ6Omg

Holographic AR limitations

• Renders at constant accommodation depth • Virtual image is presented for an unique eye accommodation, regardless

of the depth of the virtual object.


Avegant

Holographic AR limitations

• Field-of-view on Hololens

• Interpupilar distance and eye to screen displacement


Plopski, Alexander, et al. "Corneal-imaging calibration for optical see-through head-mounted displays." IEEE transactions on visualization and computer graphics 21.4 (2015): 481-490.

Questions?


Volume Visualization

09.06.2017e-Health conference www.artanim.ch000099999...000066666666....222222200000000117777eeeeee77777777777777-----HHHHHHHeeeeaaaaaalltth ccccccccooooonnnnnfffffeeerrrrrreeeeeeeeeeeenncccceeee wwwwwwwwwwwwwwwwwwwwwwww..aaaaaaaaaaarrrrrrrrrtttttttaaannnnniiiiiimmmmmmmm..ccccccchhhhhh

HOLOGRAPHIC AR

https://youtu.be/Jya2k2OCUWU

09.06.2017e-Health conference www.artanim.ch

https://youtu.be/Tne_lNJT71U

Augmented Reality

• Real environment augmented by computer-generated sensory input


Milgran 1994

The Case for AR in Medicine

• 1997 - A Survey of Augmented Reality

• What has changed?


Outline

• Artanim• Summary of activities• Medical approach• Medical applications

• Augmented Reality at Artanim• HoloMed project• Hololens and navigation systems

• How we work to use it

• Intermediate results


Ongoing development

• Adaptation of • Optimal Data superposition• Human-computer interaction (Hands free control)


Karlsruhe Institute for Technology Technische Universität München

Microsoft Hololens

HolomedDuration: September 2016 – September 2017Funding: La Tour HospitalDescription: Development of an AR tool for surgery allowing the visualization and analysis of the anatomy in real time and in motion

Navigation System


Overview

• Currently, AR involving moving real objects require external tracking.

• Consumer Holographic AR still has a long way to go:• intrinsic limitations for interaction in the reachable volume of space.


Augmented reality for medical applications...•Field-of-view on Hololens •Interpupilar distance...

Documents

Transcript of Augmented reality for medical applications...•Field-of-view on Hololens •Interpupilar distance...