Seite x

Vision and Project

Goals

Background

Standard Operating Procedure (SOP)

Troubleshooting

Team

Budget

Sponsoring

Contact

page 1

In a team of seven mechanical and electrical engineering students, we aim

to develop innovative software applications for the Microsoft HoloLens

that will be deployed in cardiac surgery. Our applications for the HoloLens

will offer various technical abilities to support the surgeon and the

technician during interventions with an ECMO (Extracorporeal membrane

oxygenation). We want to simplify the communication between the

operators, enable a standardization of the procedure and minimize

serious complications caused by human error. During these nine months

of the project, we will go through all the steps of product development

such as design, implementation and marketing.

It is our vision to increase patient safety and ultimately improve survival

rate by using the HoloLens in life-saving cardiac procedures. With

augmented reality, we can support, optimize and simplify various

processes for surgeons and technicians.

simplify optimize support

page 2

The goal will be two user-friendly and intuitive applications. One should

facilitate the training of surgeons and enhance their performance by

containing a standard operating procedure tutorial. The second

application should make operators aware of complications and assist the

operator in taking corrective action by displaying a troubleshooting guide.

To establish augmented reality in cardiac surgery we implement a

standard operating procedure on the HoloLens. We want to enable an

interactive learning tool for surgeons and technicians with integrated

feedback.

Our goal for the Troubleshooting application is that it will be used in real

world situations on patients. However, a first step towards our goal would

be a proof of concept, where the complications are simulated. Despite the

fact that this first iteration would be a simulation, our aim is to use the

most realistic data sources possible, in order to simplify the transfer of the

technology into an environment with a real patient.

Standard Operating Procedure (SOP) Troubleshooting

SOP Tutorial

Step by Step Instruction

Incorporate Computer Vision

Tool Recognition

Verification of individual steps

For Simulation-Based Training

Troubleshooting Guide

Tree structure

Navigated by Surgeon

Automated Troubleshooting

Tracking of Data

Automatic Generated Warnings

For Use in Real-Life Situations

page 3

Project Background

Students at ETH Zurich have the chance to take part in a focus project

towards the end of their Bachelor degree in mechanical or electrical

engineering. The main goal is to put the acquired knowledge into practice

and come up with innovative solutions. All teams have the opportunity to

present their products to the public at the rollout on May 26th 2020.

Technical Background

Augmented reality adds computer generated objects to real world. This is

mainly accomplished by using glasses with integrated display functions to

superimpose holograms on to your optical perception of the

surroundings. One of the devices is Microsoft’s HoloLens.

Medical Background

Extracorporeal membrane oxygenation (ECMO) is a technique to provide

a patient with respiratory and cardiac support, if their heart and lungs are

unable to provide enough gas exchange or perfusion to sustain life. Due

to the complex behavior of pump ratio, blood flow and oxygen level it is

difficult to adjust the device correctly if complications occur. With the

HoloLens we can support surgeons and technicians and enhance their

performance.

page 4

The picture above shows, how the Layout of the SOP could look like.

In order to simplify the training of surgeons, we will implement an SOP

guide for the surgical procedure of the ECMO cannulation. In a first

iteration, this will be a step-by-step tutorial with tips, tricks, pictures and

video material. One possible extension is to monitor the individual steps

of the training through computer vision and tool recognition. For

example, it would be possible to check whether the surgeon has selected

the correct cannula size using coloured stickers on the cannulas. An

advantage of this application is that the surgeon already works with the

HoloLens during training and gets to know the device. Therefore, other

augmented reality applications can be used in a real operating

environment sooner, since the user is already familiar with the

technology.

page 5

A first step towards an automatic troubleshooting application would be

to have the operator work their way through a tree-like structure of

symptoms, which then ends in a possible solution, on which they can

take action. The next step would then be that the software is able to

recognize complications without direct input from the operator.

Our plan to achieve this involves using the data generated by an ECMO

and possibly vital signs from other monitors to determine which

complication the patient might have. We would collect the data on a

laptop, where also the patient information is filled in. From there, the

combined data is transferred to the HoloLens, where the variables of

interest are tracked. If a value lies outside the normal range, our software

would then alert the wearer to the issue and suggest possible

countermeasures or solutions. An example would be that the software

detects a high arterial pressure and in response, prompts the operator to

check the arterial cannula for kinks or other mechanical stress.

The troubleshooting guide could be similar to the visualization above.

page 6

Luca Vicentini

Biomedical Engineer

Maximilian Halbe

Leading Perfusionist

Prof. Dr. Francesco Maisano

Project Partner

Prof. Dr. Mirko Meboldt

Head of pd|z

Julian Wolf

Project Coach

Livio Bereiter

Student Coach

Patrick Pfreundschuh

Student Coach

Mathias Van Hemelrijck

Assistant Surgeon

page 7

Marcial Egli

User Experience &

Testing

Viviane Wolfer

User Experience &

Testing

Matthew Hanlon

Software Development

Ladina Roffler

Software Development

Vincent van der Brugge

Software Development

Siri Rüegg

Finances & Media

Sandra Haltmeier

Organization & Sponsoring

page 8

55%

17%

14%

10%4%

Hardware

Testing

Marketing and Events

Software

Others

Our project depends on sponsors to ensure its success, as all kinds of

costs will occur. We estimate a total cost of 25’000 to 30’000 CHF. The

budget can be divided into the following categories:

Hardware: To enable software development with the HoloLens,

workstations and other infrastructure are needed.

Testing: There are costs for upgrading and maintaining the

simulation environment. This environment is equipped with

medical equipment, simulators and surgical tools. In addition,

various consumables are needed for performing the tests.

Marketing and Events: For professional appearance at various

events, supporting materials and team shirts are needed. In order

to present our product in a realistic way, it is necessary to set up a

suitable environment.

Software: Commercial software is needed for efficient work

sharing and a user-friendly interface. Additional computing power

(e.g. via cloud) is also costly.

page 9

Sponsoring

Individual sponsor

Please feel free to contact us anytime with individual offers. We will

endeavor to find a solution that suits your expectations.

Financial resources and technical expertise are indispensable in any

product development. Your support enables our success. We would be

delighted to hear from you.

ETH Zürich

Focus Project ARORA

Sandra Haltmeier – Sponsoring

Phone: +41 44 633 82 21

Mobile: +41 77 406 21 63

E-Mail: arora@ethz.ch

LEO B 8.4

Leonhardstrasse 27

8092 Zürich

Switzerland

www.arora.ethz.ch

www.instagram.com/arora_ethz

www.facebook.com/aroraethz

www.linkedin.com/company/aroraethz