Dep

art

me

nt

of

Sysyem

En

gin

eeeri

ng

an

d A

uto

mati

on

Víctor F. Muñoz Martínez

Research lines

Irene Rivas Blanco

Department of System Engineering and Automation

University of Malaga

MANEUVERS RECOGNITION IN

LAPAROSCOPIC SURGERY: ARTIFICIAL

NEURAL NETWORK AND HIDDEN

MARKOV MODEL APPROACHES

Dep

art

me

nt

of

Sysyem

En

gin

eeeri

ng

an

d A

uto

mati

on

Víctor F. Muñoz Martínez

Research lines

I. Introduction

II. Maneuver Recognition System

III. Implementation & Experiments

IV. Conclusions

INDEX

Dep

art

me

nt

of

Sysyem

En

gin

eeeri

ng

an

d A

uto

mati

on

Víctor F. Muñoz Martínez

Research lines

I. INTRODUCTION

Dep

art

me

nt

of

Sysyem

En

gin

eeeri

ng

an

d A

uto

mati

on

Víctor F. Muñoz Martínez

Research lines

I. INTRODUCTION

• Surgical assistant robot interfaces:

Direct-Teleoperation Head tracking

Voice commands

Eyes trackingVision algorithms

Dep

art

me

nt

of

Sysyem

En

gin

eeeri

ng

an

d A

uto

mati

on

Víctor F. Muñoz Martínez

Research lines

I. INTRODUCTION

(HMM) (HMM)

• Intuitive and natural Human-Machine Interface: Maneuver Recognition System

• Based on modeling the surgeon’s movements

• Comparison between two modeling approaches: Artificial Neural Networks (ANN) and Hidden Markov Models (HMM)

Four degrees of freedom

Dep

art

me

nt

of

Sysyem

En

gin

eeeri

ng

an

d A

uto

mati

on

Víctor F. Muñoz Martínez

Research lines

II. MANEUVER RECOGNITION

SYSTEM

Dep

art

me

nt

of

Sysyem

En

gin

eeeri

ng

an

d A

uto

mati

on

Víctor F. Muñoz Martínez

Research lines

II. MANUEVER RECOGNITION SYSTEM

Surgical Protocol Maneuver

Dep

art

me

nt

of

Sysyem

En

gin

eeeri

ng

an

d A

uto

mati

on

Víctor F. Muñoz Martínez

Research lines

II. MANUEVER RECOGNITION SYSTEM

SENSORIAL SYSTEM

PREPROCESSING DATA

CODING DATA

Surgical Tools’ movement

Kalman filter

Two Tracking 3D sensors

ANN Fourier

RECOGNITION SYSTEM

HMM ANN

Maneuver code

Maneuver code

Data numerical description

Modeling of surgeon’s movements

Dep

art

me

nt

of

Sysyem

En

gin

eeeri

ng

an

d A

uto

mati

on

Víctor F. Muñoz Martínez

Research lines

II. MANUEVER RECOGNITION SYSTEM

• ANN: Recognizes the trajectory of the tools’ tip

FOURIERTrajectories ANNManeuver

Code

Set of vectors

CODING DATA RECOGNITION SYSTEM

• HMM: Recognizes the interaction between the tools

ANNCharacteristic

vector

HMM1

Maneuver Code

Observablecode

CODING DATA

RECOGNITION SYSTEM

HMM2

HMMn

…

MAX

PROB.

Dep

art

me

nt

of

Sysyem

En

gin

eeeri

ng

an

d A

uto

mati

on

Víctor F. Muñoz Martínez

Research lines

III. IMPLEMENTATION &

EXPERIMENTS

Dep

art

me

nt

of

Sysyem

En

gin

eeeri

ng

an

d A

uto

mati

on

Víctor F. Muñoz Martínez

Research lines

III. IMPLEMENTATION & EXPERIMENTS

Polaris Spectra

Markers

CODE MANEUVER

1 Cutting

2 Suturing

3 Transporting

Dep

art

me

nt

of

Sysyem

En

gin

eeeri

ng

an

d A

uto

mati

on

Víctor F. Muñoz Martínez

Research lines

III. IMPLEMENTATION & EXPERIMENTS

• Experiments: maneuvers recognition

Dep

art

me

nt

of

Sysyem

En

gin

eeeri

ng

an

d A

uto

mati

on

Víctor F. Muñoz Martínez

Research lines

Maneuver Training Validation Hits

Suturing 150 35 94,29%

Cutting 150 35 91,43%

Transporting 150 35 94,29%

ANN

Maneuver Training Validation Hits

Suturing 150 35 88,57%

Cutting 150 35 82,86%

Transporting 150 35 85,71%

HMM

III. IMPLEMENTATION & EXPERIMENTS

• Experiments results:

Dep

art

me

nt

of

Sysyem

En

gin

eeeri

ng

an

d A

uto

mati

on

Víctor F. Muñoz Martínez

Research lines

IV. CONCLUSIONS

Dep

art

me

nt

of

Sysyem

En

gin

eeeri

ng

an

d A

uto

mati

on

Víctor F. Muñoz Martínez

Research lines

IV. CONCLUSIONS

• Human-Machine interface to recognize maneuvers during a laparoscopic surgery

• Comparison between two modeling approaches: Artificial Neural Networks and Hidden Markov Models

• ANN is an intuitive approach, but it is based on trajectories analysis in a specific reference frame.

• HMM provides an independent reference movement frame recognition system.

Dep

art

me

nt

of

Sysyem

En

gin

eeeri

ng

an

d A

uto

mati

on

Víctor F. Muñoz Martínez

Research lines

THANK YOU FOR YOUR ATTENTION