Post on 25-Nov-2021
Planning, optimisation and classification of 3D trajectories for robotic steerable needles in keyhole neurosurgery with a deductive reasoning approach28th April 2021
Candidate: Valentina Corbetta, 919294
Supervisor: Prof. Elena De Momi
Co-supervisor: Prof. Francesco Calimeri, Dott. Ing. Alice Segato
Politecnico di Milano
School of Industrial and Information Engineering
Department of Electronics, Information and Bioengineering (DEIB)
Master of Science in Biomedical Engineering
A.Y. 2019-2020
Presentation outline
Introduction
State of the Art
Statement of Purpose
Materials
Path planning
Optimisation and classification
Results
Conclusions and Future Work
2
Valentina Corbetta
Valentina Corbetta
Keyhole neurosurgery
3
Robotic tools Keyhole neurosurgery
Allows access to the brain througha tiny hole in the skull
Advantages:● Lower infection rates● Lower complications● Overall better post-operative
outcomes
Introduction
4
Steerable needlesIntroduction Valentina Corbetta
Rigid needles
Straight trajectories
4
Valentina Corbetta
Steerable needlesIntroduction
Rigid needles
Steerable needles
Curvilinear trajectories
Straight trajectories
4
Valentina Corbetta
Rigid needles
Steerable needles
Curvilinear trajectories
Advantages:● Active control of needle
trajectory● Compensate for target
movement● Access to deep structures
Drawbacks:● Require complex path planning
Solutions:● development of an automatic
path planner
Straight trajectoriesSteerable needlesIntroduction
5
Path planning problemState of the Art Valentina Corbetta
Uobst
Ufree
qI
qG
q1
path P(p0, p1, …, pn-1)
possible actions
Definition of a collision-free path in Ufree from an Entry Point (EP) to a Target Point (TP).
In neurosurgery:
● Clearance from safety regions: corticospinal tracts, vessels, ventricles, thalamus
● Respect of kinematic constraints of the needle:○ outer diameter (OD)○ maximum curvature (Kmax)
6
Path planning methodsState of the Art Valentina Corbetta
Graph-based
● Only minimise length● Do not take into account
expert’s knowledge● Do not take into account
the kinematic constraints
Leibrandt et al., Likhachev et al.
6
Valentina Corbetta
Path planning methodsState of the Art
Graph-based
● Only minimise length● Do not take into account
expert’s knowledge● Do not take into account
the kinematic constraints
Sampling-based
● Only minimise length● Do not take into account
the kinematic constraints● Do not take into account
expert’s knowledge
Leibrandt et al., Likhachev et al. Segato et al., Patil et al.
6
Valentina Corbetta
Path planning methodsState of the Art
Graph-based
● Only minimise length● Do not take into account
expert’s knowledge● Do not take into account
the kinematic constraints
Sampling-based
● Only minimise length● Do not take into account
the kinematic constraints● Do not take into account
expert’s knowledge
Learning-based
● Require large datasets for training
● Computationally intensive
Segato et al., Patil et al.Leibrandt et al., Likhachev et al. Tan et al., Chi et al.
6
Valentina Corbetta
Graph-based
● Only minimise length● Do not take into account
expert’s knowledge● Do not take into account
the kinematic constraints
Sampling-based
● Only minimise length● Do not take into account
the kinematic constraints● Do not take into account
expert’s knowledge
Learning-based
● Require large datasets for training
● Computationally intensive
Reasoning-based
● Explicitly represent domain knowledge
● Take into account kinematic constraints
● Take into account expert’s knowledge
● Take into account many parameters for optimisation
Path planning methodsState of the Art
Segato et al., Patil et al.Leibrandt et al., Likhachev et al. Tan et al., Chi et al.
7
Statement of Purpose Valentina Corbetta
The aim of this thesis is to try and solve the path planning problem for steerable needles in neurosurgery with a reasoning-based approach. The end goal is to develop a tool that can assist the neurosurgeon in the pre-operative phase, taking into account the kinematic constraints of the needle and leveraging the clinician’s expertise, to find the optimal path to the target structure.
8
Materials Valentina Corbetta
● Declarative programming paradigm born in the field of logic programming and non-monotonic reasoning
● A problem is modeled by a collection of rules; the solution of the encoded model is called answer set
● Rules are in the form
a0|...|ah :- b1,...,bn not bn+1,...,bm
atoms
head body
● A fact is a rule with a single element in the head and no body; it represents a certainly true information
● A constraint is a rule with empty head; they can be hard (symbol :- and must be satisfied) or weak (symbol :~ and should be satisfied)
Answer Set Programming (ASP)
8
Materials Valentina Corbetta
r1: color(X,red) | color(X,blue) | color(X, lightblue):- node(X).r2: :- arc(X,Y), color(X,C), color(Y,C).
Why ASP
9
Path planning Valentina Corbetta
Path planning with ASP
Environment
Dataset
Resize
Voxel Grid
EP and TS
selection
Path planner
arrival_at(T):-needle_at(T,X,Y,Z),finish(X,Y,Z).
arrival :- arrival_at(T).
Shortest Path
2
1
3
4
5
Set of rules and encoded problem in ASP
Segato, A., Corbetta, V., Calimeri, F., & De Momi, E. (2020, October). Inductiveand Deductive Reasoning for Robotic Steerable Needle in Neurosurgery.
In 2020 IEEE International Conference on Intelligent Robots and Systems.
10
Optimisation and classification Valentina Corbetta
System architecture
Surgeon
Dataset
Kinematic
constraints
Parameters and
risk weights
EPs and TS
Manual
trajectories
System
Dataset 3D
visualisation
Hard constraints
Soft constraints
Best trajectory
Search space
definition
ASP trajectory
optimisation
Parameters
extraction
ASP trajectory
classification
12
3
4
5
6
7
8
9
10
10
Valentina Corbetta
Surgeon
Kinematic
constraints
Parameters and
risk weights
EPs and TS
Manual
trajectories
System
Hard constraints
Soft constraints
Search space
definition
ASP trajectory
optimisation
Parameters
extraction
ASP trajectory
classification
3
4
5
6
7
8
9
10
DatasetDataset 3D
visualisation
12
12
Optimisation and classificationSystem architecture
10
Valentina Corbetta
Surgeon
EPs and TS
Manual
trajectories
System
Search space
definition
ASP trajectory
optimisation
Parameters
extraction
ASP trajectory
classification
3
4
5
6
7
8
9
10
DatasetDataset 3D
visualisation
12
12
Kinematic
constraints
Parameters and
risk weights
Hard constraints
Soft constraints
3
4
Optimisation and classificationSystem architecture
10
Valentina Corbetta
Surgeon
EPs and TS
Manual
trajectories
System
Search space
definition
ASP trajectory
optimisation
Parameters
extraction
ASP trajectory
classification
3
4
5
6
7
8
9
10
DatasetDataset 3D
visualisation
12
12
Kinematic
constraints
Parameters and
risk weights
Hard constraints
Soft constraints
3
4
Outer diameter
0.0014 mm-1
2.5 mm
Kinematic constraints
Rule weight CED
6
9
6
Wd_min
Wd_tot
Wc_max
Optimisation and classificationSystem architecture
Maximum curvature
10
Valentina Corbetta
Surgeon
Kinematic
constraints
Parameters and
risk weights
System
Hard constraints
Soft constraints
ASP trajectory
optimisation
Parameters
extraction
ASP trajectory
classification
3
4
5
6
7
8
9
10
DatasetDataset 3D
visualisation
12
12
5
6
Search space
definition
7
EPs and TS
Manual
trajectories
5
6
Optimisation and classificationSystem architecture
10
Valentina Corbetta
Surgeon
Kinematic
constraints
Parameters and
risk weights
System
Hard constraints
Soft constraints
ASP trajectory
optimisation
Parameters
extraction
ASP trajectory
classification
3
4
5
6
7
8
9
10
DatasetDataset 3D
visualisation
12
12
EPs and TS
Manual
trajectories
5
6
Search space
definition
7
Optimisation and classificationSystem architecture
Valentina Corbetta
Surgeon
Kinematic
constraints
Parameters and
risk weights
System
Hard constraints
Soft constraints
ASP trajectory
optimisation
Parameters
extraction
ASP trajectory
classification
3
4
5
6
7
8
9
10
DatasetDataset 3D
visualisation
12
12
EPs and TS
Manual
trajectories
5
6
Search space
definition
7● Set of concentric circles
with 0 ≤ r ≤ 5 mm
● tj points sampled on each
circle
● Range of radius and n.° of
circles specified by user
Optimisation and classificationSystem architecture
10
10
Valentina Corbetta
Surgeon
Kinematic
constraints
Parameters and
risk weights
System
Hard constraints
Soft constraints Parameters
extraction
ASP trajectory
classification
3
4
5
6
7
8
9
10
DatasetDataset 3D
visualisation
12
12
EPs and TS
Manual
trajectories
5
6
Search space
definition
7
ASP trajectory
optimisation
8
Optimisation and classificationSystem architecture
10
Valentina Corbetta
Surgeon
Kinematic
constraints
Parameters and
risk weights
System
Hard constraints
Soft constraints Parameters
extraction
ASP trajectory
classification
3
4
5
6
7
8
9
10
DatasetDataset 3D
visualisation
12
12
EPs and TS
Manual
trajectories
5
6
Search space
definition
7
ASP trajectory
optimisation
8
● Search space encoded as
ASP problem
Example:
node(P1).arc(X,Y).
Optimisation and classificationSystem architecture
10
Valentina Corbetta
Surgeon
Kinematic
constraints
Parameters and
risk weights
System
Hard constraints
Soft constraints
ASP trajectory
optimisation
ASP trajectory
classification
3
4
5
6
7
8
9
10
DatasetDataset 3D
visualisation
12
12
EPs and TS
Manual
trajectories
5
6
Search space
definition
7
Parameters
extraction
9
Optimisation and classificationSystem architecture
9
10
Valentina Corbetta
Surgeon
Kinematic
constraints
Parameters and
risk weights
System
Hard constraints
Soft constraints
ASP trajectory
optimisation
ASP trajectory
classification
3
4
5
6
7
8
9
10
DatasetDataset 3D
visualisation
12
12
EPs and TS
Manual
trajectories
5
6
Search space
definition
7
Parameters
extraction
9
Minimum distance from obstacles
Average distance from obstacles
Total length of trajectory
Maximum curvature
99
Optimisation and classificationSystem architecture
10
Valentina Corbetta
Surgeon
Kinematic
constraints
Parameters and
risk weights
System
Hard constraints
Soft constraints
ASP trajectory
optimisation
Parameters
extraction
3
4
5
6
7
8
9
10
DatasetDataset 3D
visualisation
12
12
EPs and TS
Manual
trajectories
5
6
Search space
definition
7
ASP trajectory
classification
10
Optimisation and classificationSystem architecture
10
10
Valentina Corbetta
Surgeon
Kinematic
constraints
Parameters and
risk weights
System
Hard constraints
Soft constraints
ASP trajectory
optimisation
Parameters
extraction
3
4
5
6
7
8
9
10
DatasetDataset 3D
visualisation
12
12
EPs and TS
Manual
trajectories
5
6
Search space
definition
7
ASP trajectory
classification
10
Hard constraints
● cmax must be lower than Kmax
● dmin must be higher than r = OD/2
Example:
:- choose(X), radius(r), distObst(X,dmin), dmin<r.
10
Optimisation and classificationSystem architecture
10
Valentina Corbetta
Surgeon
Kinematic
constraints
Parameters and
risk weights
System
Hard constraints
Soft constraints
ASP trajectory
optimisation
Parameters
extraction
3
4
5
6
7
8
9
10
DatasetDataset 3D
visualisation
12
12
EPs and TS
Manual
trajectories
5
6
Search space
definition
7
ASP trajectory
classification
10
Soft constraints
● Minimisation of dtot
● Maximisation of dmin
● Minimisation of cmax
Example:
#maximize{dmin@wd_min,X:
choose(X), disObst(X,dmin)}.
10
Optimisation and classificationSystem architecture
10
Valentina Corbetta
Surgeon
Dataset
Kinematic
constraints
Parameters and
risk weights
EPs and TS
Manual
trajectories
System
Dataset 3D
visualisation
Hard constraints
Soft constraints
Search space
definition
ASP trajectory
optimisation
Parameters
extraction
ASP trajectory
classification
12
3
4
5
6
7
8
9
10
Optimisation and classificationSystem architecture
10
11
Valentina Corbetta
Experimental setup
● Modalities: manual vs
deductive reasoning
● 5 experiments for each
modality, 10 trajectories for
each experiment
Input:
● EP, TS
● OD, Kmax
● wd_min, wd_tot, wc_max
EXP1 EXPK...
Manual EXPK
ManualTK...ManualT1
Manual ...
Manual EXP1
T1 ... Tj
Visual
ASP EXPK
ASPTK...ASPT1
ASP ...
ASP EXP1
T1 ... Tj
Classification
Optimisation and classification
Manual trajectories designed
by expert neurosurgeon from
the Oncology and Emato-
oncology department of
Università degli Studi di Milano
12
Results Valentina Corbetta
Segato, A., Corbetta, V., Zangari, J., Calimeri, F., & De Momi, E. (2021, June). Optimized 3D path planner for steerable catheters with deductive reasoning. In 2020 IEEE International Conference on Robotics and Automation.
p-value
dtot 0.043
cmax 0.002
dmin 4.3551e-05
davg 0.0031
statistically significant,
p<0.05
13
Conclusions and future work Valentina Corbetta
The proposed methodology implements a tool that can assist the neurosurgeon in the pre-operative phase for path-planning of a steerable needle
Obstacle clearanceCustomisable
preferences
Respect of
kinematic constraints
Explicit representation
of domain knowledge
Future work
● Application to Deep Brain Stimulation
● Application to other fields other than neurosurgery
● Integration with other path planning methods
Thanks for your attention
Questions?
Valentina Corbetta
Results: ASP vs A*● Comparison between A*
and ASP
● 10 experiments with a
couple of randomly
selected EP and TS, within a
maximum distance of 10
voxels
Input: EP, TS
EXP1 EXPK...
ASP
ASP
EXPK
...
ASP EXP1
A* EXPk
A* ...
A* EXP1
ASPTK...ASPT1A*TK...A*T1
Average n.° of steps
Computational time [s]
ASP 7.4 2043.01
A* 7.4 0.002
Path planning
15
16
Valentina Corbetta
Integration of Unity and ASPOptimisation and classification
C#
Simulator
Parameters extraction
d_min d_tot c_maxd_avg
Write Clingo files
Invoke Python script
Python
Define search space
Invoke Clyngor
Clyngor
Run Clingo
Output solutions