Knowledge Systems Lab JN 8/10/2002 Fusion of Multi-Modality Volumetric Medical Imagery Mario Aguilar...
-
Upload
calvin-jones -
Category
Documents
-
view
220 -
download
0
description
Transcript of Knowledge Systems Lab JN 8/10/2002 Fusion of Multi-Modality Volumetric Medical Imagery Mario Aguilar...
![Page 1: Knowledge Systems Lab JN 8/10/2002 Fusion of Multi-Modality Volumetric Medical Imagery Mario Aguilar and Joshua R. New Knowledge Systems Laboratory MCIS.](https://reader035.fdocuments.net/reader035/viewer/2022062311/5a4d1b877f8b9ab0599bd1a3/html5/thumbnails/1.jpg)
JN 8/10/2002 Knowledge Systems Lab
Fusion of Multi-Modality Volumetric Medical Imagery
Mario Aguilar and Joshua R. NewKnowledge Systems Laboratory
MCIS DepartmentJacksonville State University
Jacksonville, AL 36265
![Page 2: Knowledge Systems Lab JN 8/10/2002 Fusion of Multi-Modality Volumetric Medical Imagery Mario Aguilar and Joshua R. New Knowledge Systems Laboratory MCIS.](https://reader035.fdocuments.net/reader035/viewer/2022062311/5a4d1b877f8b9ab0599bd1a3/html5/thumbnails/2.jpg)
JN 8/10/2002 Knowledge Systems Lab
Outline
• What is the neurophysiological motivation of the fusion architecture?
• How is fusion extended to a three-dimensional operator?
• How is this operator applied to volumetric data sets?
• What are the results?
![Page 3: Knowledge Systems Lab JN 8/10/2002 Fusion of Multi-Modality Volumetric Medical Imagery Mario Aguilar and Joshua R. New Knowledge Systems Laboratory MCIS.](https://reader035.fdocuments.net/reader035/viewer/2022062311/5a4d1b877f8b9ab0599bd1a3/html5/thumbnails/3.jpg)
JN 8/10/2002 Knowledge Systems Lab
Neurophysiologically Motivated Architecture
![Page 4: Knowledge Systems Lab JN 8/10/2002 Fusion of Multi-Modality Volumetric Medical Imagery Mario Aguilar and Joshua R. New Knowledge Systems Laboratory MCIS.](https://reader035.fdocuments.net/reader035/viewer/2022062311/5a4d1b877f8b9ab0599bd1a3/html5/thumbnails/4.jpg)
JN 8/10/2002 Knowledge Systems Lab
Fusion Architecture
![Page 5: Knowledge Systems Lab JN 8/10/2002 Fusion of Multi-Modality Volumetric Medical Imagery Mario Aguilar and Joshua R. New Knowledge Systems Laboratory MCIS.](https://reader035.fdocuments.net/reader035/viewer/2022062311/5a4d1b877f8b9ab0599bd1a3/html5/thumbnails/5.jpg)
JN 8/10/2002 Knowledge Systems Lab
2D Image Fusion Example
GADPD
SPECTT2Color Fuse Result
![Page 6: Knowledge Systems Lab JN 8/10/2002 Fusion of Multi-Modality Volumetric Medical Imagery Mario Aguilar and Joshua R. New Knowledge Systems Laboratory MCIS.](https://reader035.fdocuments.net/reader035/viewer/2022062311/5a4d1b877f8b9ab0599bd1a3/html5/thumbnails/6.jpg)
2D Image Fusion Animation
![Page 7: Knowledge Systems Lab JN 8/10/2002 Fusion of Multi-Modality Volumetric Medical Imagery Mario Aguilar and Joshua R. New Knowledge Systems Laboratory MCIS.](https://reader035.fdocuments.net/reader035/viewer/2022062311/5a4d1b877f8b9ab0599bd1a3/html5/thumbnails/7.jpg)
JN 8/10/2002 Knowledge Systems Lab
Extensions to 3D
2D Fusion
ijS
sij
ijc
ijijij
IGxD
CIxBAxx
]*)[(
])[(
3D Fusion
Where:A – decay rateB – maximum activation level (set to 1)D – minimum activation level (set to 1)IC – excitatory inputIS – lateral inhibitory inputC and Gs are as follows:
2
22
2)(
2),(2
1
yx
yx eG
ijkS
sijk
ijkc
cijkijkijk
IGxD
IGCxBAxx
]*)[(
]*[)(
2
222
2)(
32),,(4
1
zyx
zyx eG
Where:A – decay rateB – maximum activation level (set to 1)D – minimum activation level (set to 1)IC – excitatory inputIS – lateral inhibitory inputC, Gc and Gs are as follows:
+-
2D Shunt Operator Symbol 3D Shunt Operator Symbol
![Page 8: Knowledge Systems Lab JN 8/10/2002 Fusion of Multi-Modality Volumetric Medical Imagery Mario Aguilar and Joshua R. New Knowledge Systems Laboratory MCIS.](https://reader035.fdocuments.net/reader035/viewer/2022062311/5a4d1b877f8b9ab0599bd1a3/html5/thumbnails/8.jpg)
JN 8/10/2002 Knowledge Systems Lab
3D Fusion Architecture
T1Images
T2Images
R
G
B
ColorRemap
Color Fuse
Result
Noise cleaning ®istration if needed
ContrastEnhancement
Between-band Fusionand Decorrelation
![Page 9: Knowledge Systems Lab JN 8/10/2002 Fusion of Multi-Modality Volumetric Medical Imagery Mario Aguilar and Joshua R. New Knowledge Systems Laboratory MCIS.](https://reader035.fdocuments.net/reader035/viewer/2022062311/5a4d1b877f8b9ab0599bd1a3/html5/thumbnails/9.jpg)
JN 8/10/2002 Knowledge Systems Lab
3D Shunt ResultsOriginal 2D Shunt 3D Shunt
![Page 10: Knowledge Systems Lab JN 8/10/2002 Fusion of Multi-Modality Volumetric Medical Imagery Mario Aguilar and Joshua R. New Knowledge Systems Laboratory MCIS.](https://reader035.fdocuments.net/reader035/viewer/2022062311/5a4d1b877f8b9ab0599bd1a3/html5/thumbnails/10.jpg)
JN 8/10/2002 Knowledge Systems Lab
3D Image Fusion Example
GADPD
SPECTT2
![Page 11: Knowledge Systems Lab JN 8/10/2002 Fusion of Multi-Modality Volumetric Medical Imagery Mario Aguilar and Joshua R. New Knowledge Systems Laboratory MCIS.](https://reader035.fdocuments.net/reader035/viewer/2022062311/5a4d1b877f8b9ab0599bd1a3/html5/thumbnails/11.jpg)
JN 8/10/2002 Knowledge Systems Lab
3D Fusion Results• 3D Explorer Views – 3D vs. 2D shunt
![Page 12: Knowledge Systems Lab JN 8/10/2002 Fusion of Multi-Modality Volumetric Medical Imagery Mario Aguilar and Joshua R. New Knowledge Systems Laboratory MCIS.](https://reader035.fdocuments.net/reader035/viewer/2022062311/5a4d1b877f8b9ab0599bd1a3/html5/thumbnails/12.jpg)
JN 8/10/2002 Knowledge Systems Lab
Conclusions
• Development of the 3D shunting operator is a natural extension of the 2D operator
• Application of the 3D shunting operator provides better definition of image details in volumetric data sets
• 3D fusion extensions developed in the context of Med-LIFE, a visualization and pattern recognition tool.
![Page 13: Knowledge Systems Lab JN 8/10/2002 Fusion of Multi-Modality Volumetric Medical Imagery Mario Aguilar and Joshua R. New Knowledge Systems Laboratory MCIS.](https://reader035.fdocuments.net/reader035/viewer/2022062311/5a4d1b877f8b9ab0599bd1a3/html5/thumbnails/13.jpg)
FUSION SYSTEM EXTENSION2D Fusion
ijS
sij
ijc
ijijij
IGxD
CIxBAxx
]*)[(
])[(
3D FusionShunting Neural Network Equation:
Where:A – decay rateB – maximum activation level (set to 1)D – minimum activation level (set to 1)IC – excitatory inputIS – lateral inhibitory inputC and Gs are as follows:
2
22
2)(
2),(2
1
yx
yx eG
ijkS
sijk
ijkc
cijkijkijk
IGxD
IGCxBAxx
]*)[(
]*[)(
2
222
2)(
32),,(4
1
zyx
zyx eG
Shunting Neural Network Equation:
Where:A – decay rateB – maximum activation level (set to 1)D – minimum activation level (set to 1)IC – excitatory inputIS – lateral inhibitory inputC, Gc and Gs are as follows:
+-
2D Shunt Operator Symbol 3D Shunt Operator Symbol
![Page 14: Knowledge Systems Lab JN 8/10/2002 Fusion of Multi-Modality Volumetric Medical Imagery Mario Aguilar and Joshua R. New Knowledge Systems Laboratory MCIS.](https://reader035.fdocuments.net/reader035/viewer/2022062311/5a4d1b877f8b9ab0599bd1a3/html5/thumbnails/14.jpg)
ResultsOriginal 2D Shunt 3D Shunt
![Page 15: Knowledge Systems Lab JN 8/10/2002 Fusion of Multi-Modality Volumetric Medical Imagery Mario Aguilar and Joshua R. New Knowledge Systems Laboratory MCIS.](https://reader035.fdocuments.net/reader035/viewer/2022062311/5a4d1b877f8b9ab0599bd1a3/html5/thumbnails/15.jpg)
JN 8/10/2002 Knowledge Systems Lab
AcknowledgementsThis work was supported by a Faculty Research Grant
awarded to the first author by the faculty research committee and Jacksonville State University. Opinions, interpretations, and conclusions are those of the authors and not necessarily endorsed by the committee or Jacksonville State University.
For additional information, please visit http://ksl.jsu.edu.