Real-Time Data Acquisition for Cardiovascular ResearchReal-Time Data Acquisition for Cardiovascular...
Transcript of Real-Time Data Acquisition for Cardiovascular ResearchReal-Time Data Acquisition for Cardiovascular...
![Page 1: Real-Time Data Acquisition for Cardiovascular ResearchReal-Time Data Acquisition for Cardiovascular Research Andras Lasso, PhD . Laboratory for Percutaneous Surgery . School of Computing,](https://reader034.fdocuments.net/reader034/viewer/2022042314/5f0225157e708231d402cb35/html5/thumbnails/1.jpg)
Real-Time Data Acquisition for Cardiovascular Research
Andras Lasso, PhD
Laboratory for Percutaneous Surgery School of Computing, Queen’s University, Kingston
![Page 2: Real-Time Data Acquisition for Cardiovascular ResearchReal-Time Data Acquisition for Cardiovascular Research Andras Lasso, PhD . Laboratory for Percutaneous Surgery . School of Computing,](https://reader034.fdocuments.net/reader034/viewer/2022042314/5f0225157e708231d402cb35/html5/thumbnails/2.jpg)
Questions / challenges
• What hardware/software interfaces to use? • How to calibrate the system? • How to implement basic data visualization
and processing?
- 2 - Laboratory for Percutaneous Surgery (The Perk Lab) – Copyright © Queen’s University, 2011
![Page 3: Real-Time Data Acquisition for Cardiovascular ResearchReal-Time Data Acquisition for Cardiovascular Research Andras Lasso, PhD . Laboratory for Percutaneous Surgery . School of Computing,](https://reader034.fdocuments.net/reader034/viewer/2022042314/5f0225157e708231d402cb35/html5/thumbnails/3.jpg)
- 3 - Laboratory for Percutaneous Surgery (The Perk Lab) – Copyright © Queen’s University, 2011
Tracking data acquisition
Image acquisition
Data fusion
Physio signal acquisition
Spatial calibration
Temporal calibration Recording
Volume reconstruction
Visualization
Spatial registration Live streaming
Pre-procedural data import
Typical intra-procedural data acquisition and processing workflow
![Page 4: Real-Time Data Acquisition for Cardiovascular ResearchReal-Time Data Acquisition for Cardiovascular Research Andras Lasso, PhD . Laboratory for Percutaneous Surgery . School of Computing,](https://reader034.fdocuments.net/reader034/viewer/2022042314/5f0225157e708231d402cb35/html5/thumbnails/4.jpg)
Software and hardware interfaces for data acquisition
- 4 - Laboratory for Percutaneous Surgery (The Perk Lab) – Copyright © Queen’s University, 2011
Examples Administrative workload for getting access
Performance limitations
Open interfaces
Framegrabber, attached external
sensor, DICOM Low High
Standard research interfaces
OpenIGTLink Medium Medium
Proprietary research interfaces
Proprietary hardware and/or software kit from
manufacturer
High Low
![Page 5: Real-Time Data Acquisition for Cardiovascular ResearchReal-Time Data Acquisition for Cardiovascular Research Andras Lasso, PhD . Laboratory for Percutaneous Surgery . School of Computing,](https://reader034.fdocuments.net/reader034/viewer/2022042314/5f0225157e708231d402cb35/html5/thumbnails/5.jpg)
Acquisition of X-ray fluoroscopy image data
- 5 - Laboratory for Percutaneous Surgery (The Perk Lab) – Copyright © Queen’s University, 2011
OR display
Imaging parameters Filtration & exposure
management
Noise filtering, dynamic range management
X-ray generator
X-ray detector
Display filtering
DICOM network
Open interfaces
Framegrabber
DICOM receiver
Proprietary research interface
![Page 6: Real-Time Data Acquisition for Cardiovascular ResearchReal-Time Data Acquisition for Cardiovascular Research Andras Lasso, PhD . Laboratory for Percutaneous Surgery . School of Computing,](https://reader034.fdocuments.net/reader034/viewer/2022042314/5f0225157e708231d402cb35/html5/thumbnails/6.jpg)
Acquisition X-ray fluoroscopy image pose
- 6 - Laboratory for Percutaneous Surgery (The Perk Lab) – Copyright © Queen’s University, 2011
Open interfaces
Built-in encoders in table and gantry 3 machine angles
Source to image distance Table position
Proprietary research interface
Tilt sensor mounted on gantry
Small, simple, inexpensive, but orientation only
Optical pose tracker mounted on table/gantry
Provides full pose, but line of sight issues
Fiducial in the field of view Provides full pose, but limited operation range
![Page 7: Real-Time Data Acquisition for Cardiovascular ResearchReal-Time Data Acquisition for Cardiovascular Research Andras Lasso, PhD . Laboratory for Percutaneous Surgery . School of Computing,](https://reader034.fdocuments.net/reader034/viewer/2022042314/5f0225157e708231d402cb35/html5/thumbnails/7.jpg)
Acquisition of tracked ultrasound images and tool positions
- 7 - Laboratory for Percutaneous Surgery (The Perk Lab) – Copyright © Queen’s University, 2011
Proprietary research interface
B-mode and RF image data Imaging parameters
Framegrabber B-mode image data
Open interfaces Pose tracker (built into or attached to
transducer and tools)
Pose tracker (built into or attached to transducer and tools)
Analog signal DAQ device
Physio signals
Character recognition Imaging parameters
![Page 8: Real-Time Data Acquisition for Cardiovascular ResearchReal-Time Data Acquisition for Cardiovascular Research Andras Lasso, PhD . Laboratory for Percutaneous Surgery . School of Computing,](https://reader034.fdocuments.net/reader034/viewer/2022042314/5f0225157e708231d402cb35/html5/thumbnails/8.jpg)
Acquiring images and tracking data using a standard research interface
- 8 - Laboratory for Percutaneous Surgery (The Perk Lab) – Copyright © Queen’s University, 2011
OpenIGTLink: • Standard, open interface,
developed for image-guided therapy research
• Supports real-time streaming of image, pose, and custom data
• Based on TCP/IP protocol • Very simple and efficient • Supported by several device
manufacturers and open-source software packages
http://www.na-mic.org/Wiki/index.php/OpenIGTLink
Siemens MRI scanner
Real-time images Scan plane pose control
BrainLab pose tracker
Standard research interface
Tool positions Stored images
OpenIGTLink
OpenIGTLink
![Page 9: Real-Time Data Acquisition for Cardiovascular ResearchReal-Time Data Acquisition for Cardiovascular Research Andras Lasso, PhD . Laboratory for Percutaneous Surgery . School of Computing,](https://reader034.fdocuments.net/reader034/viewer/2022042314/5f0225157e708231d402cb35/html5/thumbnails/9.jpg)
PLUS • Public software Library for
Ultrasound imaging • Developed at the Laboratory
for Percutaneous Surgery • Primarily for ultrasound, but
applicable to other imaging modalities
• Unified interface to a wide variety of imaging and tracking devices
• Calibration, data processing, and streaming functionalities
• Free, BSD license • Released in Oct 2011, increasing number of users
- 9 - Laboratory for Percutaneous Surgery (The Perk Lab) – Copyright © Queen’s University, 2011
![Page 10: Real-Time Data Acquisition for Cardiovascular ResearchReal-Time Data Acquisition for Cardiovascular Research Andras Lasso, PhD . Laboratory for Percutaneous Surgery . School of Computing,](https://reader034.fdocuments.net/reader034/viewer/2022042314/5f0225157e708231d402cb35/html5/thumbnails/10.jpg)
3D Slicer • Medical image visualization
and analysis application • For translational research • Free, BSD license • Multi-platform (Windows,
Linux, MacOS) • Uses VTK, ITK, QT, DCMTK
- 10 - Laboratory for Percutaneous Surgery (The Perk Lab) – Copyright © Queen’s University, 2011
• Generic framework, with plug-ins for specialization • Large world-wide developer and user community • User and developer support, extensive testing, training courses,
documentation, tutorials • Continuous improvement, now in its 4th generation: some
remaining issues, performance optimizations – by Sep. 1, 2012
![Page 11: Real-Time Data Acquisition for Cardiovascular ResearchReal-Time Data Acquisition for Cardiovascular Research Andras Lasso, PhD . Laboratory for Percutaneous Surgery . School of Computing,](https://reader034.fdocuments.net/reader034/viewer/2022042314/5f0225157e708231d402cb35/html5/thumbnails/11.jpg)
Using PLUS and 3D Slicer for data acquisition
- 11 - Laboratory for Percutaneous Surgery (The Perk Lab) – Copyright © Queen’s University, 2011
PLUS toolkit www.assembla.com/spaces/plus
3D Slicer www.slicer.org
Open interfaces
• Framegrabbers (Epiphan, ImagingControls, Video for Windows)
• NDI Aurora, Polaris, Certus trackers • Claron MicronTracker • Ascension trakSTAR, medSAFE trackers • Phidget orientation sensor
• DICOM
Standard research interfaces
• OpenIGTLink (send/receive)
• OpenIGTLink (send/receive)
Proprietary research interfaces
• Ultrasonix ultrasound scanners (Ulterius and Porta SDK)
• BK Medical ultrasound scanners
![Page 12: Real-Time Data Acquisition for Cardiovascular ResearchReal-Time Data Acquisition for Cardiovascular Research Andras Lasso, PhD . Laboratory for Percutaneous Surgery . School of Computing,](https://reader034.fdocuments.net/reader034/viewer/2022042314/5f0225157e708231d402cb35/html5/thumbnails/12.jpg)
- 12 - Laboratory for Percutaneous Surgery (The Perk Lab) – Copyright © Queen’s University, 2011
Tracking data acquisition
Image acquisition
Data fusion
Spatial calibration
Temporal calibration Recording
Volume reconstruction
Visualization
Spatial registration Live streaming
Pre-procedural data import
Physio signal acquisition
Typical intra-procedural data acquisition and processing workflow
![Page 13: Real-Time Data Acquisition for Cardiovascular ResearchReal-Time Data Acquisition for Cardiovascular Research Andras Lasso, PhD . Laboratory for Percutaneous Surgery . School of Computing,](https://reader034.fdocuments.net/reader034/viewer/2022042314/5f0225157e708231d402cb35/html5/thumbnails/13.jpg)
Spatial calibration
- 13 - Laboratory for Percutaneous Surgery (The Perk Lab) – Copyright © Queen’s University, 2011
Goal: determine pose of images and tools relative to tracked markers
![Page 14: Real-Time Data Acquisition for Cardiovascular ResearchReal-Time Data Acquisition for Cardiovascular Research Andras Lasso, PhD . Laboratory for Percutaneous Surgery . School of Computing,](https://reader034.fdocuments.net/reader034/viewer/2022042314/5f0225157e708231d402cb35/html5/thumbnails/14.jpg)
Spatial calibration methods
- 14 - Laboratory for Percutaneous Surgery (The Perk Lab) – Copyright © Queen’s University, 2011
Inputs Result
Pivot calibration • Pose of tracked marker attached to a tool Tooltip position relative to tracked marker
Landmark registration
• Point positions from the tracking system • Point positions in the tool model
Tool pose relative to tracked marker
Z-frame-based image registration
• Pose of tracked marker attached to imaging device
• Images produced by the device • Z fiducial positions in the tool model
Image pose relative to tracked marker
Intensity-based image registration
• Two images showing the same object in different coordinate systems
Relative pose of the coordinate systems
Manual registration
• Images and/or surface models of the same object in different coordinate systems
• User input for aligning the objects
Relative pose of the coordinate systems
![Page 15: Real-Time Data Acquisition for Cardiovascular ResearchReal-Time Data Acquisition for Cardiovascular Research Andras Lasso, PhD . Laboratory for Percutaneous Surgery . School of Computing,](https://reader034.fdocuments.net/reader034/viewer/2022042314/5f0225157e708231d402cb35/html5/thumbnails/15.jpg)
Temporal calibration
- 15 - Laboratory for Percutaneous Surgery (The Perk Lab) – Copyright © Queen’s University, 2011
• Goal: determine time offset between data streams • Extract the same information (e.g., position) and find the time
offset that leads to maximum correlation of the signals
• Use accurate clock (usually ∼1ms is achievable) • Timestamp filtering: detect delayed data, reduce jitter • Limit maximum speed
![Page 16: Real-Time Data Acquisition for Cardiovascular ResearchReal-Time Data Acquisition for Cardiovascular Research Andras Lasso, PhD . Laboratory for Percutaneous Surgery . School of Computing,](https://reader034.fdocuments.net/reader034/viewer/2022042314/5f0225157e708231d402cb35/html5/thumbnails/16.jpg)
Using PLUS and 3D Slicer for system calibration
- 16 - Laboratory for Percutaneous Surgery (The Perk Lab) – Copyright © Queen’s University, 2011
PLUS toolkit www.assembla.com/spaces/plus
3D Slicer www.slicer.org
Spatial calibration
• Pivot calibration • Landmark registration • Z-frame-based image registration
• Landmark registration • Intensity-based image
registration • Manual registration
Temporal calibration
• Correlation-based temporal calibration of ultrasound image and pose
All these features are readily usable without the need for any additional software development.
![Page 17: Real-Time Data Acquisition for Cardiovascular ResearchReal-Time Data Acquisition for Cardiovascular Research Andras Lasso, PhD . Laboratory for Percutaneous Surgery . School of Computing,](https://reader034.fdocuments.net/reader034/viewer/2022042314/5f0225157e708231d402cb35/html5/thumbnails/17.jpg)
- 17 - Laboratory for Percutaneous Surgery (The Perk Lab) – Copyright © Queen’s University, 2011
Tracking data acquisition
Image acquisition
Data fusion
Spatial calibration
Temporal calibration Recording
Volume reconstruction
Visualization
Spatial registration Live streaming
Pre-procedural data import
Physio signal acquisition
Typical intra-procedural data acquisition and processing workflow
![Page 18: Real-Time Data Acquisition for Cardiovascular ResearchReal-Time Data Acquisition for Cardiovascular Research Andras Lasso, PhD . Laboratory for Percutaneous Surgery . School of Computing,](https://reader034.fdocuments.net/reader034/viewer/2022042314/5f0225157e708231d402cb35/html5/thumbnails/18.jpg)
Using PLUS and 3D Slicer for basic visualization and processing
- 18 - Laboratory for Percutaneous Surgery (The Perk Lab) – Copyright © Queen’s University, 2011
PLUS toolkit www.assembla.com/spaces/plus
3D Slicer www.slicer.org
Processing • Data recording • Volume reconstruction from
slices
• Segmentation • Registration • Measurements • … many more
Visualization
• 2D/3D visualization: slice, surface, volume rendering
• Image fusion • Real-time image and tool
display • … many more
All these features are readily usable without the need for any additional software development.
![Page 19: Real-Time Data Acquisition for Cardiovascular ResearchReal-Time Data Acquisition for Cardiovascular Research Andras Lasso, PhD . Laboratory for Percutaneous Surgery . School of Computing,](https://reader034.fdocuments.net/reader034/viewer/2022042314/5f0225157e708231d402cb35/html5/thumbnails/19.jpg)
Summary • Use standard research interfaces (OpenIGTLink) • Use free open-source tools
– Conversion from proprietary interface to unified, standard interfaces
– System calibration – Visualization and processing
- 19 - Laboratory for Percutaneous Surgery (The Perk Lab) – Copyright © Queen’s University, 2011
PLUS: www.assembla.com/spaces/plus SlicerIGT: www.assembla.com/spaces/slicerigt
3D Slicer: www.slicer.org
Andras Lasso ([email protected])
![Page 20: Real-Time Data Acquisition for Cardiovascular ResearchReal-Time Data Acquisition for Cardiovascular Research Andras Lasso, PhD . Laboratory for Percutaneous Surgery . School of Computing,](https://reader034.fdocuments.net/reader034/viewer/2022042314/5f0225157e708231d402cb35/html5/thumbnails/20.jpg)
- 20 - Laboratory for Percutaneous Surgery (The Perk Lab) – Copyright © Queen’s University, 2011
PlusServer Live streaming of
tracked image data
Tracking device (orientation sensor)
3D Slicer
OpenIGTLink module
Demonstration
Pre-procedural
image (cardiac CT)
PLUS
3D Slicer + SlicerIGT
C-arm model
OpenIGTLink
![Page 21: Real-Time Data Acquisition for Cardiovascular ResearchReal-Time Data Acquisition for Cardiovascular Research Andras Lasso, PhD . Laboratory for Percutaneous Surgery . School of Computing,](https://reader034.fdocuments.net/reader034/viewer/2022042314/5f0225157e708231d402cb35/html5/thumbnails/21.jpg)
Appendix
![Page 22: Real-Time Data Acquisition for Cardiovascular ResearchReal-Time Data Acquisition for Cardiovascular Research Andras Lasso, PhD . Laboratory for Percutaneous Surgery . School of Computing,](https://reader034.fdocuments.net/reader034/viewer/2022042314/5f0225157e708231d402cb35/html5/thumbnails/22.jpg)
Software process • Source control, tickets, releases, messaging
(www.assembla.com/spaces/plus) • Standardized build environment (CMake automatically
downloads and configures all required software components • Documentation: wiki, doxygen • Automatic tests: CTest, CDash, Sikuli
- 22 - Laboratory for Percutaneous Surgery (The Perk Lab) – Copyright © Queen’s University, 2011
![Page 23: Real-Time Data Acquisition for Cardiovascular ResearchReal-Time Data Acquisition for Cardiovascular Research Andras Lasso, PhD . Laboratory for Percutaneous Surgery . School of Computing,](https://reader034.fdocuments.net/reader034/viewer/2022042314/5f0225157e708231d402cb35/html5/thumbnails/23.jpg)
Spatial calibration
- 23 - Laboratory for Percutaneous Surgery (The Perk Lab) – Copyright © Queen’s University, 2011
Tutorials with all data, models, tricks • Performing tracked ultrasound probe calibration using fCal • How to build an fCal calibration phantom
![Page 24: Real-Time Data Acquisition for Cardiovascular ResearchReal-Time Data Acquisition for Cardiovascular Research Andras Lasso, PhD . Laboratory for Percutaneous Surgery . School of Computing,](https://reader034.fdocuments.net/reader034/viewer/2022042314/5f0225157e708231d402cb35/html5/thumbnails/24.jpg)
• Extension to the Meta IO standard file format • Slices readable by many existing applications • Extra information for tracking/reconstruction
– Frame number – Unfiltered and filtered timestamp – Probe and Reference tracking transforms – …
• Used by all applications of Plus
Sequence metafile (.mha)
- 24 - Laboratory for Percutaneous Surgery (The Perk Lab) – Copyright © Queen’s University, 2011