Image Processing with KNIME · The “Zoo” of Image Processing Tools Development ImgLib OpenCV...
84
Image Processing with KNIME
Transcript of Image Processing with KNIME · The “Zoo” of Image Processing Tools Development ImgLib OpenCV...
Image Processing with KNIME
Who we are?!
Martin Horn
(+49) 07531 88-5017
Z815
Active Segmentation
Christian Dietz
(+49) 07531 88-3641
Z815
Active Classification
KNIME Developer Training 3
1. Big Picture –
Scientific Image Processing
2. KNIME Image Processing (KNIP) –
The Plugin
KNIME Developer Training 4
The “Zoo” of Image Processing Tools
Development
ImgLib
OpenCV
MatLab
NumPy
VTK
VIGRA
…
Processing UI
ImageJ
KNIP
Fiji
CellProfiler
Ilastik
CellCognition
Icy
Photoshop
…
Handling
OMERO
BioFormats
…
Single, individual, case specific, incompatible solutions
=
KNIME Developer Training 5
The “Zoo” of Image Processing Tools
• Which combination should I use?
• Reusability?
• Interoperability?
Many wheels out there…
KNIME Developer Training 6
The “Zoo” of Image Processing Tools
Development
ImgLib
OpenCV
MatLab
NumPy
VTK
VIGRA
…
Processing UI
ImageJ2/Fiji2
KNIP
CellProfiler
Icy
Ilastik
CellCognition
Photoshop
…
Handling
OMERO
BioFormats
…
Standardized Java Framework for Scientific Image Processing
=
SciJava – Overview
KNIME Developer Training 7
KNIP (Konstanz)
ImgLib (MPI Dresden) BioFormats
(Madison)
Fiji/ImageJ
(Madison)
OMERO
(Dundee)
SciJava
ImgLib2 • Developed at MPI-CBG Dresden
• Generic Framework for data (image) processing
algorithms and data-structures
KNIME Developer Training 8
SciJava – ImageJ
ImageJ • Wayne Rasband (National Institutes of Health)
• Popular, highly interactive image processing program
• Many plugins available
KNIME Developer Training 9
SciJava – ImageJ/FIJI
FIJI • Extension of ImageJ1 with plugin-update mechanism
ImageJ2 • New Version of ImageJ based on ImgLib2
• IJ1 plugins still work
KNIME Developer Training 10
SciJava - OME
Open Microscopy Environment • Joint project Dundee, Baltimore, Harvard Medical School
and LOCI
• Open tools to support data management for biological
light microscopy
• Standardized file-format (OME-XML)
KNIME Developer Training 11
SciJava - OME
OMERO • Tools for storing (database), visualizing, managing and
annotating images and metadata
BioFormats • Library for reading and writing > 120 microscopy file
formats
KNIME Developer Training 12
SciJava - KNIP
Basic information • KNIME Image Processing
• Basic data structures: ImgLib2
• High-throughput screening
• Fast prototyping
• Understandable workflows
KNIME Developer Training 13
KNIME Image Processing (KNIP)
Who? • Martin Horn (University of Konstanz)
• Christian Dietz (University of Konstanz)
• Thorsten Rieß (University of Konstanz)
• Slawek Mazur (BioQuant Heidelberg)
• …
KNIME User Meeting 14
KNIME Image Processing (KNIP)
Who? … Students • Felix Schoenenberger (University of Konstanz)
• Clemens Muething (University of Konstanz)
• Jan-Dirk Verbeek (University of Konstanz)
• Jens Metzner (University of Konstanz)
• Maximilian Ortwein (University of Konstanz)
• … several others
KNIME User Meeting 15
Why Image Processing with KNIP?
• Analysis of huge image data sets (HCS)
• Further analysis of the data
Machine learning
Visualization
Statistics
… everything which comes with KNIME
KNIME User Meeting 16
Image IO in KNIP?
Image Reader using Bio-Formats • 122 supported formats
OMERO Image Reader (experimental)
• Images from the OMERO image data base
Image Writer using Bio-Formats
• 11 supported formats (e.g. tif, jpeg, png, ome-
tiff,…)
KNIME User Meeting 17
KNIP - How images are handled?
KNIME User Meeting 18
KNIME User Meeting 19
KNIP - How images are handled?
KNIME User Meeting 20
KNIP - How images are handled?
KNIME User Meeting 21
What’s an image in KNIP?
KNIME User Meeting 22
What’s an image in KNIP?
KNIME User Meeting 23
What’s an image in KNIP?
KNIME User Meeting 24
…
What’s an image in KNIP?
KNIME User Meeting 25
time
…
KNIP - How to inspect images?
KNIME User Meeting 26
KNIME User Meeting 27
KNIP
How to inspect images?
KNIME User Meeting 28
KNIP
How to inspect images?
KNIP - How to process images?
KNIME User Meeting 29
KNIP - How to process images?
KNIME User Meeting 30
KNIP - How to process images?
KNIME User Meeting 31
KNIP - How to process images?
KNIME User Meeting 32
KNIP - How to process images?
KNIME User Meeting 33
KNIP - How to process images?
KNIME User Meeting 34
KNIP - How to segment images?
KNIME User Meeting 35
KNIME User Meeting 36
How to represent region of interests?
KNIME User Meeting 37
KNIP - Feature calculation
KNIME User Meeting 38
KNIME User Meeting 39
KNIP - Feature calculation
First order statistics • Mean, Standard Deviation, Skewness, …
Texture features • Haralick, Tamura, …
Geometric features • Circularity, Convexity, Centroid, Size, …
KNIME User Meeting 40
KNIP - Feature calculation
Shape Features • Fourier Descriptors, Radial Distances, …
Many under development • Bag of gradients, Histogram of Gradients, Zernike, …
KNIME User Meeting 41
KNIP – Feature calculation
KNIME User Meeting 42
KNIME User Meeting 43
KNIP
Feature calculation
KNIP - Feature calculation
KNIME User Meeting 44
KNIME User Meeting 45
KNIP – Segment Overlay
KNIME User Meeting 46
KNIME User Meeting 47
KNIP
KNIP – So what?
Some projects solved with KNIP • Mitotic Index
• Chromosome Counting
• Cell-Lifecycle Classification
• Membrane Breakdown (ETH - Zürich)
• Single Molecule Tracking
• DNA-Repair Measurement
• …. many more
KNIME User Meeting 48
KNIP – Chromosome Counting
KNIME User Meeting 49
KNIME User Meeting 50
Chromosome Counting
KNIME User Meeting 51
Chromosome Counting
KNIME User Meeting 52
Chromosome Counting
KNIME User Meeting 53
Chromosome Counting
KNIME User Meeting 54
Chromosome Counting
KNIME User Meeting 55
Chromosome Counting
KNIME User Meeting 56
Chromosome Counting
KNIME User Meeting 57
Chromosome Counting
KNIME User Meeting 58
Chromosome Counting
KNIME User Meeting 59
Chromosome Counting
KNIME User Meeting 60
Chromosome Counting
KNIME User Meeting 61
Chromosome Counting
KNIME User Meeting 62
Chromosome Counting
KNIME User Meeting 63
KNIP – Example
High-Content Screening
KNIME User Meeting 64
positive negative
High-Content Screening
KNIME User Meeting 65
High-Content Screening
KNIME User Meeting 66
High-Content Screening
KNIME User Meeting 67
High-Content Screening
KNIME User Meeting 68
High-Content Screening
KNIME User Meeting 69
KNIME User Meeting 70
Some more examples …
KNIME User Meeting 71
Mitosis Classification
KNIME User Meeting 72
Mitosis Classification
KNIME User Meeting 73
Mitotic Index
KNIME User Meeting 74
Mitotic Index
KNIME User Meeting 75
Mitotic Index
KNIME User Meeting 76
KNIP – Nice to know
Currently ~52 official nodes • IO nodes: ~ 4
• Image processing nodes ~ 25
• Filter nodes ~ 6
• Segmentation-related nodes ~ 10
• Feature calculation nodes ~ 3
• Viewer nodes ~ 4
KNIME User Meeting 77
KNIP – Nice to know
Sub-projects about to be released • VTK based 3D-Viewer
Sub-projects under development • Tracking framework
• IJ2 integration
• Pixel-Classification
• OMERO integration
KNIME User Meeting 78
KNIP – What’s next?
Further OMERO integration • Visit from OME-Team members early March
Data handling • Efficient data management
KNIME User Meeting 79
KNIP – What’s next?
ImageJ2 integration • High priority! Waiting for some changes in IJ2
Tracking framework • Builds up-on the network analysis framework
• Work in progress, some first results
KNIME User Meeting 80
KNIP – What’s next?
ImgLib2 synchronization • Synchronizing code for de/serialization
• Integrating KNIPLib algorithms to ImgLib2
• Refactoring KNIPLib algorithms
KNIME User Meeting 81
KNIP – Community Contributions
• Update site
• News, mailing list, SVN-Access
• Use the FORUM!
http://tech.knime.org/community/image-processing
KNIME User Meeting 82
KNIP – How to take part?
• We need feedback!
• We are open for any suggestions, help, feature requests,
bug reports…
KNIME User Meeting 83
KNIME User Meeting 84
Thank you…
