Acknowledgements: Barry KalmanStan Kwasny Koong-Nah ChungJohn Heuser
description
Transcript of Acknowledgements: Barry KalmanStan Kwasny Koong-Nah ChungJohn Heuser
![Page 1: Acknowledgements: Barry KalmanStan Kwasny Koong-Nah ChungJohn Heuser](https://reader036.fdocuments.net/reader036/viewer/2022081603/56814d62550346895dbaafbe/html5/thumbnails/1.jpg)
Acknowledgements:
Barry Kalman Stan Kwasny
Koong-Nah Chung John Heuser
Detection of Intracellular Organelles in Digitized Electronmicroscopic Images Using Wavelets and Neural Networks
Final Report for CS513John Olsen and Wei Yan
![Page 2: Acknowledgements: Barry KalmanStan Kwasny Koong-Nah ChungJohn Heuser](https://reader036.fdocuments.net/reader036/viewer/2022081603/56814d62550346895dbaafbe/html5/thumbnails/2.jpg)
The Experiment
• We used Kalman & Kwasny’s neural network tool to create a set of trained feedforward neural networks (FFNNs) that detect particular intracellular organelles, called caveolae, in scanning EM images.
![Page 3: Acknowledgements: Barry KalmanStan Kwasny Koong-Nah ChungJohn Heuser](https://reader036.fdocuments.net/reader036/viewer/2022081603/56814d62550346895dbaafbe/html5/thumbnails/3.jpg)
![Page 4: Acknowledgements: Barry KalmanStan Kwasny Koong-Nah ChungJohn Heuser](https://reader036.fdocuments.net/reader036/viewer/2022081603/56814d62550346895dbaafbe/html5/thumbnails/4.jpg)
![Page 5: Acknowledgements: Barry KalmanStan Kwasny Koong-Nah ChungJohn Heuser](https://reader036.fdocuments.net/reader036/viewer/2022081603/56814d62550346895dbaafbe/html5/thumbnails/5.jpg)
![Page 6: Acknowledgements: Barry KalmanStan Kwasny Koong-Nah ChungJohn Heuser](https://reader036.fdocuments.net/reader036/viewer/2022081603/56814d62550346895dbaafbe/html5/thumbnails/6.jpg)
Overview of Experiment
Training Set
Test Set
32 N 32 P
99 N100 P
WaveletTransform
WaveletTransform
Unsupervised,Recurrent Net:LOSRAAM
Supervised,Feed-ForwardNeural Net
Unsupervised,Recurrent Net:LOSRAAM
5 TrainedFFNNs
Final DecisionBy Vote
![Page 7: Acknowledgements: Barry KalmanStan Kwasny Koong-Nah ChungJohn Heuser](https://reader036.fdocuments.net/reader036/viewer/2022081603/56814d62550346895dbaafbe/html5/thumbnails/7.jpg)
Performance is 84% correct
0
5
10
15
20
25
30
35
Test1 Test2 Test3 Test4 Total Perfect
Negatives
Positives
![Page 8: Acknowledgements: Barry KalmanStan Kwasny Koong-Nah ChungJohn Heuser](https://reader036.fdocuments.net/reader036/viewer/2022081603/56814d62550346895dbaafbe/html5/thumbnails/8.jpg)
Images
Components: Vectors of waveletcoeffs(resolution, dimension)
Vectors ofLOSRAAM StateTransitions
K Centers ofAttraction
K x K FuzzyTransitionMatrix
Hold-outSet
FuzzyFeatureVectors
Reduced,CombinedFFV
PACSet
TestSet
TrainedFFNNs
FinalDecision
Data Flow Diagram
![Page 9: Acknowledgements: Barry KalmanStan Kwasny Koong-Nah ChungJohn Heuser](https://reader036.fdocuments.net/reader036/viewer/2022081603/56814d62550346895dbaafbe/html5/thumbnails/9.jpg)
Wavelet Transform
• Equivalent to the 1st order derivative of a smoothing function.
• Analogous to the Fourier Transform.
• Multiple levels of Resolution.
• 2-Dimensions
• Image component: The set of wavelet coefficients for a particular dimension at a particular resolution.
• Image components: Multiple representations of the entire image at different resolutions.
![Page 10: Acknowledgements: Barry KalmanStan Kwasny Koong-Nah ChungJohn Heuser](https://reader036.fdocuments.net/reader036/viewer/2022081603/56814d62550346895dbaafbe/html5/thumbnails/10.jpg)
Image L1 L2 L3 L4
320 x 240 160 x 120 80 x 60 40 x 30 20 x 15
1 4 16 64 256
76,800 19,200 4,800 1200 300
- 2 2 2 3
- 38,400 9,600 2,400 900
Level
Image size(grains)
Grain size(pixels)
Total Grains
No. of Comps (x, y, r)
No. of Coeffs
Four-level, 2-D Wavelet Hierarchy
• Maximum no. of coeffs at any one level, direction = no of grains in image. For our data, the max no. of coefficients/image = 51, 300.
• A Component consists of the set of wavelet coefficients for a particular direction and level of resolution.
• Total of 9 components: 2 for each level of decomposition, 1 for undecomposed components at the last level (DC, very low frequency).
![Page 11: Acknowledgements: Barry KalmanStan Kwasny Koong-Nah ChungJohn Heuser](https://reader036.fdocuments.net/reader036/viewer/2022081603/56814d62550346895dbaafbe/html5/thumbnails/11.jpg)
Why Wavelets ?
• Enhance contrast edges.
• Multi-Resolution emphasizes features of different sizes.
• Data reduction from thresholding.
![Page 12: Acknowledgements: Barry KalmanStan Kwasny Koong-Nah ChungJohn Heuser](https://reader036.fdocuments.net/reader036/viewer/2022081603/56814d62550346895dbaafbe/html5/thumbnails/12.jpg)
Wavelet Coefficient Thresholding
• For all images, dropped coefficients with magnitudes < 0.4
• 16 million coefficients reduced to 1.6 million.
• Threshold determined by experience and experiment: Set of coefficients after thresholding reverse transformed to image, and this image is visually compared to the original.
![Page 13: Acknowledgements: Barry KalmanStan Kwasny Koong-Nah ChungJohn Heuser](https://reader036.fdocuments.net/reader036/viewer/2022081603/56814d62550346895dbaafbe/html5/thumbnails/13.jpg)
![Page 14: Acknowledgements: Barry KalmanStan Kwasny Koong-Nah ChungJohn Heuser](https://reader036.fdocuments.net/reader036/viewer/2022081603/56814d62550346895dbaafbe/html5/thumbnails/14.jpg)
LOSRAAM Training Set
• Each component is made a separate vector: 9 components per image * 199 images = 1791 components.
• 1.6 M coefficients distributed over 1791 components.
• Each coefficient represented by a doublet: [magnitude, normalized index into component array].
• Coefficients fed into LOSRAAM Neural Network one by one.
![Page 15: Acknowledgements: Barry KalmanStan Kwasny Koong-Nah ChungJohn Heuser](https://reader036.fdocuments.net/reader036/viewer/2022081603/56814d62550346895dbaafbe/html5/thumbnails/15.jpg)
The LOSRAAM ANN
• Linear Output Recurrent Recursive Auto-Associative Memory
• AAM: Targeting the outputs to be the same as the inputs.
• Recurrence: a portion of the input is the activation pattern from the hidden layer of the previous iteration.
• Unsupervised Learning: A criterion for judging outputs is determined. The ANN learns a mapping of inputs to outputs that fits the criterion.
![Page 16: Acknowledgements: Barry KalmanStan Kwasny Koong-Nah ChungJohn Heuser](https://reader036.fdocuments.net/reader036/viewer/2022081603/56814d62550346895dbaafbe/html5/thumbnails/16.jpg)
Output Units
Linear State (“Context”)
Input Units
HiddenUnits
Coefficient Index
6 input units 4 hidden units, determined by experiment 6 output units 10 bias units 74 adjustable weights, initially all 0. Recurrent connections from hidden units to
input state units
6-4-6 LOSRAAM Neural Network
![Page 17: Acknowledgements: Barry KalmanStan Kwasny Koong-Nah ChungJohn Heuser](https://reader036.fdocuments.net/reader036/viewer/2022081603/56814d62550346895dbaafbe/html5/thumbnails/17.jpg)
LOSRAAM Training Statistics
• Running time 55 hours.• Sun Sparc, 4 parallel processors.• Error function: initial value = 1,800,000
final value = 401.
![Page 18: Acknowledgements: Barry KalmanStan Kwasny Koong-Nah ChungJohn Heuser](https://reader036.fdocuments.net/reader036/viewer/2022081603/56814d62550346895dbaafbe/html5/thumbnails/18.jpg)
Clustering LOSRAAM State Vectors
• Image structure produces a sequence of activation patterns of the 4 unit hidden layer.
• Activation patterns represented by a sequence of vectors in 4-D space.
• The trajectory of these vectors is represented by a series of point values in 4-D space.
• These points are clustered into “centers of attraction”.
• Clustering is a “fuzzy” process.
• Our LOSRAAM data yielded 4 centers of attraction.
![Page 19: Acknowledgements: Barry KalmanStan Kwasny Koong-Nah ChungJohn Heuser](https://reader036.fdocuments.net/reader036/viewer/2022081603/56814d62550346895dbaafbe/html5/thumbnails/19.jpg)
Fuzzy Transition Matricesand Fuzzy Feature Vectors
• An FTM is a 4 x 4 matrix of transitions between centers of attraction.
• FTMs were computed for each component, for each image.
• Thus, each image is represented by 9 FTMs, each containing 16 elements, 144 elements total.
• Linearization of FTMs gives 144 element FFVs, one per image.
![Page 20: Acknowledgements: Barry KalmanStan Kwasny Koong-Nah ChungJohn Heuser](https://reader036.fdocuments.net/reader036/viewer/2022081603/56814d62550346895dbaafbe/html5/thumbnails/20.jpg)
Singular Valued Decomposition of FFVs
• SVD is a data reduction and conditioning technique.• Combine FFVs for all images into a 144 x 199 matrix.• Using SVD, identified 9 of the 144 columns that accounted
for > 99% of the variance in the entire training data set.• These 1791 values used to train the Feed Forward ANNs.
144 elements
199 FFVs 199 reduced FFVs
9 elements
![Page 21: Acknowledgements: Barry KalmanStan Kwasny Koong-Nah ChungJohn Heuser](https://reader036.fdocuments.net/reader036/viewer/2022081603/56814d62550346895dbaafbe/html5/thumbnails/21.jpg)
Feed-Forward Neural Network
• Supervised Learning: Input and output are provided. The FFNN learns mapping by example.
• We used a 9-1-1 FFNN.
![Page 22: Acknowledgements: Barry KalmanStan Kwasny Koong-Nah ChungJohn Heuser](https://reader036.fdocuments.net/reader036/viewer/2022081603/56814d62550346895dbaafbe/html5/thumbnails/22.jpg)
Image’s 9 element
FFV
Input Units
9-1-1 Feed-Forward Neural Network
Output Unit
Hidden UnitFully skip-connected
![Page 23: Acknowledgements: Barry KalmanStan Kwasny Koong-Nah ChungJohn Heuser](https://reader036.fdocuments.net/reader036/viewer/2022081603/56814d62550346895dbaafbe/html5/thumbnails/23.jpg)
FFNN Training
• Input 199 training patterns (9 values each), one at a time.• Periodically check performance with PAC set. (A set of 16
images, 8 positive, 8 negative).• Harvest weights if performance on training set and on PAC are
both > 85%.• If PAC test failed, weights are discarded.• Trained 5 FFNNs to criterion performance on training set and
PAC.
![Page 24: Acknowledgements: Barry KalmanStan Kwasny Koong-Nah ChungJohn Heuser](https://reader036.fdocuments.net/reader036/viewer/2022081603/56814d62550346895dbaafbe/html5/thumbnails/24.jpg)
FFNN Training Issues
• Overtraining. First time training with a two-hidden-unit FFNN led to 100% performance for both training and PAC, but only 65% performance on the test set.
• Attempt to raise harvest criterion to 88% for one-hidden-unit FFNN failed. No trained networks were produced.
![Page 25: Acknowledgements: Barry KalmanStan Kwasny Koong-Nah ChungJohn Heuser](https://reader036.fdocuments.net/reader036/viewer/2022081603/56814d62550346895dbaafbe/html5/thumbnails/25.jpg)
Images
Components: Vectors of waveletcoeffs(resolution, dimension)
Vectors ofLOSRAAM StateTransitions
K Centers ofAttraction
K x K FuzzyTransitionMatrix
Hold-outSet
FuzzyFeatureVectors
Reduced,CombinedFFV
PACSet
TestSet
TrainedFFNNs
FinalDecision
Data Flow Diagram
![Page 26: Acknowledgements: Barry KalmanStan Kwasny Koong-Nah ChungJohn Heuser](https://reader036.fdocuments.net/reader036/viewer/2022081603/56814d62550346895dbaafbe/html5/thumbnails/26.jpg)
Can performance be improved?
• Human performance ~ 100%.
• ANN must handle several different sources of variation: position, view angle, size, number, flatness.
• Increase the size and depth of the training set, e.g., 500 images.
• Add positional ‘hint’ units, number hint units, etc.
![Page 27: Acknowledgements: Barry KalmanStan Kwasny Koong-Nah ChungJohn Heuser](https://reader036.fdocuments.net/reader036/viewer/2022081603/56814d62550346895dbaafbe/html5/thumbnails/27.jpg)
Significance of Results to Kalman and Kwasny’s ANN
• Scanning EM images are rich in detail, perhaps more so than mammograms.
• Absolute performance on EM images is better than performance on mammograms, particularly with respect to specificity.
• The results suggest that the ANN is capable of using the rich detail found in EM images to achieve a low false positive rate.
Mammograms EMs
False Pos. 44% 14%
False Neg. 25% 14%
![Page 28: Acknowledgements: Barry KalmanStan Kwasny Koong-Nah ChungJohn Heuser](https://reader036.fdocuments.net/reader036/viewer/2022081603/56814d62550346895dbaafbe/html5/thumbnails/28.jpg)
![Page 29: Acknowledgements: Barry KalmanStan Kwasny Koong-Nah ChungJohn Heuser](https://reader036.fdocuments.net/reader036/viewer/2022081603/56814d62550346895dbaafbe/html5/thumbnails/29.jpg)
Potential Applications
• Automated scanning of EM images.• Large scale screening of histological slides for
abnormal cell morphologies.