Deep Learning of Tissue Specific Speckle Representations in Optical Coherence Tomography and Deeper...

Deep Learning of Tissue Specific Speckle

Representations in Optical Coherence

Tomography and Deeper Exploration for

In situ Histology

Debdoot Sheet

@ Department of Electrical Engineering, Indian Institute of Technology Kharagpur, India.

Sri Phani Krishna Karri, Jyotirmoy Chatterjee

@ School of Medical Science and Technology, Indian Institute of Technology Kharagpur, India

Amin Katouzian, Nassir Navab

@ Chair for Computer Aided Medical Procedures, TU Munich, Germany

Ajoy K. Ray

@ Electronics and Electrical Comm. Engg., Indian Institute of Technology Kharagpur, India.

1ISBI 2015 / FrDT3.5 - Deep Learning of Tissue Specific Speckle... - Debdoot Sheet

Motivation• Soft tissues – e.g. skin

– Epithelial

– Connective

– Muscular

– Adipose

• Pathological markers– Extracellular matrix deposition

– Cellular atypia and dysplasia

– Loss of histo-architecture

– Proliferative changes

• Conventional histology– Patient discomfort

– 48-72 hours delay in processing

• Alternatives – Subsurface imaging

• Optical coherence tomography (OCT)

• Challenges with the alternative– Hard to interpret

– Stochastic uncertainty of speckles

ISBI 2015 / FrDT3.5 - Deep Learning of Tissue Specific Speckle... - Debdoot Sheet 2

Epithelium, Papillary

dermis, Dermis, Adipose

Where do we stand now?


This Paper

Text books

R. K. Das (2012), PhD Thesis

A. Barui (2011), PhD Thesis

D. Sheet et.al., ISBI 2014

State of the Art• In situ Histology with OCT

– G. van Soest et al., (2010), G. J. Ughi et al., (2013) –Cardiovascular OCT

– D. Sheet et al., (2013, 2014) –Cutaneous wounds, oral

• Challenges– Heuristic features

• Texture

• Intensity statistics

– Heuristic computational models• Transfer learning of speckle

occurrence models

– Incomplete representation dictionary


Multi-scale

modeling of

OCT speckles

Training

image

set Ground

truth

Random forest

learning

Multi-scale

modeling of

OCT speckles

Test image

Labeled

tissue

Heuristics in State of Art


The Solution


Den

oisi

ngA

uto

Enc

oder

Den

oisi

ngA

uto

Enc

oder

Logi

stic

Reg

.

Unfurling the Deep Network


Learning of Representations


Representation of speckle

appearance models learned by DAE1

Learning of Representations


Sparsity of representations learned by

DAE2

Experiment Design• Data Collection

– School of Medical Science and Technology, Indian Institute of Technology Kharagpur

– 1300 nm (HPBW 100 nm) Swept Source OCT System • OCS 1300 SS, ThorLabs, NJ,

USA

• 8 bit bitmap images

– Histology for ground truth• HE stained

• Samples– Mus musculus (small mice)

– 16 healthy skin

– 2 wounds on skin

• DNN architecture– Patch size – 36 × 36 px

– DAE1 – 400 nodes

– DAE2 – 100 nodes

– Target – Logistic Reg. • 5 outputs

– Sparsity – 20%

– Mini-batch training

• In situ Histology Performance– Epithelium – 96%

– Papillary dermis – 93%

– Dermis – 99%

– Adipose tissue – 98%


Results in Wounds


(a) OCT image of wound (b) Ground truth (c) In situ histology





Take Home Message• Photons interact characteristically with different tissues.

– Stochastic similarity exists in speckle appearance.

– Such representations are hard to heuristically encode.

• Deep learning and auto-encoders for computational imaging– Speckle imaging application viz. OCT tissue characterization

– Hierarchical learning• Locally embedded representations.

• Sparsity is in learned (auto-encoded) representations.


Queries: Debdoot Sheet ([email protected])

Deep Learning of Tissue Specific Speckle Representations in Optical Coherence Tomography and Deeper...

Education

Transcript of Deep Learning of Tissue Specific Speckle Representations in Optical Coherence Tomography and Deeper...