A Novel Image Registration Pipeline for 3-A Novel Image Registration Pipeline for 3-D Reconstruction from Microscopy ImagesD Reconstruction from Microscopy Images

Kun Huang, PhD; Ashish Sharma, PhD; Lee Cooper, MS;Kun Huang, PhD; Ashish Sharma, PhD; Lee Cooper, MS;Tony Pan, MS; Metin Gurcan, PhD; Joel Saltz, MD, PhDTony Pan, MS; Metin Gurcan, PhD; Joel Saltz, MD, PhD

Department of Biomedical InformaticsOhio State University

Creating Geometry from Images

Placenta

H+E Slides Alignment

SegmentationVisualization/Surface

Extraction

AperioScanner

Registration

Registration between different modalities (e.g, MRI and PET)

Mapping of different samples to the same reference (e.g., brain mapping)

3-D reconstruction

An optimization problem Initialization Point feature

matching Automatic vs.

manual

Issues with automatic registration

Initialization Landmark- or image- based? Linear or nonlinear? Error metric / Meaningful morphology /

Domain specific knowledge Computation Structural constraints

Fast initialization using landmarks

S1

Fast initialization

S2

S2’

S1’

S3’Matching pairs:

(S1, S1’) (S1, S3’) (S2, S2’)

S1

S2 S2’

S1’

S3’

d12

d12’d13’

Fast initialization

Maximum Clique Maximum Cyclic Structure

(S1, S1’, S2, S2’, θ, T)

Fast initialization

Difference between two images.

Difference after the automatic initialization using region features.

Difference after the MMI algorithm.

Fast initialization

Registration of Large Images Using Landmarks

Registration of Large Images Using High-Level Features

• No need to globally transform the image• Multi-level registration – rigid to nonrigid• Parallelizable – local operations

Registration of Large Images Using Landmarks

Registration of Large Images Using High-Level Features

• Point feature does NOT contain global information• For global transformation (e.g., rotation and

translation), we need “global” features such as high-level features.

• For nonlinear transformation, which is local, we need “local” features such as point features.

Global first, local second.

Stacks of microscopy images

Principal component analysis (PCA) – based rigid registration

Stacks of microscopy images

Stacks of microscopy images

Stacks of microscopy images

Stacks of microscopy images

Stacks of microscopy images

Stacks of microscopy images

3-D reconstruction vs. registration

• The current metric for registration is between two images and is just for the sake of perfect “registration”.

• We do “registration” for the sake of 3-D reconstruction.

• The structural constraint should be incorporated in the “cost function” instead of just used as a post processing or validation criterion.

New multiple image registration algorithm is needed!

3-D reconstruction via registration

Feature extraction

Feature matching/ tracking

Trajectory generation

Trajectory smoothing and

adjustment

New location for the features in every image

Nonlinear transformation

for every image

Collective adjustment of trajectories

3-D reconstruction via registration

Tracked trajectory

Smoothed trajectory

Registration moves the landmarks to the new locations.

3-D reconstruction via registration

3-D reconstruction via registration

3-D reconstruction via Registration

Summary, future work and discussion

• Technical issues related to automatic registration.• Two step approach to achieve “good” nonlinear

registration.• The paradigm for 3-D reconstruction is different with

pure registration.• New registration pipeline is proposed and implemented.

Summary, future work and discussion

• Parallelization – especially in nonlinear transformation stage.

• Multiresolution / hierarchical approach.

Acknowledgement

• BMI• Imaging group• Collaborators

Thank you !