Color correction of pathological images

*Tokiya Abe ,Pinky A. Bautista, Yukako Yagi, John Gilbertson

**Yuri Murakami, Masahiro Yamaguchi , Nagaaki Ohyama

***Hideaki Haneishi * Department of Pathology, Harvard Medical School

**Imaging Science & Engineering Laboratory, Tokyo Institute of Technology

***Research Center for Frontier Medical Engineering, Chiba University

Why the color of these images is different?Part 1

These images are generated from one H&E slide

These differences are due to the different optical characteristic of the 3 imaging systems

Imaging System A Imaging System B Imaging System C

If a pathologist who is familiar with images of system A, but uses H&E image from system C, he has to calibrate the color of the image in his brain.

These images are generated from one imaging system

These differences are due to different staining condition of 3 H&E slides

Staining Condition A Staining Condition B Staining Condition C

If pathologist uses H&E image with staining condition C,

It might be difficult for him to perform accurate diagnosis.

Why the color of these images is different?Part 2

Digital Imaging in Pathology

• Color variability in digital slide images can be stressful for pathologists performing diagnosis.

• Therefore, color correction for digital slide images is important.

color correction based on 16-band imaging system

• We have reported color correction of H&E stained images by using 16-band camera*.

Although 16 band is ideal, but RGB imaging is widely used….

This color correction method can digitally correct staining condition by adjusting dye amount.

16 bands multispectral microscope system

Staining Condition A Staining Condition B

(Digital Correction)

*T. Abe et al “Color Standardization of pathological images” in 2003 APIII

Objective

Investigate the accuracy of color correction for H&E stained images using RGB with respect to 16-band.

Simulation of Color Correction for RGB

Simulation of RGB Image

H&E stained tissue slideglass slide

H&E Stained tissue

Investigate the accuracy of color correction using different Wiener Matrices

Correction of dye images

Generation of Spectral transmittance image

Decomposition to dye images

Reconstruction of spectral transmittance image

Generation of color corrected image

Spectral Estimation by Wiener estimation

16 bands Image Acquisition

Experiment

• Single Wiener matrix for all H&E slides

• Multiple Wiener matrices corresponding to different staining condition

Spectral Estimation by using Wiener Estimation

WienerEstimation

Matrix

B G R

B G R Spectral transmittance

Spectral transmittance

Many types of spectral transmittance

Wiener estimation matrix requires many types of spectral transmittance of H&E slide

It is difficult to estimate spectral transmittance from RGB signal

H&E Stained Slides with different staining conditions

Eosi

n st

aini

ng ti

me

Five slides of H&E stained liver specimens were prepared under different staining conditions

Excess E Over

Excess H

Normal

Hematoxylin staining time

Under

When “normal” is defined as target slide with optimal staining condition, the color of four slides are corrected into “normal”

Normal slide is stained optimally in hospital

Normal and Under stained slideNormal(Target)

Under(test)

The image labeled as under and normal correspond to tissue slide that are physically stained different staining time. By using color correction method, the color of under stained image can be corrected to have the color of the normal stained image.

Result

Single Wiener matrix with RGBit is difficult to see that cytoplasm and RBC are stained with eosin.

Single Wiener Matrix Single Wiener MatrixSingle Wiener Matrix Multiple Wiener Matrix

RGB 16 bands (ideal) RGB

Multiple Wiener matrix with RGBit is easy to see that cytoplasm and RBC are stained with eosin.

Because their color is not pink.

The contrast among nucleus and RBC and cytoplasm is a lot betterThis result is closer to the ideal result generated from 16 band image

Accuracy of RGB color correction

Estimation of Spectral Transmittance

Estimation of Amount of Dye Color Correction

Single Wiener matrix

NRMSE of transmittance NRMSE of dye Color difference

Multiple Wiener Matrix

The 16-band result is the reference;

Graphs represent the average difference between 16-band and RGB for all staining conditions

Conclusion• We investigated the accuracy of color correction for H&E

stained images using RGB with respect to 16 band• With Wiener matrix corresponding to staining condition,

the color corrected image by using RGB is close to that one by using 16 bands image in previous method

Future Work• Automatically selection of Wiener matrix• Application of color correction method for Whole slide Image

Acknowledgment

• We would like to thank CAP !!