Biomedical Optics Laboratory - Spectrally-encoded color imaging...Spectrally-encoded color imaging...

Spectrally-encoded color imaging

DongKyun Kang,1 Dvir Yelin,

2 Brett E. Bouma,

1 and Guillermo J. Tearney

1*

1 Harvard Medical School and Wellman center for Photomedicine, Massachusetts General Hospital, 55 Fruit Street, Boston, MA 02114, USA

2Department of Biomedical Engineering, Technion – Israel Institute of Technology, Haifa 32000, Israel *[email protected]

Abstract: Spectrally-encoded endoscopy (SEE) is a technique for ultraminiature endoscopy that encodes each spatial location on the sample with a different wavelength. One limitation of previous incarnations of SEE is that it inherently creates monochromatic images, since the spectral bandwidth is expended in the spatial encoding process. Here we present a spectrally-encoded imaging system that has color imaging capability. The new imaging system utilizes three distinct red, green, and blue spectral bands that are configured to illuminate the grating at different incident angles. By careful selection of the incident angles, the three spectral bands can be made to overlap on the sample. To demonstrate the method, a bench-top system was built, comprising a 2400-lpmm grating illuminated by three 525-µm-diameter beams with three different spectral bands. Each spectral band had a bandwidth of 75 nm, producing 189 resolvable points. A resolution target, color phantoms, and excised swine small intestine were imaged to validate the system’s performance. The color SEE system showed qualitatively and quantitatively similar color imaging performance to that of a conventional digital camera.

©2009 Optical Society of America

OCIS codes: (170.0170) Medical optics and biotechnology; (170.2150) Endoscopic imaging; (170.3880) Medical and biological imaging.

References and links

1. G. Berci, and J. Rozga, “Miniature laparoscopy: quo vadis? The basic parameters of image relay and display systems,” Surg. Endosc. 13(3), 211–217 (1999).

2. V. R. Jacobs, M. Kiechle, B. Plattner, T. Fischer, and S. Paepke, “Breast ductoscopy with a 0.55-mm mini-endoscope for direct visualization of intraductal lesions,” J. Minim. Invasive Gynecol. 12(4), 359–364 (2005).

3. F. F. Tu, and A. P. Advincula, “Miniaturizing the laparoscope: current applications of micro- and minilaparoscopy,” Int. J. Gynaecol. Obstet. 100(1), 94–98 (2008).

4. J. Chan, S. Kumar, and N. M. Fisk, “First trimester embryo-fetoscopic and ultrasound-guided fetal blood sampling for ex vivo viral transduction of cultured human fetal mesenchymal stem cells,” Hum. Reprod. 23(11), 2427–2437 (2008).

5. E. J. Seibel, and Q. Y. Smithwick, “Unique features of optical scanning, single fiber endoscopy,” Lasers Surg. Med. 30(3), 177–183 (2002).

6. E. J. Seibel, R. S. Johnston, and C. D. Melville, “A full-color scanning fiber endoscope,” in Optical Fibers and Sensors for Medical Diagnostics and Treatment Applications VI, (SPIE, 2006), 608303–608308.

7. G. J. Tearney, M. Shishkov, and B. E. Bouma, “Spectrally encoded miniature endoscopy,” Opt. Lett. 27(6), 412–414 (2002).

8. D. Yelin, B. E. Bouma, N. Iftimia, and G. J. Tearney, “Three-dimensional spectrally encoded imaging,” Opt. Lett. 28(23), 2321–2323 (2003).

9. D. Yelin, B. E. Bouma, J. J. Rosowsky, and G. J. Tearney, “Doppler imaging using spectrally-encoded endoscopy,” Opt. Express 16(19), 14836–14844 (2008).

10. D. Yelin, I. Rizvi, W. M. White, J. T. Motz, T. Hasan, B. E. Bouma, and G. J. Tearney, “Three-dimensional miniature endoscopy,” Nature 443(7113), 765 (2006).

11. D. Yelin, W. M. White, J. T. Motz, S. H. Yun, B. E. Bouma, and G. J. Tearney, “Spectral-domain spectrally-encoded endoscopy,” Opt. Express 15(5), 2432–2444 (2007).

12. M. Stokes, M. Anderson, S. Chandrasekar, and R. Motta, “A Standard Default Color Space for the Internet: sRGB” (1996), retrieved http://www.color.org/sRGB.xalter.

13. W. D. Wright, “A re-determination of the trichromatic coefficients of the spectral colours,” Trans. Opt. Soc. 30(4), 141–164 (1929).

14. J. Guild, “The Colorimetric Properties of the Spectrum,” Philos. Trans. R. Soc. Lond., A Contain. Pap. Math. Phys. Character 230(1), 149–187 (1932).

#112805 - $15.00 USD Received 15 Jun 2009; revised 5 Aug 2009; accepted 8 Aug 2009; published 13 Aug 2009

(C) 2009 OSA 17 August 2009 / Vol. 17, No. 17 / OPTICS EXPRESS 15239

15. M. D. Abramoff, P. J. Magelhaes, and S. J. Ram, “Image Processing with ImageJ,” Biophotonics International 11, 36–42 (2004).

16. M. Mahy, L. Van Eycken, and A. Oosterlinck, “Evaluation of uniform color spaces developed after the adoption of CIELAB and CIELUV,” Color Res. Appl. 19, 105–121 (1994).

17. D. Yelin, B. E. Bouma, S. H. Yun, and G. J. Tearney, “Double-clad fiber for endoscopy,” Opt. Lett. 29(20), 2408–2410 (2004).

18. D. Yelin, B. E. Bouma, and G. J. Tearney, “Volumetric sub-surface imaging using spectrally encoded endoscopy,” Opt. Express 16(3), 1748–1757 (2008).

19. M. J. Suter, B. J. Vakoc, P. S. Yachimski, M. Shishkov, G. Y. Lauwers, M. Mino-Kenudson, B. E. Bouma, N. S. Nishioka, and G. J. Tearney, “Comprehensive microscopy of the esophagus in human patients with optical frequency domain imaging,” Gastrointest. Endosc. 68(4), 745–753 (2008).

1. Introduction

Ultraminiature endoscopy (scopes with diameters less than 1 mm), opens up new possibilities for obtaining diagnostic information in anatomic locations that have previously been inaccessible, converting inpatient techniques to outpatient procedures, and performing safer interventions. Fiber-bundle-based miniature endoscopes are available clinically and have been shown to be beneficial for several medial applications [1–4], including laparoscopy, breast ductoscopy and fetoscopy. Newer ultraminiature endoscope technology using tubular piezoelectric actuators have also been demonstrated to acquire wide-field images of human tissue [5,6].

Spectrally-encoded endoscopy (SEE) is an alternative optical imaging technology for conducting miniature endoscopy [7]. With SEE, broadband light is coupled into a single optical fiber. At the distal end of the endoscope, a diffraction grating-lens pair images the spectrum on the sample such that each spatial location along one dimension is illuminated by light with a different wavelength. One line of the image is then decoded by measuring the spectrum of the light that returns back through the probe. The second dimension is obtained by slow mechanical scanning. SEE devices have several advantages over conventional endoscopes, including potential for a high degree of miniaturization, mechanical flexibility, absence of image pixilation artifacts sometimes seen in fiber-bundle devices, and three-dimensional/Doppler imaging capabilities [8,9]. Previously, an SEE probe with a diameter of 350 µm has been demonstrated [10,11]. Due to its small size, the SEE probe was capable of being used for laparoscopic animal imaging in vivo [10].

One limitation of SEE is that color information is lost, since the spectral bandwidth is utilized to encode spatial location. In this paper, we present a new spectrally-encoded imaging configuration that acquires color images by overlapping three red, green, and blue spectral bands on the sample.

2. Methods

2.1 Color SEE Bench-top system

Figure 1 depicts a schematic of the color SEE bench-top system. A supercontinuum laser (SC400-2, Fianium, UK; Spectral band = 400~2000 nm; Spectral power density > 1mW/nm) was used as the broadband light source. Light from the source was spatially-filtered and de-magnified through spatial filtering optics. The beam diameter was decreased from its original value of 3.3 mm to 525 µm to simulate a probe with a diameter around 500 µm. The collimated beam was transmitted through a beam splitter and was divided by blue (FF495-Di02-25X36, Semrock, NY) and red dichroic reflectors (Q580SP, Chroma, VT) into three red, green, and blue spectral bands. The three beams were directed towards a diffraction grating (2400 lpmm, Wasatch photonics, CA) with three different incident angles. The grating was in the Littrow configuration for the central wavelength of the green beam (central wavelength = 537.5 nm; bandwidth = 75 nm; peak-to-valley (P-V) power variation = 1.86 dB). Thus the incidence and diffraction angles for 537.5 nm were 40.2°. In order for all of the spectral bands to overlap on the sample, the red and blue beams were configured to have different incident angles; the blue beam (central wavelength = 452.5 nm; bandwidth = 75 nm; P-V power variation = 3.68 dB) had an incidence angle of 26.2°, and the red beam (central wavelength =



622.5 nm; bandwidth = 75nm; P-V power variation = 1.51 dB) 58.1°. The diffracted beams were focused by an achromatic doublet (f = 100 mm) and scanned by a mirror-mounted galvanometer scanner (6240H, Cambridge technology, MA). A multi-mode fiber, coupled to a spectrometer (HR2000, Ocean Optics, FL), was placed on the focal plane of the objective lens to measure the illumination spectrum on the sample. Three distinctive peaks were observed in the measured spectrum for each point on the spectrally-encoded line, and the overlap of the three central wavelengths at the center of the field of view (FOV) was validated. The field angle achieved by diffraction was 13.5°, resulting in a field size of 24 mm. The NA on the sample was 0.0026, providing a depth of focus of 80 mm. The number of resolvable points provided by the bench-top system was calculated to be 189 for the green beam, 191 for the blue beam and 236 for the red beam. The light intensities of the three spectral bands on the sample were measured by an optical power sensor (S120B, Thorlabs, NJ) to be 0.46mW (red), 2.42mW (green) and 4.60mW (blue).

Fig. 1. Schematic of the color SEE bench-top system. BS – beam splitter; BR – blue reflector; M – mirror; RR – red reflector; SE line – spectrally-encoded line.

Light remitted from the sample was reflected by the beam splitter. Telescope optics (magnification = 5) were used to increase the reflected beam diameter to 2.6 mm. A spectrometer comprised of a diffraction grating (1200 lpmm, Wasatch Photonics, CA), a camera lens (f = 50mm), and a line-scan camera (SPL2048-140k, Basler, Germany; 2048 pixels; pixel width = 10 µm) was used to analyze the spectrum of the reflected light. The spectrometer had 381 resolvable points for the green beam, 453 for the blue beam and 329 for the red beam. The galvanometer was scanned at a rate of 10 Hz, and 500 lines were acquired from the line-scan camera per frame.

2.2 Image processing

The line-scan camera generated two-dimensional 10-bit monochromatic images (2048 by 500 pixels/image). Pixel intensities at or below the dark current noise level were first set to zero. Then the raw image was divided into three monochromatic images (500 by 500 pixels/image) representing red, green, and blue colors. Although the three spectral bands generated the same field size on the sample, when diffracted by the spectrometer, they illuminated different numbers of pixels. Each spectrum (2048 pixels) was therefore separated into different colors (red = 528; green = 500; blue = 490), and each segmented spectrum was resized to have 500 pixels.

Each spectrally-encoded line was compensated for the non-uniformity of the light source and the light throughput variation of the optical components, and was gamma-encoded by the following procedure. The 10-bit grayscale intensity I at the i'th pixel on a spectrally-encoded line of a segmented image for color C (C is red, green or blue) was converted into the 8-bit grayscale value p by



( ) ( ) ( )1/ 2.2

, , / , ,w wp i C p I i C I i C= (1)

where pw is a reference 8-bit grayscale value for a white reference card (Gretag Macbeth® Color Checker® White balance card, X-Rite, Inc, MI; OD = 0.05); and Iw is the 10-bit grayscale intensity measured for the white reference card. The reference spectrum, Iw, was acquired by imaging the white reference card beforehand, and pw was set as 243 to match the reference value provided from the manufacturer. The three processed 8-bit grayscale images were then combined to form a 24-bit color image by merging red, green and blue channels.

2.3 Color gamut of SEE

Since color SEE uses three monochromatic primaries in illumination and detection, the subset of colors that can be accurately imaged by color SEE is different from that of sRGB [12], a standard color space used for commercial imaging and display devices. Moreover, the combination of the three monochromatic primaries varies from point to point along the spectrally-encoded direction, resulting in different color gamuts for different points. The color gamut of SEE for the center of the field can be depicted as a triangle connecting the three monochromatic primaries (452.5nm, 537.5nm, and 622.5nm) in the CIE 1931 xy chromaticity diagram (black triangle in Fig. 2a). The gray boundary in Fig. 2a shows the monochromatic locus of CIE 2° standard observer generated from experimental results by Wright [13] and Guild [14]. The color gamut of SEE at the center of the field has large enough color region to enclose the color gamut of sRGB (blue triangle in Fig. 2a). The effective color gamut of SEE can be evaluated by overlapping all the color gamut triangles for different sets of the primaries over the entire FOV (Fig. 2b). Although the color subset realizable from all the points in the FOV has relatively small region (100% contour curve in Fig. 2b), half of the FOV (50% contour curve in Fig. 2b) can image a similar subset of colors as sRGB.

Fig. 2. Color gamuts in CIE xy chromaticity diagram: (a) color gamuts of SEE at the center of FOV and sRGB; (b) effective color gamut of SEE in contour representation.

3. Results

A 1950 USAF resolution target was used to measure the spatial resolution of the color SEE bench-top system. Color balance was evaluated by imaging a color checker target (Gretag Macbeth® Color Checker®, X-Rite, Inc, MI). To demonstrate imaging of an arbitrary object, an image of a plastic doll on a green background was acquired. A portion of excised swine small intestine and mesentery was imaged to evaluate the feasibility of imaging animal and human tissues. In color SEE images presented here, the horizontal axis corresponds to the spectrally-encoded axis, and the vertical axis represents the scanning direction of the galvanometer. Color SEE images were compared to images obtained by a digital still camera (K100D, Pentax, Japan) under fluorescent light illumination. A custom white balance setting, integrated within the digital still camera, for the given illumination condition was used.



ImageJ [15] was used to transform the acquired sRGB images into the CIE L*a*b* color space for calculating color differences.

Fig. 3. Images of USAF 1951 resolution target for red (a), green (b), and blue (c) channels and combined RGB image (d). Scale bars represent 5 mm.

Monochromatic images, of the resolution chart, processed by Eq. (1) and then color-coded as red, green, and blue, are shown in Fig. 3a-c. Figure 3d depicts the RGB color image of the resolution chart after the three different SEE channels were merged. The transverse resolution at the center was measured to be approximately 110 µm (Group 2, element 2). The spatial distortion (warping) was due to the angular scanning of the galvanometer, which caused longer wavelengths to be scanned over greater distances. Due to the chromatic aberrations of the optical components and the field curvature of the camera lens in the spectrometer, the focal planes of the three colors were somewhat dislocated with each other, resulting in a slight blurring of the red and blue images (Figs. 3a and c) when the green image (Fig. 3b) was in focus.

Fig. 4. Images of color checker target: (a) image generated by reference CIE L*a*b* values (illuminant D50) provided from X-Rite; (b) image by the color SEE bench-top system; and (c) image by the digital still camera. Scale bars represent 5 mm.



The color checker target image obtained with the SEE bench-top system (Fig. 4b) had a qualitatively similar color appearance to the computer-generated image (Fig. 4a) by using the reference CIE L*a*b* values (provided from the manufacturer; CIE illuminant D50; 2° standard observer) and the image obtained by the digital still camera (Fig. 4c). The three spectral bands used in this experiment are shown below Fig. 4b. The average L*a*b* values for all the four color boxes in the color SEE image and the digital still camera image were measured, and their differences to the reference L*a*b* values were calculated as ∆E* (Table

1). The color difference between ( )* * *

1 1 1, ,L a b and ( )* * *

2 2 2, ,L a b was calculated by

( ) ( ) ( )2 2 2* * * * * * *

1 2 1 2 1 2.E L L a a b b∆ = − + − + − (2)

The color differences for the color SEE image were larger than just noticeable difference (JND) of 2.3 [16]. The relatively large color differences may be due to lack of proper color coordinate conversion since the sets of the three primaries are different between the SEE and sRGB color spaces. The image taken by the digital camera also showed noticeable color differences, which can be explained by the difference in the illumination condition compared to the reference illumination condition (CIE illuminant D50) and imperfection of the white balancing algorithm embodied in the camera. The magnitudes of the color differences compared to the reference values were similar between the color SEE bench-top system and the digital camera, which indicates that the color SEE bench-top system has a similar capability to represent color as the digital still camera, at least for the colors of the color checker target. The standard deviations of the L*a*b* values were also calculated, and the color SEE bench-top system had larger standard deviations than the digital still camera. The larger standard deviations in the color SEE bench-top system are likely explained by the point-wise change of color gamut along the spectrally-encoded direction. Speckle noise in the color SEE image, observed most prominently in the moderate red box (top right box in Fig. 4b), also increased the standard deviation values.

Table 1. Reference CIE L*a*b* (illuminant D50; 2° observer) values of color boxes provided by manufacturer, measured L*a*b* values (mean ± std) by the color SEE

bench-top system and the digital still camera; and their difference with the manufacturer reference.

Reference Color SEE Digital still camera

L* a* b* L* a* b* ∆E* L* a* b* ∆E*

Purplish blue

40.02 10.41 −45.96 44.16 13.11 −41.05 6.97 29.24 28.98 −58.17 24.70

( ± 2.25) ( ± 6.40) ( ± 3.90) ( ± 0.84) ( ± 1.44) ( ± 1.17)

Moderate red

51.12 48.24 16.25 57.51 53.12 24.88 11.79 44.00 42.59 19.24 9.57

( ± 1.98) ( ± 3.92) ( ± 4.89) ( ± 0.67) ( ± 0.84) ( ± 1.84)

Green 55.26 −38.34 31.37 57.89 −26.93 31.05 11.72 51.95 −47.10 29.72 9.51

( ± 2.19) ( ± 9.49) ( ± 4.66) ( ± 0.70) ( ± 0.98) ( ± 1.75)

Red 42.10 53.38 28.19 47.32 48.96 31.40 7.55 35.30 44.51 36.40 13.87

( ± 1.71) ( ± 3.23) ( ± 5.65) ( ± 0.66) ( ± 0.73) ( ± 2.10)

The image of the doll obtained by the color SEE bench-top system (Fig. 5a) was also qualitatively similar to the image obtained by the digital still camera (Fig. 5b). The color differences between the two images were calculated by Eq. (2) at 5 regions with distinctive colors marked by numbers in Fig. 5b. The color differences for the regions 1 to 5 were measured as 21, 24, 14, 31, and 6 respectively resulting in the average difference value of 19. In the color SEE image, speckle noise can be seen in some parts of the image, especially in the doll’s face.



Fig. 5. Images of a plastic doll by the color SEE bench-top system (a) and the digital still camera (b). Scale bars represent 5 mm.

The excised swine small intestine tissue was imaged by the color SEE bench-top system (Fig. 6a) and the digital still camera (Fig. 6b). Due to the low reflected signal from the tissue, the color SEE image was obtained by averaging 10 frames. The signal-to-noise ratio (SNR) of the red channel was still low in the left 30% of the image, but the remaining 70% of the image marked by the red dotted box reproduced color reliably. The low SNR of the red channel was mainly caused by the gradual edge in the transmission spectrum of the red dichroic reflector, which decreased the red beam intensity for the short-wavelength region in the left. Customizing the dichroic reflector to have a sharper edge at the cut-off wavelength can improve the SNR. The two images show qualitatively similar color appearances for both of the mesentery (M) and the small intestine (SI). Blood vessels in the mesentery (arrow) and in the small intestine (arrowhead) are clearly identified in both images. The color differences between the two images at a blood vessel in the mesentery (dashed black circle) and a portion of the small intestine (dotted white circle) were calculated as 11 and 23 respectively, which are similar to the color differences observed in the plastic doll images. As before, speckle noise can be seen in the color SEE image, but the magnitude of the noise was not strong enough to make the image appear unnatural.

Fig. 6. Images of an excised swine small intestine tissue by the color SEE bench-top system (a) and the digital still camera (b). M – mesentery; SI – small intestine. Scale bars represent 5 mm.

4. Discussion

In this paper we have demonstrated a spectrally-encoded method for color imaging that utilizes three red, green, and blue spectral bands that are overlapped on the sample. Images of a color test chart, a plastic doll sample and excised swine small intestine demonstrated a good



qualitative and quantitative color correspondence between images obtained by the color SEE system and images acquired by a digital still camera.

Differences between the two were also observed. The color SEE images had larger standard deviations of L*a*b* values than those for the digital still camera images. Speckle noise in the images was found to be one of the main causes of the large standard deviations. The speckle noise can be reduced by opening up the detection aperture or by using a dual-clad fiber in the probe [17]. Multimode detection can also increase the depth of field, which is beneficial in macroscopic endoscopy imaging applications. More sophisticated image processing methods for converting raw images into RGB color images, which utilize prior knowledge of three monochromatic primaries at each point, might also improve the color appearance of the SEE images.

A shift of the focus between the red, green, and blue beams was also noticed in the color SEE images. One of the main causes of the shift was chromatic aberrations of the optical components, including the achromat, used in the bench-top system. Apochromatic optical components can be used to further correct these chromatic aberrations, which may improve image quality.

The next step in the development of this technology is to create a miniature color SEE probe. Figure 7 shows a schematic of one possible approach. The three different spectral bands can be delivered by three separate fibers, assembled using a custom ferule with three holes, or accomplished by utilizing a custom-drawn fiber with three cores. The three fiber cores can be configured to illuminate a miniature lens and a grating for spectrally-encoded imaging as conducted with the previously demonstrated monochrome probe [10]. By carefully designing the lateral offsets between the three fibers, the three spectral bands will be made to overlap on the sample. Rotational scanning of the probe to acquire two-dimensional images can be conducted at the proximal end by a galvanometer [18] or an optical rotary junction [19]. Although a coherent light source was used in the bench-top system, incoherent light sources with lower cost can also be used in the miniature color SEE probe as long as sufficient light can be coupled into the fiber.

Fig. 7. Schematic of a color SEE miniature probe.

For this design, the color SEE miniature probe will have more resolvable points than conventional fiber-bundle miniature endoscope with the same diameter. For example, a typical fiber-optic imaging bundle with a diameter of 500 µm (FIGH-10-500N, Fujikura, Japan) has 10,000 pixels. Compared to this, the color SEE probe with the beam diameter of 500 µm can have 3.2 times more resolvable points (180 by 180 resolvable points per rectangular image) or 17 times more meaningful pixels (414 by 414 pixels per rectangular image to meet the Nyquist sampling criteria).



Acknowledgements

This research was sponsored by National Institute of Health/National Institute of Biomedical Imaging and Bioengineering (Grant #R21EB007718).



Biomedical Optics Laboratory - Spectrally-encoded color imaging...Spectrally-encoded color imaging...

Documents

Transcript of Biomedical Optics Laboratory - Spectrally-encoded color imaging...Spectrally-encoded color imaging...