Jan 02/tj

Lecture outline

• Basics

• Spectral image

• Spectral imaging systems

• Applications

• Summary

Lecture material by: Markku Hauta-Kasari, Kanae Miyazawa, Jussi Parkkinen,Timo Jääskeläinen, Jouni Hiltunen, Joni Orava, Hannu Laamanen, Jarkko Mutanen

Jan 02/tj

Spectral measurement

Human cone sensitivities

CIE 1964 Color matching functions

0

0,5

1

1,5

2

2,5

360 400 440 480 520 560 600 640 680 720 760 800

Wavelength (nm )

x( )y( )z( )

Tris

tim

ulu

s v

alu

es

Jan 02/tj

Chicken cone sensitivities

Jan 02/tj

Spectral approach to color

• In spectral approach, color is represented by color signal. This causes the color sensation

• The signal is part of electromagnetic spectrum

- in human color vision the range is 380-780 nm

• In spectral approach, we are not limited into this human visual range

Jan 02/tj

Motivation for spectral color

• Not to loose important color information (To avoid the problem of metamerism)

• To define optimal color sensors

• To develop better color vision models

• To develop novel instruments

• To develop spectral color classifiers and optical implementations for them

Jan 02/tj

Outline

• Basics

• Spectral image

• Spectral imaging systems

• Applications

• Summary

Jan 02/tj

Component images of spectral image

Jan 02/tj

Spectral Image

Spectra from leaves in previous image

Jan 02/tj

MEMORY REQUIREMENTS OF IMAGES

Image size 256x256 512x512

gray-level image 65 kb 262 kb

color (RGB-) image 196 kb 786 kb

spectral, 20 nm resol. 1 Mb 4 Mb

spectral, 5 nm resol. 3 Mb 15 Mb

Jan 02/tj

Definitions• Spectral image

An image, where each pixel is represented by

a spectrum

• Hyperspectral

A term used for spectral images with large

number of spectral components

• RGB-image

spectrum in visible region, three components

• Multispectral

Jan 02/tj

Image Types

TYPE SPECTRAL COMPONENTS---------------------------------------• Gray-scale• Trichromatic• Spectral

– Hyperspectral• Real-time spectral

• Single• Three• >3• Numerous• Numerous

Jan 02/tj

Outline

• Basics

• Spectral image

• Spectral imaging systems

• Applications

• Summary

Jan 02/tj

Spectral Imaging Devices

• Spectral cameras– filter wheels– light filtering– scanning systems– multi-band detectors

Jan 02/tj

Optical principles

• Narrow band filters– interference filters, LCTF, gratings– AOTF

• Broad band filters– absorbance filters– e.g. gratings to implement optimal filters

Jan 02/tj

Joensuun yliopistoPL 11180101 Joensuu

puh. (013) 251 111fax (013) 251 2050

www.joensuu.fi

Formation of the Color Signal

Filter wheel based system

Specim Spectral Camera

http://www.specim.fi

Jan 02/tj

A spectrum sampled at 39 wavelengths

Jan 02/tj

Spectral Imaging

• One approach: to measure the spectral data accurately

A large amount of data

• Other approach: to measure component images using a few optimally designed color filters

Data is convenient for storing and transmission

Spectral image can be reconstructed computationally

Jan 02/tj

Color Filter Design

• One approach: to choose an optimized set of commercially available color filters (for example, Kodak Wratten gelatin filters)

• Other approach: to design optimal color filters computationally (our approach)

adaptive to various application rewritable filter based imaging system needed

Spectral image can be reconstructed computationally, if needed

Jan 02/tj

ACTIVE TYPE

Optimal Light Source

Sample Sample

CCDcamera

CCDcamera

OutdoorIndoor

Light Source

Optimal Filter

Sample + Optimal Light Source Sample + Optimal Filter

PASSIVE TYPE

Next, computational color filter design and thefollowing spectral imaging systems will be studied

Jan 02/tj

Outline

• Basics

• Spectral image

• Spectral imaging systems

• Applications

• Summary

Jan 02/tj

Some applications

• E-commerce

• Tele-medicine

• Image archives

• Accurate color measurement and representation

Jan 02/tj

Digital Imaging

Jan 02/tj

Display characterictics

Broadband network society

Image ProcessingSystem

DisplayImage Acquisition

NetworkDatabase

　E-commerce

Telemedicine

E-commerce, telemedicine in broadband network society

ClothesPaintsTextileBags….

Facial colorSkin diseaseExpressions…..

Jan 02/tj

Joensuun yliopistoPL 11180101 Joensuu

puh. (013) 251 111fax (013) 251 2050

www.joensuu.fi

E-commerce

Differences between images on a display and real images

Online shopping

Return of products

Figure: Tsumura-san, Chiba Univ.., Japan

Jan 02/tj

Joensuun yliopistoPL 11180101 Joensuu

puh. (013) 251 111fax (013) 251 2050

www.joensuu.fi

Environmental dependence of color

Network

Patient

Medical doctor

Jan 02/tj

Image reproduction

• multi-primary displays - six components projective display

(Natural Vision Research Center, Tokyo)

• multi-primary printing

- inkjet

Jan 02/tj

Multiprimary display

• The objects are measured as spectral images

• The display contains 6 primary colors

avoids the problem of metamerism larger color gamut natural colors colors can be seen the same in the measurement place and in the viewing place

Jan 02/tj

Jan 02/tj

Jan 02/tj

Jan 02/tj

RGB-projector

Jan 02/tj

RGB-filters

High and low pass filters

Multiprimary display

6 filters for

Jan 02/tj

Modified RGB-projector

Jan 02/tj

Multiprimary display

Jan 02/tj

Color gamuts for CRT-display and multiprimary display

Jan 02/tj

Spectral video

• Sequence of spectral images

• Shown as movie on the screen

• Very high memory requirements

• Efficient compression has been used

• Shown as RGB or multiprimary (TAO, Japan)

Jan 02/tj

Compression method

• it has been found that separate spatial and spectral compression give best results

• in spectral dimension, PCA-based compression give best results

• combination of PCA in spectral and JPEG in spatial dimension give best PSNR-values for reasonable compression ratios

Jan 02/tj

Outline

• Basics

• Spectral image

• Spectral imaging systems

• Imaging context

• Applications

• Summary

Jan 02/tj

Summary

• Spectral imaging increasing

• Novel instruments needed

• Novel detectors needed

• Novel image compression methods needed

Color research at Joensuu: WWW: http://cs.joensuu.fi/~spectral

Email: mhk@cs.joensuu.fi

Reconstructed Images

•Reconstructed Images from Measured Spectral Images

Observation of Spectral Reflectance

Church

Color Change by Cleaning

OriginalOriginal OnceOnce TwiceTwice Three TimesThree Times

Reconstructed images from Measured Spectral Images

Observation of Spectral Reflectance

OriginalOnceTwiceThree Times

Cleaning Process

1st Cleaning 2nd Cleaning 3rd Cleaning

Each subtract image is normalized by Maximum value