License: Unless otherwise noted, this material is made ... · – Image processing – spatial...

Author(s): Paul Conway, 2008-2011. License: Unless otherwise noted, this material is made available under the terms of the Creative Commons Creative Commons Attribution - Non-Commercial - Share Alike 3.0 License: http://creativecommons.org/licenses/by-nc-sa/3.0/

We have reviewed this material in accordance with U.S. Copyright Law and have tried to maximize your ability to use, share, and adapt it. The citation key on the following slide provides information about how you may share and adapt this material. Copyright holders of content included in this material should contact [email protected] with any questions, corrections, or clarification regarding the use of content. For more information about how to cite these materials visit http://open.umich.edu/education/about/terms-of-use. Any medical information in this material is intended to inform and educate and is not a tool for self-diagnosis or a replacement for medical evaluation, advice, diagnosis or treatment by a healthcare professional. Please speak to your physician if you have questions about your medical condition. Viewer discretion is advised: Some medical content is graphic and may not be suitable for all viewers.

Citation Key for more information see: http://open.umich.edu/wiki/CitationPolicy

Use + Share + Adapt

Make Your Own Assessment

Creative Commons – Attribution License

Creative Commons – Attribution Share Alike License

Creative Commons – Attribution Noncommercial License

Creative Commons – Attribution Noncommercial Share Alike License

GNU – Free Documentation License

Creative Commons – Zero Waiver

Public Domain – Ineligible: Works that are ineligible for copyright protection in the U.S. (17 USC § 102(b)) *laws in your jurisdiction may differ

Public Domain – Expired: Works that are no longer protected due to an expired copyright term.

Public Domain – Government: Works that are produced by the U.S. Government. (17 USC § 105)

Public Domain – Self Dedicated: Works that a copyright holder has dedicated to the public domain.

Fair Use: Use of works that is determined to be Fair consistent with the U.S. Copyright Act. (17 USC § 107) *laws in your jurisdiction may differ Our determination DOES NOT mean that all uses of this 3rd-party content are Fair Uses and we DO NOT guarantee that your use of the content is Fair. To use this content you should do your own independent analysis to determine whether or not your use will be Fair.

{ Content the copyright holder, author, or law permits you to use, share and adapt. }

{ Content Open.Michigan believes can be used, shared, and adapted because it is ineligible for copyright. }

{ Content Open.Michigan has used under a Fair Use determination. }

SI 675 Digitization for Preservation

Week 4 – Color in Scanning and Scanner Benchmarking

Today’s Outline

  Encoding   Performance measurement

  Spatial Frequency Response   Tone and color response   Noise and artifacts

Winter 2011 4 SI 675 Digitization for Preservation

“Original-referred images maintain a defined relationship between the original and digital version; the digital version carries forward certain pre-identified essential characteristics of the original version.” (p. 9)

•  Puglia/Rhodes, Digitization for Preservation (2007)

Winter 2011 SI 675 Digitization for Preservation 5

What is an Image… and how is it characterized ?

A two dimensional spatial structure of varying light levels and colors. It is characterized by measuring physically realizable light intensities over a two dimensional space. These variations can occur over short distances, like edges, ( high frequencies) or larger distances or areas, like sky or facial features ( low frequencies).

‘x’

‘y’

Image Science Associates

SI 675 Digitization for Preservation Winter 2011 6

What is a Digital Imaging System?

A collection of optical, software, or electronic functions that convert, encode, or otherwise act upon images or their optical or digital derivatives.

Processing

source

Illumination optics

Image forming optics

detector

sample

The performance of a digital capture system is influenced by all of the above in addition to operator training and environment.

Digital image file


Rendering Intents Revisited

  When “… digital image is a ‘replacement’ of the original…” (p. 28)   Decisions made about color space before images are

digitized. Adjustments made during scanning.   “… not be the goal to reproduce the physical properties

of the original, but to reproduce its appearance under certain viewing conditions.”

  Scanned image matches original (viewing calibration)   Adjust for errors in the source   Original appearance rendered (reverse aging)   “Original scene” (remove effects of intermediate)

•  Frey & Reilly, Digital Imaging, 1999 $ 2006


Master Files: Raw or Cooked

“One of the basic things to understand is that a great deal of image quality can be lost by processing the images at the moment they are captured.”   Frey & Reilly (1999): First, a so-called archive file containing more

than eight bits per channel should be stored. It should not be treated for any specific output.

  Frey & Chapman (2000): save unrendered image space to contain a colorimetric estimate of the original. (p. 168)

  Susstrunk (2002): capture and encode into a sensor or source space, which is device and image specific. (p. 2)

  Susstrunk (2002): transform from raw sensor data (device dependent) to original-referred color encoding; then image dependent processing to output-referred color encoding to output device coordinates. (p. 389)

  Puglia/Rhodes (2007): Original-referred is not raw (sensor)

•  Frey & Chapman, Developing Specifications, 2000.


Definitions and Terms

  Original vs. scene: scanner/camera OR artifact/nature   Color encoding: relationship of the real and its representation   Color model: abstract mathematical model of color

representation (e.g. RGB, CMYK)   Gamut: subset of colors that can be accurately represented in a

given circumstance (e.g. color space or output device).   Color spaces: color model + gamut   Absolute color space: colors are unabiguous; interpretation is

colorimetrically defined without reference to external factors.   ICC color profile: color attributes of a particular device or

viewing requirement ; map between color space and profile connection space (CIELAB or CIEXYZ)


SI 675 Digitization for Preservation 10

Digital Cameras do not Reproduce Colors, They Encode them.

They describe (encode) object color characteristics. Reproduction requires interpreting (decoding) the pixel values to display the

color image.

Source Object

Color Negative Densities ( Encoded)

Print

Capture Render

Color film negative path

Source Object

Encoded Count Values

Printed or display

Capture Render

Color digital camera path

Digital negative?

Winter 2011


Color Encoding Flowchart •  Susstrunk, Color Encodings (2002).



Raw Files, Be Careful What You Ask For

De-mosaicing Raw color interpolated Data

Original Scene or object Raw corrected

CFA Data Sensor Sensor defect, uniformity, correction

Scene referred Color encoding

white balance, camera color correction

Display transform

Softcopy image, or print

Color rendering and output referred encoding

Output referred Color encoding

Winter 2011


Sensor Encoding

  “When a high quality master file is desired, storing ‘raw image’ data encoded in sensor space is most appropriate. Any further processing can degrade the image, as algorithms are based on today’s know-how and might be improved in the future.” (p. 390)

•  Susstrunk, Managing Color (2002).


Input- and Output-Referred Encoding   Input-Referred Encoding (master)

  Scene versus Original   Photo YCC (Kodak Photo CD)   RIMM RGB   ICC PCS

  Output-Referred Encoding (derivative)

  Transformed from source or input   sRGB (screen and color printing)   E-sRGB (rendering for unknown output)   ROMM RGB (extended bit depth)   Adobe RGB (graphic arts and printing)

•  Susstrunk, Color Encodings (2002).


Attributes of color encodings •  Susstrunk, Color Encodings (2002).


Color Gamut Comparison

Range of Chromaticities Typical CRT Gamut


[Susstrunk 2007]

Color Space Comparison

Shape of Visible Color Gamut Comparisons


[Susstrunk 2007]

Color Space Comparison

Adobe RGB Color Space ROMM (ProPhoto) RGB


Winter 2011 19

Primary Imaging Performance Functions

• Signal – Any response that provides valued information - Large area response to light OECF -Opto-Electronic Conversion Function

- Spatial proximity behavior Spatial Frequency Response – SFR ( or MTF)

• Noise – Any response that detracts from a desired signal - Light intensity distortions – Total noise - Geometric/Spatial distortions

A number of these categories have ISO standards that define the metrology practice. Though intended for digital imaging devices their basis was derived from decades of analog (e.g., film) imaging experience.

Important Imaging Characteristics


Winter 2011 20

Imaging Performance Framework http://www.digitizationguidelines.gov/stillimages/documents/imaging.html


Winter 2011 21

Measurement Requirements

Measurements usually require some level of accuracy and precision. –  Accuracy: average error from an aim –  Precision: variability about the average reading

Factors that influence measurements –  Location on platen –  Image processing –  spatial sampling –  image noise –  environment –  operator skill

X

X

X X

X X X X

X X

Aim – The point or set of points intended to be hit

Performance is more about consistency ( precision) than accuracy. In imaging, accuracy is often not absolute but rather a preference.



Spatial Resolution Spatial Frequency Response (SFR) ISO 16067-1, 16067-2, 12233, 15529


Winter 2011 23

SIGNAL Any response that provides valued information


What is Spatial Resolution ….and what are all those related terms ?

Winter 2011 24

• The ability of an imaging component or system to distinguish finely spaced detail. Specifically, the ability to maintain the relative contrast of finely spaced detail. • Highest frequency ( smallest distance) at which light and dark parts of image are reliably distinguishable. Sometimes called limiting resolution

• Number of pixels •  Sampling frequency - dpi, ppi •  *Limiting Resolution, Resolving Power •  *Spatial Frequency Response - SFR •  *Modulation Transfer Function - MTF

Related Metrics

*Related Standards – ISO 16067-1, 16067-2, 12233, & 15529 all employ these metrics



Space and Frequency

Winter 2011 25

  Two ways to look at image details   Space: size of the smallest important (signal) feature (mm)   Frequency: How many small important features/area will my

image store (number/mm, cycles/mm, dots/inch, pixels/inch)

  Small size implies high frequency   Why use space vs. frequency descriptions?

  Compatible with engineering descriptions of information, bandwidth

  Simplifies some forms of system analysis   Compatible with several visual image quality descriptions



Spatial Resolution

Winter 2011 26

• Two factors in a digital imaging system dictate the level of spatial resolution or detail that can ultimately be captured.

  Quantity; sampling frequency ( i.e. samples per mm, inch )   Sets a capability limit that can be achieved

  Quality; effective optical quality   Defines the level of optical “blur” that the imaging optics,

environmental factors, hardware, and image processing impose on the captured image.

• These are necessary but insufficient components, by themselves, for capturing spatial detail. Simply improving one to reconcile deficiencies in the other will not enhance real resolution performance.



- Limiting Resolution - ….so how does one measure it ?

Winter 2011 27

PROS * Simple to measure * Easily understood * Suitable for binary scanners

CONS * Relies on human readable subjective evaluation * Results are target contrast dependent * Provides little diagnostic or image quality insight * Purely an ON/OFF (threshold) criterion

Increasing spatial frequency ( lp/mm or cycles/mm)

Yes, the features are readable

No, the features are illegible

Limiting Resolution= That spatial frequency at which finely spaced features are no longer visually detectable

Imaged target

Traditional method using bar targets and visual evaluation



Sampling Frequency is not Resolution

Winter 2011 28

Increasing # lines/inch (line width = 1/200 “)

(line width = 1/300 “)

(line width = 1/400 “)

DPI (PPI) -> Quantity of data Resolution -> Quality of data (information)

Resolution Wedge Test Feature Example



Winter 2011 29

Sampling rate (DPI) is not Resolution real data example : results are not simulated

Resolution limit = whenever all five lines are undetectable

Though the sampling rate is increased to 600 dpi the true resolution for this scanner is only 300 dpi.

Resolution

Sampling Rate



Spatial Frequency Response* (SFR) the better alternative

Winter 2011 30

Instead of using a subjective YES/NO criterion along the vertical axis, an objective contrast (i.e. modulation) transfer ratio is adopted. The SFR it is an indicator of modulation( i.e. contrast) loss as a function of spatial frequency.

Limiting resolution will generally correspond to the spatial frequency having a 0.10 modulation transfer value

Increasing spatial frequency ( lp/mm or cycles/mm)

0.0

0.5

1.0

ModulationTransfer

Modulation Transfer = Target feature modulation after being imaged

Target feature modulation before being imaged

* Also referred to as the Modulation Transfer Function (MTF)



Limiting resolution fails to predict quality&

Winter 2011 31

SFRs for RIT Targets

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 2 4 6 8 10 12

Spatial Frequency ( cycles/mm)

Mod

ulat

ion

Tran

sfer

SFR for right targets

SFR for left targets

SFR for middle targets



Winter 2011 32

Anatomy of the SFR - regions of behavior -

Increasing spatial frequency (cycles/ ?? )Half-sampling frequency

Rel

ativ

e D

etai

l Con

trast

1.0

0.10 Rayleigh Resolution Criterion

Alia

sing

art

ifact

s ( j

aggi

es, m

oire

)

Lim

iting

re

solu

tion

Acu

tanc

e an

d Sh

arpn

ess

Sharpening operators (over-sharpening, halo-ing, scream)

Flar

e (h

aze)



Winter 2011 33

0.5

1.0

0.1

0.25 0.5Frequency, cy/pixel

SFR

high sharpness, low resolution

0

0.5

1.0

0.1

0.25 0.5

low sharpness, high resolution

0.5

1.0

0.1


SFR


0

0.5

1.0

0.1

0.25 0.5


0.5

1.0

0.1


SFR


0

0.5

1.0

0.1

0.25 0.5


0.5

1.0

0.1


SFR


0

0.5

1.0

0.1

0.25 0.5


Sharpness vs. Resolution



How is the SFR Measured ?

Winter 2011 34

...all you need is an image of a GOOD slanted edge.



Winter 2011 35

Flatbed scanner using a linear array Horizontal (fast) and Vertical (slow) scan directions

fast

slow

Max. measurable frequency at 600 dpi sampling



- SFR in Review -

Winter 2011 36

  What is it?   descriptive plot that measures the extent to which image detail contrast (modulation) is maintained by

an imaging component or system.

  What is a spread function?   image resulting from a point, or very narrow line exposure.

  What it is not?   same as ‘resolution’   sampling frequency or number of pixels   complete descriptor of image quality (noise, color, …)

  What good is it?   It provides a way to analyze the influence of imaging components on the retention and reproduction of

low, medium, and high spatial frequencies ( flare, sharpening, limiting resolution )   Which digital imaging standards address it ?

  ISO 16067-1: Resolution for digital print scanners   ISO 16067-2: Resolution for digital film scanners   ISO 12233 : Resolution for digital still cameras   ISO 15529 : MTF for sampled imaging systems



Opto Electronic Conversion Function (OECF) ISO 14545 aka Tone Transfer Curve or Tone Reproduction Curve (TRC)

Winter 2011 38

SIGNAL Any response that provides valued information

ISO 22028-1


Winter 2011 39

Optical properties of objects

Density= - log10 (reflectance) * In practice, it is often more convenient to specify in terms of density, especially for densities greater than 1.0

Incident Illumination - 1000 units

1000 units reflected 1000 units incident = 1.000 reflectance

10 units reflected 1000 units incident

1 unit reflected 1000 units incident

100 units reflected 1000 units incident = 0.1000 reflectance

= 0.0100 reflectance

= 0.0010 reflectance

= 0.0 density

= 1.0 density

= 2.0 density

= 3.0 density

Reflective or transmissive hardcopy



How is the OECF created ?

Winter 2011 40

(1) Scan grayscale step target with known optical reflectances*

neutral optical

reflectance

neutral optical density

0.010 20.013 1.90.016 1.80.020 1.70.025 1.60.033 1.480.042 1.380.051 1.290.065 1.190.083 1.080.107 0.970.132 0.880.166 0.780.209 0.680.263 0.580.339 0.470.437 0.360.550 0.260.708 0.150.912 0.04

(2) Compute average count value from the digital image file of scanned grayscale target

red green blue 8 8 7 13 13 14 18 17 18 24 25 25 31 32 30 38 40 39 45 46 45 52 51 51 60 62 61 69 70 69 81 82 81 92 93 92

105 105 104 119 120 120 135 132 132 153 155 154 172 173 172 194 193 193 219 220 220 247 250 249

average count value

(3) Table of reflectance – average count value data pairs for each color channel

neutral optical

reflectance red green blue

0.010 8 8 7 0.013 13 13 14 0.016 18 17 18 0.020 24 25 25 0.025 31 32 30 0.033 38 40 39 0.042 45 46 45 0.051 52 51 51 0.065 60 62 61 0.083 69 70 69 0.107 81 82 81 0.132 92 93 92 0.166 105 105 104 0.209 119 120 120 0.263 135 132 132 0.339 153 155 154 0.437 172 173 172 0.550 194 193 193 0.708 219 220 220 0.912 247 250 249

average count value

* Most grayscale targets come with documentation on reflectances/densities of each patch. Over time, these may fade or change though. A densitometer can be used to verify any change.


Graphical Representation of OECF

Winter 2011 41

OECF example

0

50

100

150

200

250

0.000 0.200 0.400 0.600 0.800 1.000

neutral ( gray) reflectance

aver

age

gree

n co

unt v

alue

( 8

bit)

OECF example

0

50

100

150

200

250

0 0.5 1 1.5 2

neutral (gray) densityav

erag

e gr

een

coun

t val

ue (

8 bi

t)

Reflectance Density Using Density is more revealing since the data values tend to be more evenly distributed



Standards, Guidelines, Exposure - efficiently assigning density to count values -

Winter 2011 42

Typical Aim OECF Curves

0

50

100

150

200

250

0 0.5 1 1.5 2 2.5

density

8 bi

t cou

nt v

alue

ECI v2 gamma 2.2 gamma 1.8

Collection Specific OECF Aims‐ NARA Guidelines ‐

0

50

100

150

200

250

0 0.5 1 1.5 2 2.5density

8 bi

t cou

nt v

alue

NARA 2.0 max NARA 2.4 max

If most measured data values are above a selected OECF aim curve, over-exposure is indicated, conversely if most measured data values are below a selected OECF aim curve, under-exposure is indicated



White Balance / Neutrality -Keep all of the Neutrals neutral -

Winter 2011 43

• OECFs can be measured for each color channel using a target’s neutral gray patches. • 85% of good color imaging performance is keeping the Red, Green, and Blue OECFs the same… Really!

This is a good example of a well white balanced capture Note that all color channel OECFs lie on top of each other



White Balance / Neutrality -Keep all of the Neutrals neutral -

Winter 2011 44

This is an example where the white balance performance is poor. Note how the blue channel OECF departs from the red and green OECF




Question of Balance: Keep the Neutrals Neutral

Winter 2011



Question of Balance: Keep ^ the Neutrals Neutral

Neutral Tone Transfer - OECF

Note departure from neutrality in highlights

all of

Winter 2011


A few words on

Color Encoding Error ( ∆E)

- erroneously called color accuracy -

Winter 2011 47

• Digital cameras/ scanners do not reproduce colors; they encode them.

• Color accuracy implies that two physically realizable colors are at hand to compare.

• Assumptions on how RGB code values will ultimately be interpreted ( i.e. rendered or decoded) as physically reproduced color are often very wrong.

• Delta E ( i.e. color error ) measures for digital capture devices are suitable for performance consistency monitoring but not as absolute measures of color accuracy



Winter 2011 48

Example ∆E Analysis for Adobe RGB encoding

Summary measures for color encoding error



Winter 2011 49

OECF – Summary and Recommendations

•  There is no single best or standardized OECF. Most are based on policy maker or user preferences for the “look” of the image with respect to a particular use case.

•  Generally, any gamma setting between 1.5 – 2.5 is fine for capturing all

significant image information for common reflection media densities.

•  The important practice is to document what the OECF is for ease of future image reproduction and rendering.

•  85% of good color management relies on keeping the neutrals neutral. Do not assume that individual color channel OECFs for a gray scale target will be identical. By documenting the OECF though, any mismatch can usually be corrected. ( called “corrective action” )

•  Leave sufficient CV ”buffer” in the OECF for future tone and spatial image manipulations. Dmax ≈15 count value (CV), Dmin ≈ 240 CV

•  An alternative to documenting the OECF is to include an image of a grayscale target in each digital image file and maintain it as part of the image file. Be sure the optical densities of each patch are somehow indicated.



Noise Random and otherwise


Winter 2011 51

NOISE Any response that detracts from a desired signal


Scanner Noise Characterization

red green blue8 8 7

13 13 1418 17 1824 25 2531 32 3038 40 3945 46 4552 51 5160 62 6169 70 6981 82 8192 93 92105 105 104119 120 120135 132 132153 155 154172 173 172194 193 193219 220 220247 250 249

average count valuered green blue2.00 1.95 2.12.11 1.88 2.221.95 1.57 2.41.8 1.65 1.99

1.79 1.45 1.91.82 1.4 1.881.45 1.37 1.561.5 1.15 1.77

1.49 1.04 1.461.34 1.23 1.31.23 1.12 1.511.39 1 1.21.1 0.99 1.251.2 1.04 1.161 1.15 1.1

0.88 0.94 1.290.95 0.93 1.040.92 0.77 0.90.87 0.76 1.090.9 0.65 0.97

count value variability- noise

Winter 2011 52

OECF data RMS Noise data


Scanner Noise Characterization

Winter 2011 53

RMS noise (CV)behavior for scanner

0.00

0.50

1.00

1.50

2.00

2.50

3.00

0.00 0.50 1.00 1.50 2.00

Average Density

Cou

nt v

alue

dev

iatio

n

redgreenblue

density red green blue2.00 2.00 1.95 2.11.89 2.11 1.88 2.221.80 1.95 1.57 2.41.70 1.8 1.65 1.991.60 1.79 1.45 1.91.48 1.82 1.4 1.881.38 1.45 1.37 1.561.29 1.5 1.15 1.771.19 1.49 1.04 1.461.08 1.34 1.23 1.30.97 1.23 1.12 1.510.88 1.39 1 1.20.78 1.1 0.99 1.250.68 1.2 1.04 1.160.58 1 1.15 1.10.47 0.88 0.94 1.290.36 0.95 0.93 1.040.26 0.92 0.77 0.90.15 0.87 0.76 1.090.04 0.9 0.65 0.97



Two Forms of Random Noise

Winter 2011 54

Fine-grain noise -> high frequency fluctuations

Course-grain noise -> low frequency fluctuations

Both of these images have the same amount of total noise



One-dimensional random noise - banding and streaking -

Winter 2011 55

Following contrast enhancement

• Often found on devices using linear arrays • Contrast and equalize enhancements are useful in detecting such noise behaviors



Artifacts

Winter 2011 56

  Odd imaging behaviors that seem to defy objective measurement…for the time being.

  De-texturing   Checkerboarding   Localized geometric distortions   Sensor defects   Color aliasing

  Many are technology specific. Just knowing what type of scanner technology was used allows one to prepare for certain artifacts


Winter 2011 57

NOISE Any response that detracts from a desired signal


Winter 2011 58

Visual Inspection for Artifacts

Color Misregistration



Visual assessments of spatial artifacts is quite reliable

Example: Visual inspection needed for adaptive processing artifacts

Winter 2011


Winter 2011 60

Visual assessments of spatial artifacts is quite reliable Visual Inspection for Artifacts

optical resolution

limit

resolution limit due to CFA de-mosaicing



Winter 2011 61

Visual assessments of spatial artifacts is quite reliable

Visual Inspection for Artifacts

Bad Cam in a Linear array scanner




Consider Intra-image and Inter-image Variation

Inter: image-to-image Intra: across image field

W.D.Wright, Photometry and the Eye, Hatton Press, 1949

OECF Color balance

Winter 2011



Intra-image: Across field

Device-level target

OECF Spatial frequency response

Variation in •  OECF optical falloff •  Effective resolution •  Color misregistration

Winter 2011


References   Frey and Reilly. [1999] Digital Imaging for Photographic Collections: Foundations

for Technical Standards. RIT, Image Permanence Institute.   Frey/Chapman. [2000] “Developing Specifications for Archival Digital

Images.” IS&T 2001 PICS Conference Proceedings.   Puglia/Rhodes. [2007] “Digitization for Preservation Reformatting of

Photographs (DRAFT).”   Süsstrunk. [2002] “Color Encodings for Image Databases,” Proc. IS&T/SPIE's

Electronic Imaging: Internet Imaging, Vol. 4672, pp. 179-185.   Süsstrunk. [2007] “Image Formation, Colorimetry, Color Spaces, Color

Encodings and Color Image Encodings, Color Management,” in Focal Encyclopedia of Photography: 4th Edition, Elsevier (Focal Press).

  Süsstrunk. [2002] “Managing Colour in Digital Libraries,” in Colour Engineering: Achieving Device Independent Colour, John Wiley & Sons, Ltd.


Thank you!

Paul Conway Associate Professor School of Information University of Michigan www.si.umich.edu


License: Unless otherwise noted, this material is made ... · – Image processing – spatial...

Documents

Transcript of License: Unless otherwise noted, this material is made ... · – Image processing – spatial...