Color Image Quality Assessment Part II: Image Quality Metrics T4C tutorial notes... · Simone, G.;...
Transcript of Color Image Quality Assessment Part II: Image Quality Metrics T4C tutorial notes... · Simone, G.;...
Color Image Quality Assessment Part II:
Image Quality Metrics Marius Pedersen
The Norwegian Colour and Visual Computing Laboratory, Gjøvik University College, Gjøvik, Norway
Jan P. Allebach
School of ECE, Purdue University
West Lafayette, Indiana
Synopsis • What is an image quality metric • Classification of metrics
– Mathematically based metrics – Low-level based metrics – High-level based metrics
• Important factors for metrics – Masking – Pooling
• Evaluation of metrics • Image quality attributes
What is an image quality metric? • An objective mathematical way to calculate
quality without asking observers.
3
Image Metric Measure of quality
Different types of metrics • Three main types of metrics:
– Full-reference. – No-reference. – Reduced-reference.
4
Reproduction
Original
Metric Measure of quality
Existing image quality metrics • Metrics usually follow a common framework.
• Different stages:
• Unless stated otherwise we focus on full-reference
5
Color space transforms
Human visual system models
Quality calculation
Pooling
Fewer
Many
Quality value
Original and reproduction
Colour space transforms
• Preparation for applying a model of the Human Visual System (HVS).
• This step is a tranformation from RGB (or another colour space) into a more suitable space.
• This space is usually adapted to the filtering, where a better representation of the perception of colour is achieved. – For example an opponent colour space.
Color space transforms
6
Human visual system models
• These models usually simulate low level features of the HVS, such as contrast sensitivity functions (CSFs) or masking.
• Other possibilites are high-level features, such based on the idea that our human visual system is adapted to extract information or structures from the image.
Human visual system models
7
Quality calculation • Usually quality calculation is a distance.
– Assumes that the original has the highest quality.
– Euclidean distance
• Done in a perceptually uniform color space. – Nonlinearly transformed color space so that distance is
proportional to ones ability to perceive changes in color.
– Recently, CIELAB most commonly used.
Quality calculation A
B
25/09/12 Eq. from http://en.wikipedia.org/wiki/Euclidean_distance 8
Pooling • Pooling is the reduction of quality values.
– Quality map reduced to fewer values. – Values from different metrics to an overall value.
• Motivation: Easier to manage one value than many.
• Most metrics pool by taking the average.
9
Pooling
Quality map
Pooling
Fewer
Many
Synopsis • What is an image quality metric • Classification of metrics
– Mathematically based metrics – Low-level based metrics – High-level based metrics
• Important factors for metrics – Masking – Pooling
• Evaluation of metrics • Image quality attributes
Classification of metrics
• Metrics can be classified into several categories:
– Mathematically based metrics.
• MSE or ∆𝐸𝑎𝑏∗
11
operate only on the intensity of the distortions.
M Pedersen, JY Hardeberg. Full-Reference Image Quality Metrics: Classification and Evaluation. Foundations and Trends® in Computer Graphics and Vision 7 (1), 1-80
Classification of metrics
• To understand the metrics we propose a classification of them into: – Mathematically based
metrics.
• MSE or ∆𝐸𝑎𝑏∗
– Low-level based metrics. • S-CIELAB or S-DEE.
12
take into account the visibility of the distortions using low-level models of the human visual system.
M Pedersen, JY Hardeberg. Full-Reference Image Quality Metrics: Classification and Evaluation. Foundations and Trends® in Computer Graphics and Vision 7 (1), 1-80
Classification of metrics
• To understand the metrics we propose a classification of them into: – Mathematically based metrics.
• MSE or ∆𝐸𝑎𝑏∗
– Low-level based metrics. • S-CIELAB or S-DEE.
– High-level based metrics. • SSIM or VIF.
13
are based on the idea that our human visual system is adapted to extract information or structures from the image.
M Pedersen, JY Hardeberg. Full-Reference Image Quality Metrics: Classification and Evaluation. Foundations and Trends® in Computer Graphics and Vision 7 (1), 1-80
Classification of metrics • To understand the metrics we
propose a classification of them into: – Mathematically based metrics.
• MSE or ∆𝐸𝑎𝑏∗
– Low-level based metrics. • S-CIELAB or S-DEE.
– High-level based metrics. • SSIM or VIF.
– Other metrics. • VSNR or CISM.
14
are either based on other strategies or combine two or more of the above groups.
M Pedersen, JY Hardeberg. Full-Reference Image Quality Metrics: Classification and Evaluation. Foundations and Trends® in Computer Graphics and Vision 7 (1), 1-80
Mathematically based metrics: MSE • MSE is a mathematically based metric; it
calculates the cumulative squared error between the original image and the distorted image.
• MSE is given as:
– where x and y indicate the pixel position, M and N are the image width and height.
• These simple mathematical models are usually not well correlated with perceived image quality.
– Still been of influence to other metrics.
15
Mathematically based metrics: ∆𝐸𝑎𝑏∗
• Metrics measuring color difference also belong to the group of mathematically based metrics.
– Lr,ar,br is the sample color and Lo,ao,bo is the reference
color in CIELAB.
• ∆𝐸𝑎𝑏∗ has served as a satisfactory tool for measuring perceptual
difference between uniform color patches
16
Mathematically based metrics: ∆𝐸𝑎𝑏∗
• ∆𝐸𝑎𝑏∗ has also been used to measure natural
images, where the color difference of each pixel of the image is calculated.
• The mean of these differences is the overall indicator:
17
Example mathematically based metrics
Original image from R. Halonen, M. Nuutinen, R. Asikainen, and P. Oittinen. Development and measurement of the goodness of test images for visual print quality evaluation. In S. P. Farnand and F. Gaykema, editors, Image Quality and System Performance VII, volume 7529, pages 752909–1–10, San Jose, CA, USA, Jan 2010. SPIE.
18
Example – image difference maps
19
Synopsis • What is an image quality metric • Classification of metrics
– Mathematically based metrics – Low-level based metrics – High-level based metrics
• Important factors for metrics – Masking – Pooling
• Evaluation of metrics • Image quality attributes
Low-level based metrics
• Low-level based metrics simulates the low level features of the HVS, such as contrast sensitivity functions (CSFs) or masking.
• Contrast sensitivity is a measure of the ability to discern between luminance of different levels in a static image.
21
Typical CSF functions • As introduced in the
first part. • CSF varies with many
physical attributes: – spatial frequency, – orientation, – light adaptation
level, – image area, – viewing distance, – retinal eccentricity.
Figure from C. A. Bouman: Digital Image Processing - January 9, 2012 (25/09/12: https://engineering.purdue.edu/~bouman/ece637/notes/pdf/Opponent.pdf)
22
Low-level based metrics: S-CIELAB
• ∆𝐸𝑎𝑏∗ was not correlated with perceived image
difference. • Zhang and Wandell proposed a spatial extension
based on ∆𝐸𝑎𝑏∗
• They had two goals: – a spatial filtering to simulate the blurring of the HVS. – consistency with the basic CIELAB calculation for large
uniform areas.
23
Zhang, X. & Wandell, B. A. A spatial extension of CIELAB for digital color image reproduction. Proc. Soc. Inform. Display 96 Digest, Soc. Inform. Display 96 Digest, 1996, 731-734
Low-level based metrics: S-CIELAB • Color separation:
– Image transformed into the O1O2O3 opponent color space.
• Spatial filter: – Data in each color channel is filtered by
a 2-dimensional separable spatial kernel.
• Color difference: – CIELAB color space
– ∆𝐸𝑎𝑏∗ to calculate color differences.
• Pooling: – Usually taking the average.
Figure from http://white.stanford.edu/~brian/scielab/scielab3/scielab3.pdf 14/09/12
24
S-CIELAB CSFs
Figure from Johnson, G. M. & Fairchild, M. D. Darwinism of Color Image Difference Models. Color Imaging Conference, 2001, 108-112
25
Example S-CIELAB • Want to test S-CIELAB? Matlab code available online at
http://white.stanford.edu/~brian/scielab/scielab.html • Loading Hats and HatsCompressed
– load images/hats – load images/hatsCompressed
• Define viewing conditions: – We choose two different conditions
• SPD = 23 (18in/72dpi) and SPD = 56 (44.5in/72dpi) • SPD(DPImonitor/((180/pi)*atan(1/NoINCH)))
• Run S-CIELAB code
26
Example S-CIELAB - maps
20 40 60 80 100 120 140 160 180
20
40
60
80
100
1205
10
15
20
25
30
35
40
45
50
18 inches viewing distance Mean=3.4, Min=0.4, Max=52.6, median=2.4
20 40 60 80 100 120 140 160 180
20
40
60
80
100
120
5
10
15
20
25
44.5 inches viewing distance Mean=2.3, Min=0.02, Max=28.2, median=1.7
27
Other low-level based metrics
• Spatial-DEE (S-DEE) – This metric follows the S-CIELAB framework, but
∆𝐸𝑎𝑏∗ is replaced with ΔEE.
– Spatial filters from Johnson and Fairchild.
• Adaptive Bilateral Filter (ABF) – uses a bilateral filter to blur the image, while
preserving edges, which is not the case when using CSFs.
28
Simone, G.; Oleari, C. & Farup, I. PERFORMANCE OF THE EUCLIDEAN COLOR-DIFFERENCE FORMULA IN LOG-COMPRESSED OSA-UCS SPACE APPLIED TO MODIFIED-IMAGE-DIFFERENCE METRICS. 11th Congress of the International Colour Association (AIC), 2009 Wang, Z. & Hardeberg, J. Y. Development of an adaptive bilateral filter for evaluating color image difference. Journal of Electronic Imaging, 2012, 21, 023021-1-023021-10
Comparison of filtering methods
• Different filtering methods: CSFs (S-CIELAB), bilateral filter (from ABF), CSFs in NSCT (Pedersen et al.).
Original 29
S-CIELAB NSCT ABF
Pedersen, M.; Liu, X. & Farup, I.. Improved Simulation of Image Detail Visibility using the Non-Subsampled Contourlet Transform. Color and Imaging Conference, 2013
Synopsis • What is an image quality metric • Classification of metrics
– Mathematically based metrics – Low-level based metrics – High-level based metrics
• Important factors for metrics – Masking – Pooling
• Evaluation of metrics • Image quality attributes
High level based metrics
• High-level based metrics quantify quality based on the idea that our HVS is adapted to extract information or structures from the image.
31
High level based metrics: SSIM • SSIM defines the structural information in an image as those attributes
that represent the structure of the objects in the scene, independent of the average luminance and contrast.
• Quantifies perceived change in structural information. – Incorporates luminance masking and contrast masking.
32
Figure from Wang, Z.; Bovik, A. C.; Sheikh, H. R. & Simoncelli, E. P. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 2004, 13, 600-612
High level based metrics: SSIM
– where μ is the mean intensity for signals x and y, and σ is the standard deviation of the signals x and y. signals x and y are of size MxN.
– C is a constant defined as
– where L is the dynamic range of the image, and K1<<1. C2 is similar to C1 and is defined as: • where K2<<1. These constants are used to stabilize the
division of the denominator. 33
High level based metrics: SSIM
• SSIM is calculated for local windows in the image.
• A single value is given as:
– where X and Y are the reference and the distorted images, 𝑥𝑗 and 𝑦𝑗 are image content in local window j, and W indicates the total number of local windows.
34
Example SSIM
• Want to test SSIM? https://ece.uwaterloo.ca/~z70wang/research/ssim/
• Transform the images to grayscale – In the following example I have used Rgb2gray() in Matlab
• Run ssim_index(img,img2)
• Using default parameters – K = [0.05 0.05];
– window = ones(8); (window size)
– L = 100; (dynamic range)
35
Example SSIM - maps
20 40 60 80 100 120 140 160 180
10
20
30
40
50
60
70
80
90
100
110 0.3
0.4
0.5
0.6
0.7
0.8
0.9
Mean=0.89, Min=0.23, Max=0.995, median=0.92
36
Other approaches • Others metrics considered in this group are based on other
approaches or metrics combining two or more of the above groups.
• Visual Signal to Noise Ratio (VSNR), based on near-threshold and suprathreshold properties of the HVS, incorporating both low-level features and mid-level features.
• Color image similarity measure, this can be divided into two parts; one dealing with the HVS and one with structural similarity. – Generalization: S-CIELAB framework + SSIM
37
Chandler, D. M. & Hemami, S. S. VSNR: A Wavelet-Based Visual Signal-to-Noise Ratio for Natural Images. IEEE Trans. Image Processing, 2007, 16, 2284-2298 J. Lee and T. Horiuchi. Image quality assessment for color halftone images based on color structural similarity. IEICE Trans. Fundamentals, E91A:1392–1399, 2008.
Synopsis • What is an image quality metric • Classification of metrics
– Mathematically based metrics – Low-level based metrics – High-level based metrics
• Important factors for metrics – Masking – Pooling
• Evaluation of metrics • Image quality attributes
More on HVS modelling – masking
• There are additional aspects of the HVS that can be modeled: – Luminance masking
– Contrast masking
• Masking in sound: – Auditory masking occurs when the perception of
one sound is affected by the presence of another sound.
39
Luminance masking – Perception of lightness is a nonlinear function of luminance. – Luminance masking: the luminance of the original image signal masks the
variations in the distorted signal. – Visibility threshold increases as background luminance increases
– Each image has the same amplitudes but different mean (lowest on the left). – As can be seen, the pattern is more noticeable towards the left. – When the average brightness is higher, the same amount of regional change
amounts to a lower contrast as compared to a lower average brightness. Thus the same variation in a bright region would be less visible than in a darker region.
05/10/12: http://scien.stanford.edu/pages/labsite/1998/psych221/projects/98/dctune/yuke/page2.htm 40
Contrast masking
• The reduction in visibility of one image component caused by the presence of another image component with similar spatial location and frequency content is called “contrast masking”.
• Contrast masking can occur – within a colour channel,
– across channels,
– across subbands,
– across orientations.
41
Example contrast masking • A test contrast pattern
(left) and three different masking contrast patterns (middle).
• The sum of the test and masks are shown to the right.
• The test pattern is difficult to see when the frequency of the test and mask are similar.
Beach image 17/09/15: https://foundationsofvision.stanford.edu/chapter-7-pattern-sensitivity/ 42
Pooling – one step further
• Pooling is very important for achieving an IQ metric correlated with the percept.
Pooling
Quality map
Pooling
Fewer
Many
Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4):600–612, 2004. Z. Wang and X. Shang. Spatial pooling strategies for perceptual image quality assessment. In International Conference on Image Processing, pages 2945–2948, Atlanta, GA, Oct 2006. IEEE.
43
Type of pooling • Pooling can usually be applied in three
different stages:
– 1) Spatial pooling: combining values in the quality map in the image domain.
• Spatial pooling is always needed
Spatial pooling 44
Type of pooling • Pooling can usually be applied in three different stages:
– channel pooling: pooling values from different (color) channels. • needed only when the image is decomposed into different
channels.
Channel pooling
45
Type of pooling • Pooling can usually be applied in three different stages:
– quality attribute pooling: combining several quality maps generated from different quality attributes (i.e. color, lightness)
– Only needed when different quality maps are calculated for each quality attribute.
Quality attribute pooling
46
General formulation of pooling • A general form of a spatial pooling approach is given by
– where wi is the weight given to the ith location and mi is the quality
measure of the ith location. – M is the pooled quality value.
• Most spatial pooling methods can be formulated in this way. • In a simple average pooling method, wi is the same over the image
space. • Pooling can be divided into two categories:
– Quality based pooling – Content based pooling
47
Quality based pooling • Quality based methods assume that the weights wi are
related to the quality value mi at the ith location of the quality map, i.e.
• These methods follow the principle that low quality values should be weighted more heavily compared to higher quality values.
• Common approaches: – Minkowski pooling – Monotonic function pooling (Wang and Shang) – Percentile pooling (Moorthy and Bovik)
48
Z. Wang and X. Shang. Spatial pooling strategies for perceptual image quality assessment. In International Conference on Image Processing, pages 2945–2948, Atlanta, GA, Oct 2006. IEEE. A. K. Moorthy and A. C. Bovik. Perceptually significant spatial pooling techniques forimage quality assessment. In D. E. Rogowitz and T. N. Pappas, editors, Human Vision and Electronic Imaging XIV, volume 7240 of Proceedings of SPIE, page 724012, San Jose, CA, Jan 2009.
Content based pooling • Content based methods assume that the weights wi might be related to
image content in the local region around the ith pixel.
• where ci is a measure of perceptual significance of image content in the local region around the ith location.
• Assumption: an error that appears on a perceptually significant region is much more annoying than a distortion appearing in an inconspicuous area.
• Common methods: – Information-content weighting pooling – Gaze based pooling – Saliency pooling
49
Evaluation of pooling techniques • Comparing the results from different metrics with
different pooling methods against perceptual data. – Gong, M. & Pedersen, M. Spatial Pooling for Measuring
Color Printing Quality Attributes. Journal of Visual Communication and Image Representation, 2012, 23, 685-696. • 25/09/12: http://www.sciencedirect.com/science/article/pii/S1047320312000600
• The overall results indicate that: – Pooling parameters are important. – Pooling is metric dependent.
50
Synopsis • What is an image quality metric • Classification of metrics
– Mathematically based metrics – Low-level based metrics – High-level based metrics
• Important factors for metrics – Masking – Pooling
• Evaluation of metrics • Image quality attributes
Introduction: evaluation of metrics
• In order to know if an image quality metric correlates with the human percept, some kind of evaluation of the metric is required.
• The most common to compare the results of the metrics to the results of human observers.
52
Pair comparison • In pair comparison experiments observers judge quality
based on a comparison of image pairs, i.e which image in the pair is the best according to a given criterion – For example which has the highest quality or is the least
different from an original.
• These experiments can be either – forced-choice, where the observer needs to give an answer, or – the observer is not forced to make a decision and may judge
the two reproductions as equals (tie).
• No information on the distance between the images is recorded, making it less precise than category judgment, but less complex.
• Pair comparison is the most popular method to evaluate e.g. gamut mapping*, and is often preferred due to its simplicity, requiring little knowledge by the user.
* CIE. Guidelines for the evaluation of gamut mapping algorithms. Technical Report ISBN: 3-901-906-26-6, CIE TC8-03, 156:2004. 53
Example pair comparison experiment
• For the first trial the observer judged the left patch to be closer to the reference, the same with the second trial, and in the third trial the right. The observer judges all combinations of pairs.
54
Category judgement • In category judgment the observer is
instructed to judge an image according to a criterion, and the image is assigned to a category.
• Five or seven categories are commonly used, with or without a description of the categories.
• One advantage of category judgment is that information on the distance between images is recorded, but the task is more complex than pair comparison for the observers.
• Category judgment experiments are often faster than pair comparison, with fewer comparisons necessary. 55
Category judgment experiment
Reference
Test set
50 30 70 50
30
Trial 1
Categories: 1-7
70 40 50 60 4 2 1 2 4
56
Rank order
• The observer is presented with a number of images, who is asked to rank them based on a given criterion.
• Rank order can be compared to doing a pair comparison of all images simultaneously.
• If the number of images is high, the task quickly becomes challenging to the observer.
• However, it is a fast way of judging many images and a simple type of experiment to implement.
57
Rank order example • The observer ranks the reproductions from
best to worst according to a given criteria.
58
Correlation • The most common measure
of correlation is the Pearson product-moment correlation coefficient – a linear correlation between
two variables (X and Y)
– The correlation value r is between −1 and +1.
b
Reproduction Original
59
Non-linear correlation • Metric scores might not linearly fit the results from observers. • Solution: non-linear fitting.
– Sheikh et al. proposed a 5-parameter logistic function:
• Various number
of parameters used by different researchers.
• Overfitting can be a problem.
60 H.R. Sheikh et al., A statistical evaluation of recent full reference image quality assessment algorithms, IEEE Trans. Image Processing, vol. 15, no. 11, pp. 3440-3451, 2006“ Image from http://sse.tongji.edu.cn/linzhang/IQA/IQA.htm (04/07/13)
Other performance measures • Rank correlation (Spearman and Kendall Tau) • Root-Mean-Squared-Error • F-statistic for comparing the variance of two sets of sample
points. • Outlier ratio (percentage of the number of predictions
outside the range of ±2 times of the standard deviations) of the predictions. – Requires access to the individual scores, which is normally not
given in databases.
• Rank order method (Pedersen and Hardeberg, CGIV, 2007)
Video Quality Experts Group. FINAL REPORT FROM THE VIDEO QUALITY EXPERTS GROUP ON THE VALIDATION OF OBJECTIVE MODELS OF MULTIMEDIA QUALITY ASSESSMENT, PHASE I. 2008 Pedersen, M. & Hardeberg, J. Y. Rank Order and Image Difference Metrics 4th European Conference on Colour in Graphics, Imaging, and Vision (CGIV), IS&T, 2008, 120-125
61
Example - evaluation of metrics
• Evaluation of metrics is very important to ensure their performance.
• Requires a database of images and corresponding subjective scores.
• Use an existing database or create a new database.
62
Existing image quality databases
63 Thanks to Xinwei Liu for putting together the table.
Name
CID
:IQ
TID LIVE (R
elease 2
)
Toyam
a
CPIQ IRCCyN/IVC
VC
L@FER
VA
IQ
TUD
JPEG
XR
HTI
IBB
I
MM
SP 3
D
A5
7
WIQ
TID2
01
3
TID2
00
8
CSIQ
DR
IQ
IVC
Watermarking 3D
image
Art im
age
TUD
1
TUD
2 En
rico
Bro
ken
Arro
ws
Fou
rier Su
bb
and
Meerw
ald
Year 2014 2013 2008 2006 2008 2010 2012 2005 2007 2009 2009 2009 2008 2009 2011 2009 2010 2010 2011 2011 2011 2010 2007 2009
Color or Gray Color Color Color Color Color Color Color Color Gray Gray Gray Gray Color Color Color Color Color Color Color Color Color Color Gray Gray
Number of reference
image 23 25 25 29 14 30 26 10 5 10 5 12 6 8 23 42 8 11 10 12 12 9 3 7
Number of distortion type 6 24 17 5 5 6 3 5 10 2 6 2 15 3 4 1 1 1 1 1 6 1
Number of distortion level 5 5 4 X 6 5 5 2 6 7 5 1 5 6 2 4 6 5 5 3 X
Number of image 690 3000 1725 808 196 896 104 195 105 130 315 132 96 120 575 42 16 55 60 60 60 60 54 80
Number of observer 17 985 838 29 16 35 9 15 16 17 7 14
No Specif
y 20 118 15 12 20
No Specif
y 18 18 20 7 30
Our evaluation of metrics • 6 state-of-the-art databases.
– Compression, gamut mapping, noise, contrast, color, etc.
• 22 state of the art metrics selected. – SSIM, S-CIELAB, VSNR, SHAME, PSNR, etc.
• Compare the results from the observers to the quality values from the metrics. – Correlation as performance measure.
64 M Pedersen, JY Hardeberg. Full-Reference Image Quality Metrics: Classification and Evaluation. Foundations and Trends® in Computer Graphics and Vision 7 (1), 1-80
Evaluation results • Results show that performance depends on:
– Images, type of distortion, and magnitude of the distortion.
• Metrics perform better for simple and single distortions, and worse for complex and multiple distortions.
65
-0,15
0,35
0,85
∆E*ab SHAME S-CIELAB PSNR SSIM VSNR
Pear
son
co
rrel
atio
n
IVC database, Le Callet et al. Gamut mapped images, Dugay et al.
Luminance changed images, Pedersen et al. JPEG and JPEG2000 compressed images, Caracciolo et al.
Images altered in contrast, lightness, and saturation, Ajagamelle et al. TID2008, Ponomarenko et al.
Evaluation
• CID:IQ database (www.colourlab.no/CID)
• 60 image quality metrics
• Results from 50 cm viewing distance
• Compare the results from the observers to the quality values from the metrics.
– Correlation as performance measure.
66 Marius Pedersen. EVALUATION OF 60 FULL-REFERENCE IMAGE QUALITY METRICS ON THE CID:IQ. International Conference on Image Processing (ICIP). 5 pages. September 2015. Quebec, Canada.
Linear Pearson correlation 50 cm
• CID has the highest correlation coefficient, but it not statistically significantly different from many other metrics, such as MAD, WSSI, colorPSNRHA, and VIF.
67
Non-linear Pearson correlation 50 cm
• WSSI has the highest Pearson correlation coefficient, but it is not statistically significantly different from MSSIM. The highest performing color metric is CID.
68
Synopsis • What is an image quality metric • Classification of metrics
– Mathematically based metrics – Low-level based metrics – High-level based metrics
• Important factors for metrics – Masking – Pooling
• Evaluation of metrics • Image quality attributes
One metric for overall quality?
• Researchers still search for «the holy grail»: – one metric to measure overall quality.
• However, image quality is complex, and one metric might not be suitable to measure all aspects.
• Solution: – Divide overall quality into quality attributes.
• Image quality attributes = terms of perception – Sharpness, contrast, color, etc.
70
Subset of quality attributes – CPQAs • Pedersen et al. proposed six Color Printing Quality Attributes (CPQAs):
– Color contains aspects related to color, such as hue, saturation, and color rendition, except lightness.
– Lightness is considered so perceptually important that it is beneficial to separate it from the color CPQA. Lightness will range from ”light” to ”dark”.
– Contrast can be described as the perceived magnitude of visually meaningful differences, global and local, in lightness and chromaticity within the image.
– Sharpness is related to the clarity of details and definition of edges. – In color printing some artifacts can be perceived in the resulting image. These artifacts, like
noise, contouring, and banding, contribute to degrading the quality of an image if detectable. – The physical CPQA contains all physical parameters that affect quality, such as paper properties
and gloss.
• Even though these are made for printing, they are general enough to be used in other areas; i.e. display.
• The selection of metrics must be based on the proporties of the attributes. – I.e. for sharpness the metrics should account for details and edges.
Pedersen, M.; Bonnier, N.; Hardeberg, J. Y. & Albregtsen, F. Attributes of Image Quality for Color Prints. Journal of Electronic Imaging, 2010, 19, 011016-1-13 71
Evaluation of printer workflows
• Using the quality attributes proposed by Pedersen et al. (2010)
• Suitable metrics for each of the attributes were found.
• Four different printers evaluated.
• Details can be found in – Pedersen, M. Image quality metrics for the evaluation
of printing workflows. University of Oslo, 2011
Pedersen, M.; Bonnier, N.; Hardeberg, J. Y. & Albregtsen, F. Attributes of Image Quality for Color Prints Journal of Electronic Imaging, 2010, 19, 011016-1-13
72
Framework
• Creating a digital version of the printed image.
– Using the framework by Pedersen and Amirshahi.
Print the images
Scan Perform
registration
Calculate metrics for different attributes
Visualize results
Pedersen, M. & Amirshahi, S. A. Framework the evaluation of color prints using image quality metrics. 5th European Conference on Colour in Graphics, Imaging, and Vision (CGIV), IS&T, 2010, 75-82
73
Sharpness
• Visualization of results are done with spider plots.
74 Pedersen, M. Image quality metrics for the evaluation of printing workflows. PhD thesis. University of Oslo, 2011
Noise
75 Pedersen, M. Image quality metrics for the evaluation of printing workflows. PhD thesis. University of Oslo, 2011
Color
76 Pedersen, M. Image quality metrics for the evaluation of printing workflows. PhD thesis. University of Oslo, 2011
Evaluation of projection systems
• Similar to the printing evaluation, but using a camera instead of a scanner.
77
Ping Zhao, and Marius Pedersen. Measuring Perceived Sharpness of Projection Displays with A Calibrated Camera. Submitted. Ping Zhao, Marius Pedersen, Jon Yngve Hardeberg, and Jean-Baptiste Thomas . Measuring the Relative Image Contrast Of Projection Displays. Journal of Imaging Science and Technology (JIST), Volume 59, Issue 3, Page 030404-1-030404-13, Society for Imaging Science and Technology, May, 2015. Ping Zhao, Marius Pedersen, Jon Yngve Hardeberg, and Jean-Baptiste Thomas. Image Registration for Quality Assessment of Projection Displays. Published in Proceedings of 21st International Conference on Image Processing (ICIP 2014), Page 3488-3492, Paris, France, October, 2014.
Thank you for your attention
Contact information: Marius Pedersen
E-mail: [email protected]
Web: www.colourlab.no
Phone: (+47) 61 13 52 46
Mobile: (+47) 93 63 43 85