No-reference Blur Estimation Based on the Average ConeRatio in the Wavelet Domain

Ljiljana Platisa, Aleksandra Pizurica, Ewout Vansteenkiste and Wilfried Philips

Ghent University, Department of Telecommunications and Information ProcessingTELIN-IPI-IBBT, St-Pietersnieuwstraat 41, B-9000 Ghent, Belgium

ABSTRACT

With extensive technological advancements in electronic imaging today, high image quality is becoming animperative necessity in the modern imaging systems. An important part of quality assurance are techniquesfor measuring the level of image distortion. Recently, we proposed a wavelet based metric of blurriness in thedigital images named CogACR. The metric is highly robust to noise and able to distinguish between a greatrange of blurriness. Also, it can be used either when the reference degradation-free image is available or whenit is unknown. However, the metric is content sensitive and thus in a no-reference scenario it was not fullyautomated. In this paper, we further investigate this problem. First, we propose a method to classify imagesbased on edge content similarity. Next, we use this method to automate the CogACR estimation of blur ina no-reference scenario. Our results indicate high accuracy of the method for a range of natural scene imagesdistorted with the out-of-focus blur. Within the considered range of blur radius of 0 to 10 pixels, varied in stepsof 0.25 pixels, the proposed method estimates the blur radius with an absolute error of up to 1 pixel in 80 to90% of the images.

Keywords: blur estimation, image quality, wavelets

1. INTRODUCTION

With the continuous advancement of digital imaging technologies, assessment and improvement of image qualitybecome indispensable parts of the imaging systems. The quality of an image gets distorted in various stagesof the imaging chain, starting from the acquisition process, through various image processing steps, up to thedisplay or printing phase. In this work, we are mainly interested in measuring the blurriness of the image whichoften occurs already in the stage of image acquisition. The blur can be observed as a loss of sharpness in theedges and other image discontinuities.

In particular, we investigate the out-of-focus blur, or circular blur. This type of blur is commonly causedby focus problems with digital cameras and lens abberations.1 Considering this, it is not surprising that theout-of-focus blur frequently appears in the digital images of natural scenes, especially when these are taken withnon-specialized cameras and by non professional photographers. To make things more difficult, the blur oftenappears together with other artifacts such as noise.

Estimating the amount of blur (or in this particular case defocus) is of crucial importance for various tasks.For example, image restoration algorithms that perform image deblurring and denoising rely on estimates of thedegradation parameters. Further on, for evaluating human perception of image quality it is necessary to relatevisual impressions to accurate quantitative measurements of blurriness. This is important not only for theoreticalpsychovisual studies but also for practical video distribution systems where it is usually needed to compromisebetween different image degradations such that the visual experience of the end users is maximized.2 Moreover,estimating the amount of defocus appears interesting for depth estimation in the image.3 In all these cases it isof interest to estimate the bluriness independently of the other degradations in the image (such as noise).

It is common to differentiate between two scenarios in which the image degradation can be assessed: a fullreference (FR) scenario in which the reference, or non-degraded, image is also available, and a no-reference (NR)scenario in which the reference image is not available.

A. Pizurica is a postdoctoral researcher of the Fund for the Scientific Research in Flanders, FWO.Send correspondence to Ljiljana Platisa, e-Mail: Ljiljana.Platisa@Telin.UGent.be

The literature reports on a number of methods for NR blur identification. While these are either waveletbased or not, many of them4–8 rely on the results of edge detection in the degraded image. The main problemthere stems from the techniques that are used for edge detection and their sensitivity to noise. This is especially ofinterest when the degraded image is contaminated with blur and noise at the same time. Then, the performanceof the NR blur metric gets affected by the noise in the degraded image and its accuracy drops significantly.

Recently, the authors of this paper proposed a blur metric named CogACR.9 The metric is based on thewavelet decomposition and histogram of the average cone ratio (ACR) defined by Pizurica et al.10 The CogACRmetric has two important advantages: (1) it is nearly insensitive to noise, and (2) it can be used as either an FRor as an NR metric. However, original design of the method9 is not fully automated in the NR scenario. Thisproblem is addressed in the current paper.

The contributions of this paper are twofold: (1) we propose a novel image similarity metric based on theedge content of the images, and (2) we introduce a novel method for NR blur identification. Both techniques arebuilt around the CogACR metric and they are tested for 8 different categories of natural scene images.11 Ourexperimental results indicate high accuracy of the proposed NR blur estimation method for all 8 data categories.

2. BACKGROUND

2.1 Digital image blur

In general, when a two-dimensional image f(x, y) is degraded by blur and corrupted by random noise, thedegraded image g(x, y) can be described as

g(x, y) = f(x, y) ∗ h(x, y) + n(x, y). (1)

Here, h(x, y) stands for the blurring function that acts on a degradation-free image, * denotes the two-dimensionallinear convolution and n(x, y) is the noise contribution at the corresponding position (x, y). We remark thatfor the scope of this study, the noise component will not be specifically addressed since our previous work hasproven high robustness of the CogACR metric to noise. Hence, g(x, y) = f(x, y) ∗ h(x, y). For more details, thereader is referred to Ref.9.

The experiments in the current study are focused on out-of-focus blur. To model the blur, we use a circularaveraging filter (pillbox) within the square matrix of side 2r + 1, where r is the radius of circular blur:

h(x, y) =

{0,

√x2 + y2 > r

1/(πr2), otherwise. (2)

Blur is usually perceived as a loss of sharpness in the image, i.e. smoothening of the edges. In Figure 1 weillustrate the effect of blurring for four different types of edges known in the literature:6,12 Dirac edge structure,roof structure, and two step structures, A-step and G-step. For all four edge types, two different levels of out-of-focus blur are considered: one lower level of blur causing minor deterioration in perception of the image (r8 = 2),and a higher one resulting in more serious degradation of the perceived image quality (r20 = 5). It is clear fromthese plots that the influence of blur is most pronounced in case of Dirac-like edges, already with the low level ofblur. The remaining edge structures get significantly degraded only with the higher level of blur. Consequently,we expect the fine edges in the image to be most affected by the blur. Examples of the corresponding blur-freeand blurry images are shown in Figure 2. Here, the “Cactus” image illustrates an image with a great number ofDirac edge structures while the “Face” image is mainly comprised of step edge structures. At lower blur level,Figure 2(b),(e), the “Face” image appears less degraded than “Cactus” image, which is in line with our earlierexpectation about the relationship between edge types and perceived level of image blurriness.

2.2 Average cone ratio, ACR

We have established by now that blur is determined by the sharpness of edges in an image. Mathematically,the edges are characterized as singularities. Significant advancements in singularity detection using wavelets andanalysis of singularities in the wavelet domain were reported in the works of Jaffard,13 Mallat and Hwang,14

a) Dirac Structure b) Roof Structure c) A−Step Structure d) G−Step Structure

Ref. edger8 = 2

r20

= 5

Figure 1. Influence of blur on four different types of edges:6 (a) Dirac structure, (b) roof structure, (c) A-step structure,and (d) G-step structure. Each graph depicts three conditions: a reference or non-degraded edge, and the same edgeblurred with two different levels of out-of-focus blur, one with the radius of r8 = 2 and the other with r20 = 5.

(a) (b) (c)

(d) (e) (f)

Figure 2. Influence of blur on different type of image content: “Cactus” image with more high frequency content, and“Face” image with prevalent low frequency content. (a), (d) degradation-free images; (b), (e) images distorted without-of-focus blur of radius r = 2 pixels; (c), (f) images distorted with out-of-focus blur of radius r = 5 pixels.

Mallat and Zhong,15 Malfait and Roose,16 and Hsung et al.17 They explained mathematical characterization ofsingularities using Lipshitz exponent α and proposed early techniques for estimation of α using multiscale imagerepresentation.

Following these efforts, Pizurica et al.10 proposed a wavelet based technique to estimate local Lipshitzexponents of the irregularities which has major advantage of high robustness to noise. The metric is namedaverage cone ratio (ACR). For each spatial position l, the ACR measure between two dyadic scales 2n and 2k,where n, k ∈ Z and k ≥ n + 1, is defined as

βn→k,l , log2

1

k − n

k−1∑

j=n

|Ij+1,l||Ij,l|

, Ij,l ,

∑

m∈C(j,l)

|wj,m|. (3)

Here, C(j, l) is used to denote a discrete set of wavelet coefficients at the resolution scale 2l which belong

Figure 3. Flowchart of the proposed CogACR image blur metric in a no-reference environment (the degradation-free imageis not available).

to the cone of influence of the analyzed position. Thus, the ACR can be seen as a measure of evolution of thewavelet coefficients across and inside a cone of influence centered at a given spatial position.

As a result of its ability to accurately estimate the local Lipshitz exponent, in particular Lipshitz α+1 whilebeing nearly insensitive to noise, the ACR metric was selected by Ilic et al.9 and later Lukic et al.18 as a basisfor a novel blur identification metric named CogACR. The details are discussed in the next section.

3. METHOD

3.1 Blur metric, CogACR

The algorithm for calculating the metric of image blurriness, CogACR, is illustrated in Figure 3. Once in thewavelet domain, we first aim to select the set of spatial positions in the image for which we want to calculatethe ACR measure, the edge positions in the image. This is done by thresholding the wavelet coefficients of theimage such to preserve a given percent, Tp, of those with the highest magnitude. In line with the discussion inRef.,9 the value of Tp is kept the same for all experiments in this study, Tp = 10%. The ACR values are thencalculated for all selected edge positions, in particular the ACR between scales 22 and 24. Finally, the center ofgravity of the ACR histograms are used to evaluate the level of blur in the image. We refer to this metric asCogACR.

Figure 4 illustrates ACR histograms and CogACR for the two images shown in Figure 2, the images of “Face”and “Cactus”. The scores are shown for the blur-free images as well as for three different levels of blur, r8 = 2,r20 = 5, and r40 = 10. We notice in Figure 4(a),(b) that the center of gravity of ACR histograms shift with theincrease of blur in the image. This allows the CogACR metric to clearly distinguish between a wide range ofblurriness in the image, as we can see in the graph (c) of Figure 4.

As discussed in detail in,9 the ACR and hence the CogACR metric are sensitive to the content of image data.Looking at Figure 4(a), (b), we notice that the histogram of ACR coefficients of the “Cactus” image is differentin shape from the one of the “Face” image. This suggests different characteristics of the prevailing type of edgesin the images. In this case, the “Cactus” image has more high frequency content compared to the “Lady” imagewhich corresponds to fewer strong step edges in the “Cactus” image and more Dirac edge structures. Thisobservation is reflected in the trends in CogACR metric depicted in Figure 4(c). There, at the lower levels ofblur (r = 1 to r ≈ 6), the CogACR is more sensitive to the changes of blur the “Cactus” image than to thosein the “Lady” image. As the level of blur increases, the difference between shape of ACR histograms, as well asthe CogACR trends for the two images, gradually get diminished. This behavior is in line with the discussion inSection 2.1 about the influence of blur on different types of edges.

3.2 Novel similarity measure

Unlike many existing image similarity measures that draw from the contextual information in the image, thenovel similarity measure is built around the edge-related information in the image. It is designed to distinguishbetween images with similar edge characteristics.

We assume there are to non-overlapping data sets: a set of training images, X = {xi ∈ X : i = 1, 2, ..., NTR},and a set of testing images, Y = {yj ∈ Y : j = 1, 2, ..., NTS}. Here, NTR and NTS are used to denote thenumber of images in the training and in the testing set, respectively.

−0.5 0 0.5 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

ACR

−0.5 0 0.5 1ACR

Ref. image r8 = 2

r20

= 5

r40

= 10

0 2 4 6 8 100.8

1

1.2

1.4

1.6

1.8

2

Radius of blur, r

Cog

AC

R

(a) (b) (c)

’Face’

’Cactus’’Face’’Cactus’

Figure 4. (a) Histograms of ACR coefficients for “Cactus” image; (b) histograms of ACR coefficients for “Face” image.The plots in (a) and (b) are shown for degradation-free images as well as for the images distorted with 3 different levels ofout-of-focus blur, r = {2, 5, 10} pixels. (c) CogACR curves for “Cactus” and “Face” images over a range of out-of-focusblur distortions, r = {0.25, ..., 10} pixels.

For a given test image, y, we are interested to find the image, x, from the training set which is most similarto it. In that sense, the problem can be seen as the best match problem, or a well-known image dictionarymatching problem.19 In our case, the similarity criteria is twofold: (1) the CogACR values of the two imagesare similar, and (2) the ACR histograms of the two images are similar. Thus, our algorithm for finding bestmatching image consists of two stages, each focusing on one similarity requirement: (1) a candidate selectionstage, and (2) a candidate verification stage.

Candidate selection stage. Before starting the search for the best matching image in the training set, webuild a dictionary D. The entries, E, of the dictionary are comprised of ordered pairs in which the first elementis histogram h of the ACR coefficients in the edge positions of the training image x, and the second element isthe corresponding radius of blur, r = r(x).

In the remainder of this section, when the discussion is not explicitly related to the classification problem,we use g to denote an arbitrary image, either g = x or g = y. Further on, for simplification of the notation,we use h to denote the vector of frequency counts in the histogram of ACR coefficients of an image g, h =h(g) = h(ACR(g)). The number of bins used in histogram computation is denoted by Nbins. Likewise, weuse CogACR(g) to refer to the center of gravity of the ACR histogram of image g , that is CogACR(g) =CogACR(h(g)). Last, the similar notation shall be used to denote the CogACR of the ACR histogram in thedictionary element Ei, CogACR(Ei) = CogACR(h(xi)).

Given the preceding notation, the dictionary D with a total of NTR entries Ep can be described as

D ={Ep ∈ RNbins+1 : Ep = (h(xp), rp) , p = 1, 2, ..., NTR

}. (4)

Now, we take a test image y and search the dictionary for all entries that satisfy the first similarity criterion:the CogACR of the test image is similar to the CogACR of the entry in D. As a similarity measure, dC , betweenCogACR values of the two images, g1 and g2, we use simple Euclidean distance

dC(g1,g2) = |CogACR(g1)− CogACR(g2)| . (5)

The subset of dictionary entries which fall in the TC neighborhood of the CogACR(y) constitutes the candidateset, C, of size NC . This can be described as

C ={Cq ∈ RNbins+1 : dC(y,xq) ≤ TC , q = 1, 2, ..., NC ; NC ≤ NTR

}. (6)

Clearly, the smaller the threshold value of TC the less number of candidates NC . The influence of TC on theclassification process is further discussed in Sections 4.4 and 5.

As remarked in Ref.,19 the candidate selection stage of image dictionary matching problem can be seen asa variation of the k -nearest neighbor search problem. Namely, similar to the k -nearest neighbor problem wherethe goal is to find k -nearest neighbors in the reference set, the candidate selection problem we described is aimedto find all dictionary entries whose distances are within a neighborhood determined by the threshold TC .

Candidate verification stage. In this stage, the distances between ACR histograms of the test image andthe candidates are computed in order to verify the candidates and select the best matching one. To measurethe distance between two histograms, we explored three common metrics known in the literature: histogramintersection, Kullback–Leibler distance (KL), and symmetrical Kullback-Leibler distance (SKL). The metricwhich proved best suited to the task is the SKL distance defined as

dSKL(h1,h2) =12

Nbins∑

b=1

(h1b − h2b) log(

h1b

h2b

). (7)

Here, b denotes the b-th bin of histogram h while hb is used to denote the count in the bin b, and b = 1, 2, ..., Nbins.Eventually, the candidate whose dSKL distance to the test image is the smallest is selected as the best match.

3.3 Novel NR blur estimation methodThe proposed method for automated no-reference blur identification relies on the image similarity measuredefined in the previous subsection. A schematic illustration of this method is given in Figure 5. The input andoutput of the method are:

Input: A test image y, a dictionary of ACR histograms D.Output: Estimated blur radius r.In this model, the value of the threshold TC can be seen as a configurable parameter of the system.The algorithm starts from its inputs and enters the candidate selection stage. Once it selects the set of

most similar candidates, given the threshold TC , it proceeds to the candidate verification stage to find the bestmatching candidate. The estimate of the blur radius in the input image y is retrieved from the selected bestmatch candidate entry. If we assume that the best match entry is Ep = (h(xp), rp), where p ≤ NTR, then theestimated radius of blur is r(y) = rp. For convenience, this later stage of searching for the best match withinthe candidate set and finding the corresponding blur radius r of the test image y can also be described by thefollowing formulation

r = r(y) = argminr∈C

(dSKL), (8)

where argmin denotes the argument of the minimum.

4. EXPERIMENTS AND RESULTS4.1 Experimental dataThe images we use in the experimental study are taken from the image collection proposed by Oliva and Tor-ralba11 and recently used by Moorthy and Bovik20 to assess statistics of natural image distortions. There are2688 natural scene images in this collection, split in 8 categories (databases): coast and beach (DB1), forest(DB2), highway (DB3), city center (DB4), mountain (DB5), open country (DB6), street (DB7), and tall build-ing (DB8). Each category consists of several hundreds of color images in size 256×256 pixels. The details aboutsize of each database, NDB , are mentioned in Table 1.

Firstly, we convert the original color images to grayscale. These are degradation-free data. Two such examplesfrom each category are shown in Figure 6. Next, we create the distorted images by introducing blur to the data.This is done using the out-of-focus blur model from Equation (2). Each image is distorted using NBL = 40different values of blur radius varied in the range from 0.25 pixels to 10.00 pixels and with a uniform step of0.25, r = {r ∈ R : rm = 0.25m, m = 1, 2, ..., NBL}.

The images in Figure 2 illustrate the perceived extent of image blurriness at r8 = 2 pixels and r20 = 5 pixels.Note that the ’Face’ and ’Cactus’ images are not in the 8 databases that we use in the following experiments.

Figure 5. Flowchart of the proposed technique for no-reference (NR) image blur identification based on the CogACRmetric.

4.2 CogACR performance

We start with a simple experiment to objectively evaluate the performance of the CogACR metric in the task ofestimating the radius of out-of-focus blur.

For all images in each DB1 to DB8, we compute the Spearman rank-order correlation coefficient (SROCC)between the CogACR scores and the actual blur radii. The experiments are carried out for degradation-freeimages together with their blur distorted realizations, thus a total of NDB(NBL + 1) images per database. Theresults are summarized in Table 1.

For the purpose of comparison, the SROCC is also computed for two existing image distortion metrics: thepeak signal-to-noise ratio (PSNR) and kurtosis which has been proposed as a measure of blur by Li et al.21

These calculations are done in exactly the same experimental setup as those for the CogACR metric. The resultsare included in Table 1.

Based on the SROCC, the CogACR appears more correlated to the blur radius than the other two metrics,and this for each of the 8 image categories. Furthermore, it exhibits a relatively high correlation to the level ofblurriness in the image, ranging from about 0.93 in case of DB1 up to over 0.98 in case of DB7 or DB2.

4.3 Best matching images

To illustrate the performance of the proposed image similarity measure, we run a set of experiments searchingfor the best match. The experiments are carried out with the degradation-free images.

For each image category, we select a random subset of NTR = 200 degradation-free images to build thedictionary and another non-overlapping subset of NTS = 50 degradation-free images to be used as the testimages. Using the similarity based method from Section 3.2, we select the best match from the dictionary foreach test image.

The results of these experiments are illustrated in Figure 6. We show one pair of the selected most similarimages for each of the 8 data categories. Among these, the greatest similarity is observed between the best matchimages from DB2 (Forest) and DB5 (Mountain), while the smallest one is found among the best match imagesfrom DB1 (Coast & Beach), DB6 (Open Country) and DB8 (Tall Building).

4.4 NR blur identification

Next, we perform an extensive experimental study to evaluate the proposed NR blur metric based on the CogACRand the novel similarity metric. The study involves a range of experiments performed for each image category

Table 1. Comparison of the performance of three image distortion metrics: PSNR, kurtosis and CogACR. The evaluationis done using the Spearman rank order correlation coefficients (SROCC). The calculations are performed for each of the8 image categories using a total of NDB(NBL + 1) images per data base: NDB blur-free images together with NBL theirblur distorted copies, where NBL = 40.

DB size PSNR Kurtosis CogACRDB ID DB name NDB SROCC SROCC SROCC

DB1 Coast & Beach 360 -0.5518 -0.0850 0.9346DB2 Forest 328 -0.4967 0.0185 0.9830DB3 Highway 260 -0.6421 -0.0902 0.9685DB4 City Center 308 -0.7053 0.0920 0.9722DB5 Mountain 374 -0.5742 -0.0178 0.9677DB6 Open Country 410 -0.5027 -0.0240 0.9693DB7 Street 292 -0.7316 0.0759 0.9807DB8 Tall Building 356 -0.6039 0.0100 0.9499

DB1 DB2 DB3 DB4 DB5 DB6 DB7 DB8

Figure 6. The best match image pairs for each of the 8 image databases used in the study. The images are taken fromthe image collection proposed in11 which consists of 2688 color natural scene images split in 8 categories: coast and beach(DB1), forest (DB2), highway (DB3), city center (DB4), mountain (DB5), open country (DB6), street (DB7), and tallbuilding (DB8). All images are converted to gray scale and kept in the original size of 256 × 256 pixels.

separately. Each experiment is characterized by the following list of parameters: training set X of the size NTR,testing set Y of the size NTS , the number of the considered blur levels NBL, and the value of the thresholdparameter TC .

First, from each database DBi, i = 1, 2, ..., 8, we select two random subsets of images to build the trainingand the testing set, X and Y, respectively. These data splits are done such that there is no overlap between thetwo subsets. Also, note here that in these experiments the degradation-free and its corresponding blur distortedimages are kept together. That is, for the purpose of describing the experiment design, we assume that a subsetof NRef degradation-free images will also include the corresponding NRefNBL of the blurry images, where NBL

is the number of considered blur levels (different blur radii). This considered, the size of each training set isNTR = NRefTR(NBL +1) and the size of each testing set is NTS = NRefTS(NBL +1). In particular, we considerthe following parameter values: NRefTR ∈ {50, 100, 150, 200, 250, 300, 350} and it is chosen depending on thesize of the given database, NRefTS = 50, while the number of blur levels is fixed at NBL = 40.

Next, for each training set X, we compute the ACR histograms and build the dictionary D. This completesthe set of necessary inputs for NR blur estimation algorithm described in Section 3.3 and Figure 5.

Now, we run the NR blur estimation procedure. For each of the NTS test images, we extract the candidate setof entries and find the best matching candidate. The radius of blur corresponding to the best match candidateis the estimated radius of blur for the given test image, r. In these experiments, the value of the threshold TC

75

80

85

90

PC

[%]

∆ ≤ 1

10

15

20P

C [%

]1 < ∆ ≤ 2

50 100 150 200 250 300 3500

2

4

6

Number of training images, NTR

PC

[%]

∆ > 2

Tc = 5 10−3 Tc = 5 10−4 Tc = 5 10−5

Figure 7. Performance of the proposed NR blur estimation algorithm evaluated by comparing the percent correct (PC)on 7 different sizes of the training dataset, with NRefTR={50, 100, 150, 200, 250, 300, 350} and on 3 different thresholdvalues for candidate selection, TC={0.00005, 0.0005, 0.005}. The number of reference test images is kept constant in allexperiments, NRefTS = 50, while the number of different blur radii is NBL = 40. The three plots differ in the value ofabsolute error ∆ of the estimated blur radius r: (top) ∆ ≤ 1 pixel, (middle) 1pixel < ∆ ≤ 2 pixels, (bottom) ∆ > 2pixels. The scores are shown for image category of Open Country (DB6). Error bars indicate the ± standard deviationof the scores across 5 different random selections of training and testing image subsets. In each of the 5 selections, nooverlap exists between the trainer images and the tester images.

used in selecting the subset of candidates is varied among three values, TC ∈ {0.00005, 0.0005, 0.005}. All threeTC values are considered for each dictionary D and each testing image set Y.

The results of all experiments for the largest among 8 image database, DB6 (Open Country), are depicted inFigure 7. In the performance analysis of the proposed NR blur metric, we use as a figure of merit the absoluteerror between the estimated blur radius and the actual blur radius, ∆ = |r − r∗|. Here, r is the estimated valueof blur radius and r∗ is the actual value of blur radius. There, we distinguish three conditions: (1) ∆ ≤ 1 pixel,(2) 1 pixel < ∆ ≤ 2 pixels, and (3) ∆ > 2 pixels. The top, middle and bottom plots in Figure 7, respectively,correspond to these three ranges of the absolute error in blur radius estimation. Different lines in each plotrepresent different value of the threshold TC .

Importantly, in order to avoid bias of the results and allow for their statistical significance to be evaluated,for each parameter setup of NRefTR and NRefTS , we explore 5 different random realizations of data grouping,training versus testing data. We then use the scores from these 5 realizations to estimate the error bars for ourresults. In Figure 7, these are shown as ± standard deviation of the scores across 5 random initializations ofeach experiment.

The results for DB6 which indicate that the highest and approximately stable performance of the NR blurmetric is achieved with NRefTR = 200 and TC = 0.005. In Figure 8, we use these parameters to illustrate theperformance of the method across 8 different image categories. The results are shown as a bar chart where colorsrepresent different ranges in accuracy of the method, similar as in Figure 7, and the error bars are ± standarddeviation of the scores across 5 random initializations of each experiment.

DB1 DB2 DB3 DB4 DB5 DB6 DB7 DB80

10

20

30

40

50

60

70

80

90

100

Image database identifier, DB ID

PC [

%]

∆ ≤ 1

1 < ∆ ≤ 2

∆ > 2

Figure 8. Performance of the proposed NR blur estimation algorithm evaluated by comparing the percent correct (PC) onall eight databases. For each database, the following parameter values are used: NRefTR = 200, NRefTS = 50, NBL = 40,and TC=0.005. For each data category, 5 different random selections of training and testing image subsets are considered.In each experiment, no overlap exists between the trainer images and the tester images. Error bars are ± standarddeviation of the scores across 5 data groupings.

The bars from Figure 8 indicate that in 6 out of 8 considered image categories the absolute error ∆ inestimated blur radius is less than or equal to 1 pixel in 85% or more of the considered cases of blur estimation.In addition, for all 8 databases, the percentage of cases in which ∆ > 2 pixels remains below 5%, and most oftenit is even below 3%.

5. DISCUSSION

As an important indication of the performance of the novel CogACR blur metric, we observed in Table 1 thatthe correlation of the metric and the image blur radius is relatively high, in most considered image setupsSROCC > 0.95. Compared to the PSNR, and especially to the kurtosis metric, performance of the CogACR isnotably better across all considered image databases.

It is interesting to note here that the CogACR and the PSNR do not follow the same relative trends. Forexample, CogACR is highest correlated to the level of blurriness in DB2 where the PSNR is least correlated,while for DB4 or DB7 both distortion metrics are highly correlated with image blur. One possible reason forthis difference in trends for the two metrics, PSNR and CogACR, could be the scope of image content whichthey use in their calculations. Namely, the PSNR is averaged over all image pixels while the CogACR is focusedon the strongest edges in the image. Knowing that it is the edges in the image who are primarily influenced byblur,8,22 it may be expected that CogACR is able to give more accurate predictions of the level of blurriness.

Related to the proposed method for NR blur identification and the threshold parameter of the candidateselection, TC , we noted from the results shown in Figure 7 that the greater TC allows higher performance ofthe method. We remind that the greater TC means less strict condition of the CogACR based similarity, dC , asdefined in Eq. 5. Then, the observed trend of greater TC resulting in greater accuracy of the blur estimation,may suggest that, once we are in the right neighborhood of CogACR values, TC neighborhood, the finer details

of the ACR histogram become of high importance. That is exactly the point in the algorithm where we measurethe similarity of ACR histograms, dSKL, as defined in Eq. 7. Again, this increased significance of all componentsof the ACR histogram, rather than only its center of gravity, point to the strong relationship between the contentof the image and the effect of blur that can be measured or even perceived.

To end with, one more interesting aspect to discuss is the influence of size and structure of the dictionaryon the performance of the proposed NR blur estimation. While always relatively high, we notice in Figure 8that the performance of the metric slightly varies among different image categories. One reason for this couldbe different sensitivity of the metric to different types of image content, as discussed earlier. In that sense, wenote that the NR blur metric approximately follows the trends in correlation coefficient of CogACR: the higherCogACR correlation, the higher percent of cases with high estimation accuracy, ∆ > 1.

Nevertheless, the oscillation in algorithm performance could also be due to the effect of the dictionary. Wenoted in Figure 7 that the size of dictionary affects the performance of the method, the larger the dictionary thehigher the performance, especially in the range of “smaller” dictionaries (NRefTR ≤ 200, that is NTR ≤ 8200).Next to its size, it is very important to know how well and how complete the dictionary represents the data. Incase of our method, one way to anticipate this would be looking at the statistics of dSKL obtained for the testingimages. For example, in the experiments from Figure 8, the mean value of the dSKL across all images in thetesting set of the given image category (not shown here) are exactly proportional to the accuracy of the methodin that image category: the higher the accuracy of the method, the smaller the average distance between thetest image and its best match.

6. CONCLUSION AND FUTURE WORK

The work we present in this paper addresses the problem of blur identification in the scenario where the referencedegradation-free image is not available. We start from our recently proposed blur metric named CogACR anduse it to develop a novel fully automated algorithm for no-reference blur estimation. As part of the algorithm,we propose also a novel measure of image similarity. In contrast to the existing image similarity measures whichare mainly context-based, the novel measure quantifies similarity of the edge-content in the images.

Our experimental results indicate high accuracy of the proposed NR blur metric over a wide range of outof focus blur. This, together with the earlier detailed analysis of the CogACR metric9 which proved its highrobustness to noise, suggest a promising potential for the proposed method of NR blur identification. Theseoptimistic expectations are further encouraged by our recent investigations18 indicating that the metric can beefficiently implemented on a commercially available processors where it achieves real-time performance even forHD video inputs.

Next to the more extensive testing and validation of the proposed method, our future efforts will be directedto the options for further improvement and efficiency of the classification methodology. One potential directionthere, similar to the approach of23 could be exploring the possibilities for SVM based design of the technique.

Finally, knowing that CogACR is sensitive to image content in the way which intuitively complies to thesensitivity of the human visual system, higher sensitivity to small distortions in high frequency image contentcompared to the low frequency one, there is a potential for using CogACR to build an algorithm for objectiveevaluation of the perceptual quality of images. Going in this direction, it might be worthwhile exploring theproposed metric performance on the salient regions of image24 rather than on the whole image area as we do itnow.

REFERENCES[1] R. L. Lagendijk and J. Biemond, “Basic methods for image restoration and identification.” Handbook of

Image and Video Processing 2nd edition, 1999.[2] I. Papp, N. Lukic, Z. Marceta, N. Teslic, and M. Schu, “Real-time video quality assessment platform,” in

Int. Conf. Consumer Electronics, ICCE ’09., pp. 1 –2, 10-14 2009.[3] A. Levin, R. Fergus, F. Durand, and W. T. Freeman, “Image and depth from a conventional camera with

a coded aperture,” in SIGGRAPH ’07, p. 70, (New York, NY, USA), 2007.

[4] F. Rooms, A. Pizurica, and W. Philips, “Estimating image blur in the wavelet domain,” in Proc. of theFifth Asian Conf. on Computer Vision, D. Suter and A. Bab-Hadiashar, eds., pp. 210–15 vol.1, 2002 2002.

[5] P. Marziliano, F. Dufaux, S. Winkler, and T. Ebrahimi, “A no-reference perceptual blur metric,” in IEEEInternat. Conf. Image Proc., 3, pp. 57–60, 2002.

[6] H. Tong, M. Li, H. Zhang, and C. Zhang, “Blur detection for digital images using wavelet transform,” inMultimedia and Expo, ICME ’04. IEEE Internat. Conf. on, 1, pp. 27–30, 2004.

[7] X. Wang, B. Tian, C. Liang, and D. Shi, “Blind image quality assessment for measuring image blur,” inImage and Signal Processing, 2008. CISP ’08. Congress on, 1, pp. 467 –470, 27-30 2008.

[8] T. Jeong, Y. Kim, and C. Lee, “No-reference image-quality metric based on blur radius and visual blocki-ness,” J. Opt. Engineering 49(4), p. 045001, 2010.

[9] L. Ilic, A. Pizurica, E. Vansteenkiste, and W. Philips, “Image blur estimation based on the average cone ofratio in the wavelet domain,” in Proc. SPIE - The Int. Soc. for Opt. Eng., 7248, p. 72480F (10 pp.), 2008.

[10] A. Pizurica, W. Philips, I. Lemahieu, and M. Acheroy, “A joint inter- and intrascale statistical model forbayesian wavelet based image denoising,” IEEE Trans. Image Proc 11, pp. 545–557, 2002.

[11] A. Oliva and A. Torralba, “Modeling the shape of the scene: A holistic representation of the spatialenvelope,” Internat. J. Computer Vision 42, pp. 145–175, 2001.

[12] Y. Tang, L. Yang, and J. Liu, “Characterization of dirac-structure edges with wavelet transform,” IEEETrans. on Systems, Man, and Cybernetics 30, pp. 93 –109, Feb 2000.

[13] S. Jaffard, “Pointwise smoothness, two-microlocalization and wavelet coefficients,” Publications Mathema-tiques 35, pp. 155–168, 1991.

[14] S. Mallat and W. Hwang, “Singularity detection and processing with wavelets,” IEEE Trans on InformationTheory 38, pp. 617–643, Mar 1992.

[15] S. Mallat and S. Zhong, “Characterization of Signals from Multiscale Edges,” IEEE Trans. Pattern Analysisand Machine Intelligence 14, pp. 710–732, Jul 1992.

[16] M. Malfait and D. Roose, “Wavelet-based image denoising using a Markov random field a priori model,”IEEE Trans. Image Proc. 6, pp. 549–565, Apr 1997.

[17] T. Hsung, D. Lun, and W. Siu, “Denoising by singularity detection,” IEEE Trans. Signal Proc. 47, pp. 3139–3144, Nov 1999.

[18] N. Lukic, L. Platisa, A. Pizurica, W. Philips, and M. Temerinac, “Real-time wavelet based blur estimationon cell BE platform,” in Proc. SPIE - The Internat. Soc. for Opt. Eng., 7535, p. 75350C (11 pp.), 2010.

[19] S.-H. Cha, “Efficient algorithms for image template and dictionary matching,” J. Math. Imaging Vis. 12(1),pp. 81–90, 2000.

[20] A. K. Moorthy and A. C. Bovik, “Statistics of natural image distortions,” in IEEE Internat. Conf. Acoustics,Speech and Signal Proc. (ICASSP), 2010.

[21] D. Li, R. Mersereau, and S. Simske, “Blur identification based on kurtosis minimization,” in IEEE Internat.Conf. Image Proc. (ICIP), Vols 1-5, pp. 109–112, 2005.

[22] Z. Wang, E. Simoncelli, and A. Bovik, “Multi-scale structural similarity for image quality assessment,” in37th Asilomar Conference on Signals, Systems & Computers, Matthews, MB, ed., 1,2, pp. 1398–1402, 2003.

[23] M.-J. Chen and A. Bovik, “No-reference image blur assessment using multiscale gradient,” in Internat.Workshop Quality of Multimedia Experience, QoMEx 2009, pp. 70 –74, 29-31 2009.

[24] A. Ninassi, O. Le Meur, P. Le Callet, and D. Barbba, “Does where you gaze on an image affect yourperception of quality? Applying visual attention to image quality metric,” in IEEE Internat. Conf. ImageProc, ICIP 2007, 2, pp. II –169 –II –172, sept. 2007.