10th International Conference on Information Science, Signal Processing and their Applications (ISSPA 2010)

EDGE-REGION INFORMATION WITH DISTORTED AND DISPLACED PIXELS MEASURE FOR IMAGE QUALITY EVALUATION

Zianou Ahmed Seghi/, Fella Hachou/ lInstitute of sciences and technology, academic centre of Abbes Laghrour Khenchela

zianou _ ahmed _ seghir@yahoo.fr, fhachouf@wissal.dz 2 Automatic Laboratory, Mentouri Constantine University, Algeria,

ABSTRACT

Evaluating the adequacy of a quality image for a given application (image processing applications) is a requisite. However, objective quality assessment is far from being a solved problem. In this paper, a measure for image quality assessment (IQA) is introduced after a brief review of previous work. A metric based on the distance between blocks of distorted and original image is proposed to overcome some of the limitations of existing approaches. Distorted pixel, edge information and visual regions of interest are computed. The obtained results illustrate the effectiveness of the suggested method.

Index Terms- IQA, displaced pixel, distorted pixel

1. INTRODUCTION

The main objective of IQA metrics is to provide an automatic and efficient system to evaluate visual quality. It is imperative that these measures exhibit good correlation with perception by the human visual system (HVS). The most widely used objective image quality metrics, namely mean square error (MSE) and peak signal to noise ratio (PSNR), as widely observed do not correlate well with human perception [1] besides requiring the original reference image to compute distortion. IQA measures are classified into two categories: subjective and objective. The firsts are subjective methods which based on human observer to indicate the quality of image. This one is time consuming and has a very strict definition of observational conditions. The seconds are objective methods which use tow images; distorted and reference (original) images to calculate an image quality value. The objective quality evaluation is easier and faster than the subjective one because observers are not needed.

In this work, the problem of IQA with reference image is reduced to edge information, visual region, distorted and displaced pixels. The experiment is carried out using sets of original images with its deformed versions from LIVE image quality database [2]. The performance of

978-1-4244-7167-6/101$26.00 ©2010 IEEE 77

the proposed quality metric is compared with the Gradient-based Structural Similarity (GSSIM) [4], Visual region of interest Weighted Quality Index (VroiWQI) [5] and Peak Signal to Noise Ratio (PSNR).

Rest of the paper is organized as follows. Some algorithms of image quality measures are presented in Section 2. Proposed image quality measure is presented next. The proposed method performance will be showed in Section 4. The section 5 is intended for the conclusion.

2. RELATED WORKS

Existing algorithms to assess image quality have used a variety of methods to do so. In [3], a new philosophy of the Mean Structural Similarity Index (MSSIM) is introduced. However, MSSIM fails in measuring badly blurred images [4]. In [4], an improved quality assessment is proposed; called GSSIM based on the edge information as the most important image structure information. This measure is very interesting in using edge information with MSSIM but, it cannot use for measuring quality of the region information. In [5], VroiWQI is introduced. The index is based on weighted quality indices of local regions that capture structural distortion in the local regions between the test image and the original image. VroiWQI is not interested in using edge information. In [7], an edge and region information with distorted pixel is introduced. This method is very interesting in using edge and region information with distorted pixel but, it cannot use for measuring quality of displaced pixels.

Motivated with these studies, we proposed in this paper edge-region information with distorted and displaced pixels measure (ERDM) which doesn't exploit a priori knowledge about the distorted image and the types of artifacts.

3. PROPOSED METHOD

Both images (original and distorted) have MxN pixels. Thus, the images are partitioned into overlapped 11 x 11 blocks, where the overlapping area is on one pixel (see Fig.l). The choice of 11 x 11 overlap is motivated by the subdivision the image on local 8x8 square window, often exhibits undesirable "blocking" artifacts problem [3]. Thus, this kind of problem is hidden, or mostly

disappeared, when the partition superior on 8x8 block of pixel is used. Let bkj and bki be blocks of distorted image and original image respectively.

llXll llX11 Partially overlapping block

,

Q { !x.� ��.�ix�:k:ix::-��;;x.:ix.::::x�.-�::. x;-';-:::: x;-;;-::::::-rx,-:-...... 'x.,1 X •• l)c-.. .,IX ... �.� Xa. '.ka,1 Xa. I)<' •• IIX ... X.n' ...... x ... I ...... XaN

�J!.� X""£��.!.!+�3:;-l�!.�_:!"!!.}�s.!-t�!:".��.!!.��� !!.!!!k= !!.!!L� M I IX., >5.�.I, IX.,. 1>.c�IX .. s .x. .... IX .. , 1JS'� .... IX .. . Ix::� ..... x.n,······ x •• li ...... X ... � IX.,I X.a !)(.aIlX .. !)c •• IX •• I¥ . . IIX •• !)c •• ux.,. X.nl. ..... x ... I ..... Xu.

tJ!.� �.!��.!.!l-t·:;J�!.·J!.!!.""l.x_ • .!.1.r���!.!.�.!� !!!.!!.l:"= !!.!!F=-� � { :x .. )(ujx •• :>5..u :x .. 1Nn .. 1x" I�:X ... :>.C�..I0 XU I······ Xu. I······ X u

IX., I ", ... .))( •• 11)( •• i)c •• ' .. x!.! k.".)( •. ijc •• IIX ... X.nl ...... X......... X ... IX .. I Ji' li#� .:{I k"".-; "XIS IX:' I )( . . �;; ¥.�.-.; X u ! ... · .. x . . !�XtN

� { ;;.�: ;��t���::���I�r���;:������:�: �;�:� �;;�� . . . . - .. _.

Ix •. x •• lx •• lx •. lx •• lx •• lx •• lx •• lx •• lx._ x· .. I···· ..... I····· x._ Fig.I. Image pixels and its centers of gravity

Pixel distortions are estimated using the following features. First, difference measure (DM) of a central pixel is estimated within 3 x3 neighborhood pixels which is applied for all available central pixels in the image. DM values are then averaged within a 3 x 3 partially overlapping block (see Fig.I). Let X23 be the centrals pixels of the 3 x3 pixels "neighborhood" in original image as shown in Fig. 1 also let Mj; be the mean of pixels within the 3 x3 pixels of the distorted image and Xj;j its intensities. All available central pixels of images (original) are located within the bolded line boundary as shown in Fig. 1. The mean can be estimated as follows:

3 3

Mfi = �LL Xfij (1) i�l j�l

The same procedure is applied on the original image. This one has also a mean � and J(j its intensities. DM between the two 3 x3 partially overlapping blocks of the original image and distorted image can be estimated as follows:

1

DM � (�t,t,(Mf' -x,J + (M, -Xi'll'),

(2)

The distorted pixels measurement Id between the two bki and bkj is defined by:

A

Id = �L DMi (3) i�l

Where A (A=IIxl1l3x3) denotes the total number of centrals pixels of the 3 x3pixels "neighborhood" within block of 11 x 11 pixel. All Id can be arrayed in a (m xn) matrix idg/ohaF [Idi.J ] (1 � i � m=MIll and 1 � j � n=N Il1).

In order to reflect the differences between two images at the local level, we calculated centers of gravity of the original and distorted images on the basis of 11 x 11 blocks. In this step, displaced pixels measure will be computed. This criterion works in three stages. The first stage consists of identifYing all centers of gravity of sub­blocks (3 x3 pixels "neighborhood" and 2x3 pixels "neighborhood' ') of bkj and bki. Thus, let sbkjsuh and sbkisuh be sub-blocks of distorted image and original image respectively. Furthermore, the distance between a center of gravity of sbkjsuh of bkj and the whole sub-

78

blocks of bki of the original image is measured. All available centers of gravity of image (original) are located within the dotted circles as shown in Fig. 1. The third step computes all displacement of sub-blocks center of gravity pixel related to bkj of the distorted image. This stage is based on computing the center of gravity of a sub-block defined by moments of order pp and qq:

endl end2 mmppqq = L L xxPPyyqq

xx�beginl yy�begin2 (4)

Where endl, end2 are the length and the width of a sub­block; and beginl and begin2 are the positions of the first pixel in the sub-block; for example beginl=1 and begin=1 are the position of the first pixel (Xll) of sub­block 3x3. The center of gravity is computed by:

_ (mm1o mmol) G- -- , --mmoo mmoo (5)

where mmoo is the area of the sub-block, mmlO is the sum of pixels positions xx that belong to the sub-block, mmOl is the sum of pixels positions yy, that belong to the sub­block. One defines this criterion for each 'db (x,y)' sub­block center of gravity pixel of the bkj; by using equation (6); where Gi is sub-block center of gravity pixel belongs to bki of the original image,j can be the intensity of the sub-blocks center of gravity pixel or any other attribute, d((x,y),(X'i,yj)) is Euclidian distance between two sub­blocks center of gravity pixel, x and y are the sub-block center of gravity pixel position of 'dp' in bkj, X'i and yj are a sub-block center of gravity pixel position of Gi in bki of original image, dlftJ(db(x,y), bki) is the distance between 'db(x,y)' center of gravity pixel of sbkjsub and bki, bkj ={ GiG (x'jQ, yjo,), Gil (X'jhyjl,), .. }.

diffi,j (db (x, y), bk;) fo if x = x; andy = y; and [(db) = f(G;) - min(position, color) if x "* x; and y "* y;

Where : postion = min (d ((x, y), (x;o' yj,)) , d ((x, y), (x;" y;,)), ... ), color = min( lf(db) - f( GiJI, I[(db) - f( GiJI, . . · ).

(6)

The following formula is used to calculate the displaced sub-block center of gravity pixel measure related to bkj of the distorted image:

L

Dp = �L diffi,j (dbj(x,y),bk;) (7) j�l

Where L (L=20) is the total number of sub-blocks of a bkj• All Dp can be arrayed in a (m xn) matrix Dpg/ohaF [DPij] (1 :'0 i:'O m=MIJJ and J :'OJ:'O n=NIJJ).

The overall distorted and displaced pixels evaluation between two blocks bki and bkj is obtained by combining the two results defined previously is given by: M B (! di,j' DPi,j) = M ax(! di,j' DPi,j ) (8)

In this stage, we have used the same function developed in [5] where the regions in the original image is modeled by computing the entropy "e" in each block of 11 x 11 pixels of the original image, using equation (9), where h is a random variable indicating intensity, p(h ) is the histogram of the intensity levels in a region, K is the number of possible intensity levels. K varies from 0 to 255 for gray scale images.

k e = - L p(ha log2 p(ha (9)

i=l All e can be arrayed in a (m xn) matrix E= [e;J] (1 :"0 i :"0 m=MIJJ and J :''OJ :"0 n=NIJJ), where each element e;,j is the entropy of the block. In this step, the formula for measuring the region structure comparison measures is as follows: sr( bki,bkj) = eij * s( bki,bkj) (10)

Where s(bk;,bk)=( lTbki bkj +C3)/( lTbki lTbkj+C3), C3=Cz/2 where C2=(K2L2), K2« 1, L2 is the dynamic range of the pixel values (255 for 8-bit grayscale images), abk; and abkj are the standard deviation of blocks bk; and bkj respectively, abki bkj is the covariance of blocks bk; and bkj. Fig.3 shows the original "bikes" (768 x 512) image (Fig.3 (al)) and its normalized entropy map E (Fig.3 (dl)), with a value of 1 indicating the region of highest interest represented as white, and a value of 0 indicating the region of lowest interest as black. Thus, levels of interest decrease from white to black which represent the visual regions of other levels.

In this step, GSSIM [4] is used. The edge information is obtained by the Sobel operators. We setXorg and Ydist represent the gradient map of the original image and the distorted one respectively, and let bk;' and bk/ be blocks (theirs size is llxll pixels) of Xorg and Ydist respectively. This measure is used in determination edge information where the contrast comparison c(bk;, bk) and structure comparison s(bk;, bk) in [3] are replaced with the gradient-based contrast comparison cibk/, bkj) and structure comparison sibk/, bkj) respectively.

Fig.3 shows the original "bikes" image ((Fig.3 (al ))) and its transform using Sobel operators (Fig.3 (bI)). In our proposed measure, we have used only structure comparison sibk/, bkj); this later represents the edge information where its formula is as follows:

(abk'bk' + c3) Sg( bk;,bk;) = ' J + c (11)

abk[abkj 3 where abki' and abk/ are the standard deviation of blocks

bk;' and bk/ respectively, abk;' bkj' is the covariance of blocks bk;' and bk/

Finally, the edge-region and distorted pixels with pixel Displaced structural similarity (ERDM) is described as follows

ERDM(bk;,bkj) = [1(bk;,bkj)t[c(bk;,bkj)([SER(bk;,bkj)Y (12)

+ w * pie(bk;, bkj) Where:pie(bk;,bk)=MB(Id;J,Dp;), SER(bk;,bk)=(sibk;',bkj)+ sr(bk;,bk))12, ro«1. We set a=fJ=y=1. The overall error measure of the original image and a distorted one are calculated as sum of ERDM (bk;,bkj) ; The normalized measure is defined as EO ERDM(Original, distorted) = 2 xl Eb L ERDM(bk;, bkj) (13)

i=l Where Eb=(MxN)111 x 11 is the total number of blocks. This measure is closer to "zero" when the image has the best quality and closer to the "one" in the other case. Furthermore, ERDM is intended for color image. This later is divided on three images: red, blue and green. For each one the measures defined before are calculated. The latest m�asure means ERDM, is divided on three. So,

79

ERDM for color image is equal ERDM for red image plus ERDM for blue image plus ERDM for green image divided by three.

4. RESULTS

The experiments are carried out on the Live Image Quality Assess Database Release2 [3], to test the proposed measure. We have also compared the performance of our algorithm against PSNR, VroiWQI and GSSIM. Table 1 summarizes the validation results. Fig.3 shows original images and its transformed images. In order to provide quantitative measures on the performance of our proposed ERDM quality assessment method, we follow the standard performance evaluation procedures employed in the video quality experts group (VQEG) FR-TV Phase II test [6], where mainly three evaluation metrics were used (Metric 1, Metric 2 and Metric 3). Non-linear mapping was carried out between the subjective and objective results. Thus, we use five-­parameters (el, e2, e3, e4 and es) to transform the Difference Mean Opinion Score (DMOS) to predicted DMOS (DMOSp). The equation 14 shows the mapping function.

DMOSp = 811ogistic(82, (VQR - (3)) + 84 + 85 (14) 1 1

logistic(x, VQR) = "2 - 1 + exp(VQR x) (15)

where VQR is the quality rating by the objective method (ERDM, VroiWQI, GSSIM or PSNR) and el, e2, e3, e4, es are chosen for best fit. The fitting (optimization of e parameters) was done with MATLAB's fminsearch function. After the non-linear mapping, the following metrics are using in calculation between the subjective and objective scores. Non-linear regression correlation coefficient (CC). Spearman rank-order correlation coefficient (ROCC). Outlier ratio (OR) of "outlier­points" to total points N. Root mean square error (RMS). Mean absolute error (MAE). The simulation results for all the distorted images are summarized in Table 1. The CC means the degree of correlation between each measure and DMOS, and the larger value of CC means better exactitude. The ROCC, it is considered as a measure of prediction monotonicity, higher ROCC means the better prediction monotonicity. The MAE, RMS and OR are measures of prediction consistency, smaller value means better performance. Fig.2 shows the scatter plots of DMOS versus ERDM, PSNR, VroiWQI and GSSIM. It is clear that the fit between the ERDM and the DMOS is linear, whereas, as expected, the fit between the PSNR, VroiWQI, GSSIM and the DMOS is nonlinear. From the Figure, we can see that the proposed ERDM based image quality emulates the DMOS better than the other conventional methods. This can also be deduced from the quantitative performance analysis. From the table, it can be inferred that MAE, RMS and OR errors of the proposed measure is less than the VroiWQI and the GSSIM models. Similar observations are made for the PSNR based image quality metric. Furthermore, the ROCC and CC statistical estimate provide clear picture on the effectiveness of the

proposed image quality method over the PSNR, VroiWQI and GSSIM. PSNR does not correlate well with subjective scores as seen in Table. From the table, the results clearly indicate that our ERDM measure performs quite well and is competitive with other IQA measures.

5. CONCLUSION

In this paper, we have addressed the problem of assessment of images quality, and we have developed a technique for evaluation of image quality. Our approach is based on distorted and displaced pixels. Simulation results with well known measures show that the performance of the proposed metric is found to be better than previously metrics which does not use the displaced pixels in calculation. Thus, the results show that this method can be useful in the assessment of image quality. In the conducted experiments, the proposed IQA method showed the ability to estimate the image quality according to what a human observer would do.

Tablet. Performance comparison of IQA methods (PSNR, GSSIM, VroiWQI, and the ERDM) for JPEG, JPEG2000, Gaussian blurs, white noise, and Fastfading distorted images.

Model CC ROCC MAE RMS OR

GSSIM 0,8821 0,7797 4,9120 5,8063 0,066

PSNR 0,8322 0,6628 6,7516 7,0766 0,079

VroiWQI 0,8733 0,8114 5,2010 5,7880 0,063

ERDM 0,9436 0,9246 4,6200 5,0580 0,053

6. REFERENCES [I] Z. Wang and A.c. Bovik, "Mean squared error: Love it or

leave it? ," A new look at signal fidelity measures, IEEE Signal Processing Magazine, vol 26, no. I, pp. 98-117, January 2009.

[2] www.cns.nyu. eduJ �zwanglfi1es/research/ quality_index.

[3] Z.Wang A.c. Bovik, H. R. Sheikh, and E .P. Simocelli, "Image quality assessment: From error measurement to structural similarity," IEEE Trans. Image Processing, vol. 13, no.4, pp.600- 612, Apr .2004.

[4] C. Guan-Hao, Y. Chun-Ling, X. Sheng-Li, "Gradient-based structural similarity for image quality assessment," in Proc. ICIP06, pp. 2929-2932.

[5] D. Venkata Rao, N. Sudhakar, I. Ramesh Babu, L. Pratap Reddy,"Image quality assessment complemented with visual regions of interest," in Proc. ICCTA 2007, pp. 681-687

[6] VQEG: Final Report from the video quality experts group on the validation of objective models of video quality assessment, FR-TV Phase II, August 2003 (http://www.vqeg.orgl)

[7] Z.A. Seghir, F. Hachouf. "Image quality assessment based on edge-region information and distorted pixel for JPEG and JPEG2000," in Proc. ACIVS 2009, LNCS 5807, pp. 156-166.

80

lOOr-----------�_= ••

4.

I.

o 0,1 0,2 0,3 0,4 O,S 0,6 0,1 0,8 0,9 1 ERDM

(a)

o �l U �) � u o� 0,7 U o� 1 GSSIM

(c)

o 0,1 0,1 0,1 0,4 D,S 0,6 0,7 0,8 0,9 1 PSNR

(b) 100 ,------------=.__----,

3D 20 10

o 0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 1 VroiWOI

(d)

Fig.2 Scatter plots of DMOS against measure prediction for JPEG, white noise, Gaussian blur, JPEG2000, Fastfading distorted images. (a)ERDM (b)PSNR (c)GSSIM (d) VroiWQI.

(c2) (d2) Fig.3 (a) Original image (b) Simulated image showing the edge information of original image (c) Simulated image showing the edge information of distorted image (d) and Simulated image showing the Visual regions of interest map of original image