IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed...

45
Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning Vladimir Kulyukin Sarat Kiran Andhavarapu Department of Computer Science Utah State University

Transcript of IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed...

Page 1: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

Image Blur Detection with 2D Haar Wavelet Transform and

Its Effect on Skewed Barcode Scanning

Vladimir Kulyukin Sarat Kiran Andhavarapu

Department of Computer ScienceUtah State University

Page 2: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

Outline

● Background● Image Blur Detection with 2D Haar Tile

Clustering● Evaluation of 2D Haar Tile Clustering● Effects of Image Blur Detection on Skewed

Barcode Scanning

Page 3: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

Background

Page 4: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

Theoretical Foundations

● Mallat & Hwang [1] argue that signals carry information via irregularites

● These researchers show that the local maxima of the wavelet transform detect locations of irregularites

● For example, 2D HWT maxima indicate possible locations of edges in images

Page 5: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

Indirect & Direct Blur Detection Methods

● Tong et al. [2] classify image blur detection methods into direct and indirect

● Indirect methods characterize image blur as a linear function IB = B *

IO + N, where I

O is the original image, B is an unknown blur function,

N is a noise function, and is the result image blur and noise are introduced in the image

● Direct methods are based on detection of distinct features directly computed in the images, e.g., corners, edges, color histograms, etc

Page 6: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

Edge Classification

Tong, H., Li, M., Zhang, H., and Zhang, C. "Blur detection for digital images using wavelet transform," In Proceedings of the IEEE International Conference on Multimedia and Expo, vol.1, pp. 27-30, June 2004.doi: 10.1109/ICME.2004.1394114.

Page 7: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

Edge-Based Blur Detection

● Tong et al. [2] propose a direct method based on 2D Haar Wavelet Transform

● Main assumption of their research is that introduction of blur has different effects on the four main types of edges

● In blurred images, Dirac and A-Step edges are absent whereas G-Step and Roof edges lose their sharpness

● Images are classified as blurred on the basis of presence/absence of Dirac & A-Step edges

Page 8: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

Image Blur Detectionwith

2D Haar Tile Clustering

Page 9: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

Theory

● Our method (2D Haar Tile Clustering) is based on the hypothesis that it may not be necessary to detect any explicit features such as corners or edges

● Rather, it may be possible to detect regions with pronounced changes without explicitly computing the causes of those changes

● After those regions are detected, they can be combined into larger segments

● Those larger segments can be used to classify images as blurred

Page 10: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

Theory

● Sharp images have many easily distinguishable features, e.g., edges, corners, textures

● Blurred images have fewer easily distinguishable features because distinctions among image regions are less pronounced

● Easily distinguishable features consists of regions with pronounced changes

Page 11: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

Finding Regions with Pronounced Changes

Image is split into 64 x 64 blocks (aka tiles)

Page 12: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

Tile Processing

● Each tile is processed by four iterations of the 2D HWT● The number of iterations is a parameter, and can be

increased/decreased● Each tile is represented by three 2D Haar wavelets:

horizontal change (HC), vertical change (VC), and diagonal change (DC)

● These values are thresholded to retain the tiles with only large changes

Page 13: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

Applying 2D HWT to Image

● Image on the right shows two iterations of ordered 2D HWT

● The whiteness of each pixel is proportional to the magnitude of the wavelet in the corresponding array cell

Page 14: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

Representation of 64 x 64 Blocks with 2D Wavelets

Each region is represented in terms of 3 wavelets: vertical (first double); horizontal (second double), diagonal (third double); Ignore the numbers after “|”: they are used for debugging

Page 15: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

Tile Cluster Buildup with DFS

● After all tiles with pronounced areas are found, as shown in the image on the right

● Now we can run DFS to find all tile clusters

Page 16: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

Tile Cluster Buildup with DFS

Page 17: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

Tile Cluster Buildup with DFS

Page 18: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

Tile Cluster Buildup with DFS

Page 19: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

Tile Cluster Buildup with DFS

Page 20: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

Tile Cluster Buildup with DFS

Page 21: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

Tile Cluster Buildup with DFS

Page 22: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

Tile Cluster Buildup with DFS

Page 23: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

Tile Cluster Buildup with DFS

Page 24: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

Tile Cluster Buildup with DFS

Page 25: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

Tile Cluster Buildup with DFS

Page 26: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

Tile Cluster Buildup with DFS

Page 27: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

Tile Cluster Buildup with DFS

Page 28: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

Tile Cluster Buildup with DFS

Page 29: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

Tile Cluster Buildup with DFS

Page 30: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

Tile Cluster Buildup with DFS

Page 31: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

Tile Cluster Buildup with DFS

Page 32: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

Tile Cluster Buildup with DFS

Page 33: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

Tile Cluster Buildup with DFS

Page 34: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

Tile Clustering in Sharp Images

Eventually we run out of unmarked tiles with sufficiently high pronounced changes, as shown in the image on the right

Page 35: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

Tile Clustering in Blurred Images

Title clusters shown in the right image are found by DFS run on tiles with pronounced changes found in the left image

Page 36: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

Tile Cluster Filtering● After the tile clusters are found, two cluster-related rules are used to

classify a whole image as sharp or blurred

● The 1st rule uses the percentage of the total area of the image covered by the found tile clusters

● The 2nd rule uses the number of the tiles in each cluster to discard small clusters

● The 1st rule captures the intuition that sharper images have many tiles with pronounced changes

● The 2nd rule captures the intuition that small clusters should be discarded as irrelevant

Page 37: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

2D Haar Tile Clustering: Algorithmic ChainTake Image Find Tiles Find Tile

ClustersFilter Tile

Clusters

BLURRED / SHARP

Page 38: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

Evaluation of 2D Haar Tile Clustering&

Effects of Image Blur on Skewed Barcode Scanning

Page 39: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

Image Sample

● 500 random RGB images* were selected from a set of 506 smartphone video recordings of common grocery products

● Three human volunteers were recruited to classify each image as blurred or sharp

● An image was classified as sharp if at least two of the three volunteers classified it as sharp – this is the ground truth

*Images are available at https://app.box.com/s/n4s2ve0dajz5gkzqx9vpm1f6fzhw5upz

Page 40: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

Three Evaluated Algorithms

● Algorithm 1: Kulyukin, V. & Andhavarapu. S. “Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning.” To appear in Proceedings of the 19th International Conference on Image Processing, Computer Vision, & Pattern Recognition (IPCV 2015). Las Vegas, NV, USA

● Algorithm 2: Tong, H., Li, M., Zhang, H., and Zhang, C. "Blur detection for digital images using wavelet transform," In Proceedings of the IEEE International Conference on Multimedia and Expo, vol.1, pp. 27-30, June 2004.doi: 10.1109/ICME.2004.1394114

● Algorithm 3: Blur effect: perception and estimation with a new no-reference perceptual blur metric.” In Proceedings of SPIE 6492, Human Vision and Electronic Imaging XII, 64920I, San Jose, CA, USA, January 28, 2007. doi:10.1117/12.702790

Page 41: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

True & False Positives on Blurred and Sharp Images

Algorithm True Positives on Blurred Images

False Positives on Blurred Images

True Positives on Sharp Images

False Positives on Sharp Images

Algorithm 1 163 4 254 79

Algorithm 2 167 0 183 150

Algorithm 3 81 86 268 65

Ground Truth 167 0 333 0

Page 42: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

Relative Difference Table

Algorithm Relative Difference on Blurred Images

Relative Difference on Sharp Images

Algorithm 1 2.39 23.72

Algorithm 2 0.00 45.05

Algorithm 3 51.50 19.52

images arpblurred/shon truth ground theis and algorithmby found

images arpblurred/sh ofnumber theis where,100,max

,fferenceRelativeDi

GA

AGA

GAGA

Page 43: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

Observations

● On blurred images, Algorithm 1 & Algorithm 2 do not deviate from ground truth; Algorithm 3 shows a significant deviation (52%)

● On sharp images, Algorithm 1 & Algorithm 3 deviate from ground truth by 20% whereas Algorithm 2 deviates by 45%

Page 44: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

Effects of Blur Detection on Barcode Scanning

Sample Sharp Blurred Barcodes in Sharp

Barcodes in Blurred

1 15 15 12 1

2 13 17 11 0

3 16 14 12 0

images. allin barcodes recognized of ratio theand images sharpin only

barcodes recognized of ratio theb/w difference theisGain ed.investigat wasdetection

blur accurate ofeffect theand algorithm scanning barcode skewedour into integrated

wasdetection blur The images. blurred 15 and sharp 15 had sampleeach :evalutorshuman

3by classified images 500 from selected each were images 30 of samples random Three

Sample Blurred Sharp Total Gain

1 1/15 12/15 13/30 0.37

2 0/17 11/13 11/30 0.50

3 0/16 12/14 12/30 0.46

Page 45: IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning

Paper & Code References

[1] Mallat, S. and Hwang, W. L. “Singularity detection and processing with wavelets.” IEEE Transactions on Information Theory, vol. 38, no. 2, March 1992, pp. 617-643.

[2] Tong, H., Li, M., Zhang, H., and Zhang, C. "Blur detection for digital images using wavelet transform," In Proceedings of the IEEE International Conference on Multimedia and Expo, vol.1, pp. 27-30, June 2004.doi: 10.1109/ICME.2004.1394114.

[3] Cretea,F., Dolmierea, T., Ladreta, P., Nicolas, M. “The Blur effect: perception and estimation with a new no-reference perceptual blur metric.” In Proceedings of SPIE 6492, Human Vision and Electronic Imaging XII, 64920I, San Jose, CA, USA, January 28, 2007. doi:10.1117/12.702790.

[4] Python implementation of the blur detection algorithm proposed in reference [2] is available at https://github.com/VKEDCO/PYPL/blob/master/haar_blur

[5] MATLAB implementation of blur detection algorithm proposed in reference [3] is

available at http://www.mathworks.com/matlabcentral/fileexchange/24676-image-blur-metric