IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed...
-
Upload
vladimir-kulyukin -
Category
Science
-
view
201 -
download
10
Transcript of IPCV 2015 Presentation: Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed...
Image Blur Detection with 2D Haar Wavelet Transform and
Its Effect on Skewed Barcode Scanning
Vladimir Kulyukin Sarat Kiran Andhavarapu
Department of Computer ScienceUtah State University
Outline
● Background● Image Blur Detection with 2D Haar Tile
Clustering● Evaluation of 2D Haar Tile Clustering● Effects of Image Blur Detection on Skewed
Barcode Scanning
Background
Theoretical Foundations
● Mallat & Hwang [1] argue that signals carry information via irregularites
● These researchers show that the local maxima of the wavelet transform detect locations of irregularites
● For example, 2D HWT maxima indicate possible locations of edges in images
Indirect & Direct Blur Detection Methods
● Tong et al. [2] classify image blur detection methods into direct and indirect
● Indirect methods characterize image blur as a linear function IB = B *
IO + N, where I
O is the original image, B is an unknown blur function,
N is a noise function, and is the result image blur and noise are introduced in the image
● Direct methods are based on detection of distinct features directly computed in the images, e.g., corners, edges, color histograms, etc
Edge Classification
Tong, H., Li, M., Zhang, H., and Zhang, C. "Blur detection for digital images using wavelet transform," In Proceedings of the IEEE International Conference on Multimedia and Expo, vol.1, pp. 27-30, June 2004.doi: 10.1109/ICME.2004.1394114.
Edge-Based Blur Detection
● Tong et al. [2] propose a direct method based on 2D Haar Wavelet Transform
● Main assumption of their research is that introduction of blur has different effects on the four main types of edges
● In blurred images, Dirac and A-Step edges are absent whereas G-Step and Roof edges lose their sharpness
● Images are classified as blurred on the basis of presence/absence of Dirac & A-Step edges
Image Blur Detectionwith
2D Haar Tile Clustering
Theory
● Our method (2D Haar Tile Clustering) is based on the hypothesis that it may not be necessary to detect any explicit features such as corners or edges
● Rather, it may be possible to detect regions with pronounced changes without explicitly computing the causes of those changes
● After those regions are detected, they can be combined into larger segments
● Those larger segments can be used to classify images as blurred
Theory
● Sharp images have many easily distinguishable features, e.g., edges, corners, textures
● Blurred images have fewer easily distinguishable features because distinctions among image regions are less pronounced
● Easily distinguishable features consists of regions with pronounced changes
Finding Regions with Pronounced Changes
Image is split into 64 x 64 blocks (aka tiles)
Tile Processing
● Each tile is processed by four iterations of the 2D HWT● The number of iterations is a parameter, and can be
increased/decreased● Each tile is represented by three 2D Haar wavelets:
horizontal change (HC), vertical change (VC), and diagonal change (DC)
● These values are thresholded to retain the tiles with only large changes
Applying 2D HWT to Image
● Image on the right shows two iterations of ordered 2D HWT
● The whiteness of each pixel is proportional to the magnitude of the wavelet in the corresponding array cell
Representation of 64 x 64 Blocks with 2D Wavelets
Each region is represented in terms of 3 wavelets: vertical (first double); horizontal (second double), diagonal (third double); Ignore the numbers after “|”: they are used for debugging
Tile Cluster Buildup with DFS
● After all tiles with pronounced areas are found, as shown in the image on the right
● Now we can run DFS to find all tile clusters
Tile Cluster Buildup with DFS
Tile Cluster Buildup with DFS
Tile Cluster Buildup with DFS
Tile Cluster Buildup with DFS
Tile Cluster Buildup with DFS
Tile Cluster Buildup with DFS
Tile Cluster Buildup with DFS
Tile Cluster Buildup with DFS
Tile Cluster Buildup with DFS
Tile Cluster Buildup with DFS
Tile Cluster Buildup with DFS
Tile Cluster Buildup with DFS
Tile Cluster Buildup with DFS
Tile Cluster Buildup with DFS
Tile Cluster Buildup with DFS
Tile Cluster Buildup with DFS
Tile Cluster Buildup with DFS
Tile Cluster Buildup with DFS
Tile Clustering in Sharp Images
Eventually we run out of unmarked tiles with sufficiently high pronounced changes, as shown in the image on the right
Tile Clustering in Blurred Images
Title clusters shown in the right image are found by DFS run on tiles with pronounced changes found in the left image
Tile Cluster Filtering● After the tile clusters are found, two cluster-related rules are used to
classify a whole image as sharp or blurred
● The 1st rule uses the percentage of the total area of the image covered by the found tile clusters
● The 2nd rule uses the number of the tiles in each cluster to discard small clusters
● The 1st rule captures the intuition that sharper images have many tiles with pronounced changes
● The 2nd rule captures the intuition that small clusters should be discarded as irrelevant
2D Haar Tile Clustering: Algorithmic ChainTake Image Find Tiles Find Tile
ClustersFilter Tile
Clusters
BLURRED / SHARP
Evaluation of 2D Haar Tile Clustering&
Effects of Image Blur on Skewed Barcode Scanning
Image Sample
● 500 random RGB images* were selected from a set of 506 smartphone video recordings of common grocery products
● Three human volunteers were recruited to classify each image as blurred or sharp
● An image was classified as sharp if at least two of the three volunteers classified it as sharp – this is the ground truth
*Images are available at https://app.box.com/s/n4s2ve0dajz5gkzqx9vpm1f6fzhw5upz
Three Evaluated Algorithms
● Algorithm 1: Kulyukin, V. & Andhavarapu. S. “Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning.” To appear in Proceedings of the 19th International Conference on Image Processing, Computer Vision, & Pattern Recognition (IPCV 2015). Las Vegas, NV, USA
● Algorithm 2: Tong, H., Li, M., Zhang, H., and Zhang, C. "Blur detection for digital images using wavelet transform," In Proceedings of the IEEE International Conference on Multimedia and Expo, vol.1, pp. 27-30, June 2004.doi: 10.1109/ICME.2004.1394114
● Algorithm 3: Blur effect: perception and estimation with a new no-reference perceptual blur metric.” In Proceedings of SPIE 6492, Human Vision and Electronic Imaging XII, 64920I, San Jose, CA, USA, January 28, 2007. doi:10.1117/12.702790
True & False Positives on Blurred and Sharp Images
Algorithm True Positives on Blurred Images
False Positives on Blurred Images
True Positives on Sharp Images
False Positives on Sharp Images
Algorithm 1 163 4 254 79
Algorithm 2 167 0 183 150
Algorithm 3 81 86 268 65
Ground Truth 167 0 333 0
Relative Difference Table
Algorithm Relative Difference on Blurred Images
Relative Difference on Sharp Images
Algorithm 1 2.39 23.72
Algorithm 2 0.00 45.05
Algorithm 3 51.50 19.52
images arpblurred/shon truth ground theis and algorithmby found
images arpblurred/sh ofnumber theis where,100,max
,fferenceRelativeDi
GA
AGA
GAGA
Observations
● On blurred images, Algorithm 1 & Algorithm 2 do not deviate from ground truth; Algorithm 3 shows a significant deviation (52%)
● On sharp images, Algorithm 1 & Algorithm 3 deviate from ground truth by 20% whereas Algorithm 2 deviates by 45%
Effects of Blur Detection on Barcode Scanning
Sample Sharp Blurred Barcodes in Sharp
Barcodes in Blurred
1 15 15 12 1
2 13 17 11 0
3 16 14 12 0
images. allin barcodes recognized of ratio theand images sharpin only
barcodes recognized of ratio theb/w difference theisGain ed.investigat wasdetection
blur accurate ofeffect theand algorithm scanning barcode skewedour into integrated
wasdetection blur The images. blurred 15 and sharp 15 had sampleeach :evalutorshuman
3by classified images 500 from selected each were images 30 of samples random Three
Sample Blurred Sharp Total Gain
1 1/15 12/15 13/30 0.37
2 0/17 11/13 11/30 0.50
3 0/16 12/14 12/30 0.46
Paper & Code References
[1] Mallat, S. and Hwang, W. L. “Singularity detection and processing with wavelets.” IEEE Transactions on Information Theory, vol. 38, no. 2, March 1992, pp. 617-643.
[2] Tong, H., Li, M., Zhang, H., and Zhang, C. "Blur detection for digital images using wavelet transform," In Proceedings of the IEEE International Conference on Multimedia and Expo, vol.1, pp. 27-30, June 2004.doi: 10.1109/ICME.2004.1394114.
[3] Cretea,F., Dolmierea, T., Ladreta, P., Nicolas, M. “The Blur effect: perception and estimation with a new no-reference perceptual blur metric.” In Proceedings of SPIE 6492, Human Vision and Electronic Imaging XII, 64920I, San Jose, CA, USA, January 28, 2007. doi:10.1117/12.702790.
[4] Python implementation of the blur detection algorithm proposed in reference [2] is available at https://github.com/VKEDCO/PYPL/blob/master/haar_blur
[5] MATLAB implementation of blur detection algorithm proposed in reference [3] is
available at http://www.mathworks.com/matlabcentral/fileexchange/24676-image-blur-metric