Journal of Information & Computational Science 10:11 (2013) 3497–3502 July 20, 2013Available at http://www.joics.com

Image Restoration Through Dictionary Learning and

Sparse Representation ⋆

Xiaoyu Wang a,∗, Qi Ran a, Deyun Chen a, Feng Jiang b

aSchool of Computer Science and Technology, Harbin University of Science and TechnologyHarbin 150080, China

bSchool of Computer Science and Technology, Harbin Institute of Technology, Harbin 150001, China

Abstract

Based on the content dual-dictionary learning and sparse representation, we put forward a novel methodof image restoration. This method can improve the adaptive ability of the image. To restore the image,the dual-dictionary is trained with sparse representation. Comparing with the traditional dictionarylearning algorithm, the method in this paper can capture more high-frequency information and enhancethe image quality further in image reconstruction. The experimental results show that the proposedmethod is useful for image restoration and much better than other methods.

Keywords: Image Restoration; Dictionary Learning; Sparse Representation

1 Introduction

Nowadays, image has become one of the most important ways human getting information. Highquality images have important value in our life. The image, however, can be infected by noise.So image restoration has become one of the important measure in image processing.

Super-resolution image reconstruction is one of the most important methods in restoration ofhigh-resolution image. The super-resolution restoration method can be divided into two kinds.They are based on reconstruction algorithm and learning-based algorithm.

At present, a lot of researchers consider the learning-based algorithm as a very promisingapproach and think it is a hot spot in the field of super-resolution [1]. The algorithm of learning-based in super-resolution is through learning to get the relation between high-resolution and low-resolution to guide the high-resolution image reconstruction [2]-[5]. The learning-based algorithmmakes full use of the prior knowledge of image itself and is applicable to the face and text imagerestoration [6, 7].

⋆Project supported by the National Nature Science Foundation of China (No. 61100096, No. 60672090)and Science and Technology Research Projects in Office of Education of Heilongjiang province (No. 11531049;No. 12511098).

∗Corresponding author.Email address: cfwang@hit.edu.cn (Xiaoyu Wang).

1548–7741 / Copyright © 2013 Binary Information PressDOI: 10.12733/jics20101953

3498 X. Wang et al. / Journal of Information & Computational Science 10:11 (2013) 3497–3502

The traditional learning-based super-resolution reconstruction algorithm, however, does notconsider differences among the image blocks and just trains a dictionary without distinction. Itcan’t timely according to the nature of the image to choose the suitable algorithm which probablecauses deviation of the restoration image. Although based on dictionary learning combined withsparse representation can reconstruct the high-quality high-resolution image, it still can’t avoidthe lost part of high-frequency detail information that will influence the recovery image qualitygreatly.

In order to solve the above problems, in this paper, a kind of dual-dictionary learning-based re-construction algorithm has been proposed. Firstly, the images have been divided into several sortsby the clustering algorithm. The features in clustering has been selected to construct the mostaccurate coding book. Then, the most matching classification is selected, so the low-resolutionimage has been reconstructed with the most suitable dictionary. Secondly, the high-frequencyinformation is divided into the Main High-frequency information (MHF) and the Residual High-frequency information (RHF) to construct the dual-dictionary, the main dictionary learning andthe residual dictionary learning, corresponding to restoring MHF and RHF. Compared with thetraditional method, the dual-levels restore algorithm can get better effect in reconstruction.

2 Based on the Dual-dictionary Algorithm

2.1 Classifying Images Based on Content

In this stage, we input some high-resolution images as training images, which are processed byfuzzy filter and down-sampling to get the low-frequency in low-resolution LLF . Then we geta low-frequency in high-resolution HLF by bicubic-interpolation and gain the high-frequency inhigh-resolution HHF through the original image HORG subtracting the low-frequency in high-resolution HLF . Therefore, we divide the original training image HORG into HHF and HLF .

We can make this two parts of image as a sparse coding book [8, 9]. We use the histogramto express the each of image sparse radix in training set. So all of the training image can beexpressed clearly. In different training image, because its content is distinct, the histogram alsohas a big difference. Therefore it can be differentiated by the histogram. The training imagehistogram can be clustered and divided into K class by the K-means algorithm. We can obtainK clustering center (m1,m2,m3 · · ·mk) that each kind of class can express one image content.The training images can be classified according to the different image content.

Then, we input a low-resolution image, and obtain the histogram in the same method withthe sparse coding book. Using the histogram to compare with the classified histogram set of thetraining images, we find the classification with most similar content. We utilize this classifiedtraining dictionary to reconstruct the low-resolution image. As shown in Fig. 1.

2.2 Dual-dictionary Learning and Sparse Representation

In this stage, the High-frequency (HF) to be estimated is divided into a combination of twocomponents, Main High-frequency (MHF) and Residual High-frequency (RHF). We design a dual-dictionary, the main dictionary learning and the residual dictionary learning, to restore MHF andRHF, respectively. In the process of the MHF restoration, the training images which be trained

X. Wang et al. / Journal of Information & Computational Science 10:11 (2013) 3497–3502 3499

Original training

images

Sparse

representation Sparse

coding bookHistogram set

K-means K clustering

center

Best match

HistogramThe low-resolution

image

Reconstruct the

high-resolution image

The most appropriate

classification dictionary

Fig. 1: Dictionary classification based on the image content

into a sparse coding book can be expressed by the histogram and divided into K class throughthe K-means algorithm to train for K-pairs dictionary respectively. According to the imagecontent, every kind of image can be expressed by one content, that is {(HFD1STi, LFD1STi), i =1, 2, 3 · · · k}, and is corresponding to the training image of class i. Using the method described inSection 2.1, we can get the main high-resolution reconstruction image, utilizing the low-resolutionimage histogram to match up the training images and choosing the most suitable classificationfor image reconstruct operation.

The training image is divided into K class by K-means algorithm to obtain K clustering center(m1,m2,m3 · · ·mk) simultaneously. We can find the most appropriate classification through thelow-resolution histogram. Then the K classified training image is trained by the K-pair dictionaryrespectively use the classical K-SVD algorithm, that is {(HFD1STi, LFD1STi), i = 1, 2, 3 · · · k}:

LFD1STi, {a∗} = argminLFD1STi,{a∗}∑K

∥y − LFD1STi, a∗∥2s.t.∥a∗∥0 ≤ L∀k

That a∗ means the sparse representation coefficient, y means a image block in low-frequency inLFD1STi and L means the control parameter.

We can obtain LFD1STi from the different classified dictionary and the sparse representationvector a∗ of a image block in low-frequency image in the LFD1STi.

The HHF and HLF of the image which are corresponding to the high- and low-frequency imageblocks have a same property which is the sparse representation vectors are uniform in differentdictionary. So that the sparse representation vector of HFD1STi is also a∗:

HFD1STi = argminHFD1STi

∑K

∥y −HFD1STi, a∗∥2

Thus we can obtain the HFD1STi and LFD1STi. To every image block y in a random HLF ,according to sparse representation vector a∗ and HFD1STi, the image block involving the MHFcan be achieved by product. The HMHF is reconstructed by MHF image blocks, and the firsthigh-resolution reconstruction image (H1ST ) can be obtained by the HMHF with the addition ofHLF .

3500 X. Wang et al. / Journal of Information & Computational Science 10:11 (2013) 3497–3502

Then, HRHF will be built by the high-frequency of original high-resolution (HHF ) subtractingthe HMHF . The second dictionary training process includes HRHF and H1ST . According to thetraditional algorithm based on dictionary learning can obtain the second dictionary that involvesSecond High Frequency Dictionary (HFD2ND) and Second Low Frequency Dictionary (LFD2ND).As described in Fig. 2.

The i kind of

training images

HLF

HMHFResidual

dictionary

Main

dictionary

H1ST

HRHF

−

HHF

Fig. 2: The training process of dual-dictionary

3 Image Synthesis

In this stage, the low-resolution image can be obtained by down-sampling and smooth operationof the original image. We use the traditional bicubic interpolation algorithm to amplify in orderto obtain HLF , which is divided into overlapping image block through block operation. Thatis because the overlapping image block can avoid the blocking artifacts. For each block y weadopt the OMP [8] algorithm, a greedy algorithm that is via gradual approximation to achievethe sparse representation to solve sparse representation vector in classified LFD1STi.

min∥a∗∥1s.t.y = LFD1STi · a∗

We can gain sparse representation coefficient a∗ by sparse decomposition algorithm. Accord-ing to the relation between the high-resolution and low-resolution dictionary, that is one-to-onecorrespondence, the high-resolution image can be obtained by the following formula that meansreconstruct HMHF by the sparse representation vector and HFD1STi.

x = HFD1STi · a∗

where x is high-resolution image block. All blocks x overlap together synthetize HMHF . TheH1ST is reconstructed by HLF with the addition of HLF . The H1ST can be further processed tooverlapping image x1st. Then we use OMP algorithm to solve the sparse representation parameterin LFD2nd:

min∥β∗∥1s.t.x1st = LFD2nd · β∗

Then, the residual high-resolution section can be reconstructed by HFD2nd and sparse repre-sentation vector:

z = HFD2nd · β∗

All the blocks z are overlapped to synthesis the residual high-resolution(HRHF ). Finally, thesecond high-resolution image H2nd can be built by H1ST and HRHF .

X. Wang et al. / Journal of Information & Computational Science 10:11 (2013) 3497–3502 3501

4 Experimental Results

We use the improved algorithm to test the image, and compare the test results with other algo-rithm such as, bicubic interpolation, sparse representation.

In this paper, the size of main high-resolution dictionary and residual dictionary are bothset to 500, equalling to the sum of the size of main- and residual high-resolution dictionary, inproposed scheme. The number of the base of the reconstructed image blocks is set to 3, the sizeof the image block is set to 9 × 9 and there is a pixel overlap to avoid block effect. We adoptFoliage, Monarch, Sailboat and Lena image to test the algorithm in this paper (see Fig. 3 andFig. 4).

Table 1 lists the PSNR value of the different algorithms. We can easily judge the PSNR valueof the improved method is the highest in this paper.

Fig. 3: The performance test images

(a) Original image (b) Bicubic interpolation (c) Sparse representation (d) The proposed algorithm

(e) Original image (f) Bicubic interpolation (g) Sparse representation (h) The proposed algorithm

(a) Original image (b) Bicubic interpolation (c) Sparse representation (d) The proposed algorithm

Fig. 4: Visual comparison by different algorithms

3502 X. Wang et al. / Journal of Information & Computational Science 10:11 (2013) 3497–3502

Table 1: PSNR comparisons with different algorithms (dB)

Images Bicubic Sparse representation Proposed

Leaf 31.65 (3.95) 34.73 (0.87) 35.60

Lena 32.19 (2.80) 34.66 (0.33) 34.99

Sailing 30.56 (2.36) 32.36 (0.56) 32.92

Monarch 22.78 (3.24) 30.34 (0.68) 31.02

5 Conclusion

In this paper, the image super-resolution restoration algorithm of based on the dual-dictionarylearning and sparse representation is presented. In order to strengthen the dictionary adaptiveability and improve the sparse representation description ability, we choose the classificationwhich is similar to the low-resolution image to restore. For loss of high-frequency in the processof reconstruction we use double level recovery. The high-frequency information is divided intothe Main High-frequency (MHF) and the Residual High-frequency (RHF). Then we constructthe dual-dictionary. Finally, the algorithm is verified by the experiments. Compared with thetraditional dictionary learning, the test results show that the algorithm in this paper can capturemore image details and get better reconstruction effect.

References

[1] W. T. Freeman, T. R. Jones, E. C. Pasztor, Example-based super-resolution, IEEE ComputerGraphics and Applications, 22(2), 2002, 56-65

[2] Z. Xiong, X. Sun, F. Wu, Image hallucination with feature enhancement, IEEE Conference onComputer Vision and Pattern Classification, 1 (2009), 2074-2081

[3] H. Chang, D. Y. Yeung, Y. Xiong, Super-resolution through neighbor embedding, IEEE Conferenceon Computer Vesion and Pattern Classification, 1 (2004), 275-282

[4] J. Sun, N. N. Zheng, H. Tao, H. Shum, Image hallucination with primal sketch priors, IEEEConference on Computer Vision and Pattern Recofnition, 2 (2003), 729-736

[5] L. Ma, F. Wu, D. Zhao, W. Gao, S. Ma, Learning-based image restoration for compressed imagethrough neighboring embedding, Berlin: Springer-Herdelberg, 2008, 297-286

[6] Wei Zhang, Wai-Kuen Cham, Hallucinating face in the DCT domain, IEEE Transactions on ImageProcessing, 10(20), 2011, 2769-2779

[7] X. Wang, X. Tang, Hallucination face by eigentransformation, IEEE Transactions on System,Man, and Cybernetics-Part C: Application and Reviews, 35(3), 2005, 425-434

[8] M. Aharon, M. Elad, A. Bruckstein, K-SVD: An algorithm for designing overcomplete dictionariesfor sparse representation, IEEE Trans. on Signal Processing, 54(11), 2000, 4311-4322

[9] H. Lee, A. Battle, R. Raina, A. Y. Ng, Efficient sparse coding algonthrns, in Advances in NeuralInformation Processing Systems (NIPS), 2007