Synthesising and reducing film grain

www.elsevier.com/locate/jvci

J. Vis. Commun. Image R. 17 (2006) 163–182

Synthesising and reducing film grain

Antonio De Stefano a,*, Bill Collis b, Paul White a

a Institute of Sound and Vibration Research (ISVR), University of Southampton, Highfield,

Hants S017 1BJ, UKb The Foundry, 35-36 Great Marlborough Street, London W1F 7JE, UK

Received 1 June 2004; accepted 8 June 2005Available online 24 August 2005

Abstract

This paper describes some tools for adding and removing film grain. The film grain is rep-resented using an additive signal-dependent model. The approach adopted for artificial grainsynthesis avoids subjectivity and an assumption of Gaussianity. The grain within a user-de-fined plain area is analysed and the synthesis routine generates grain with matching spatialstructure having the same probability distribution function as the original. The grain reductionmethod is based on manipulation of the coefficients achieved using a bi-orthogonal undecimat-ed wavelet decomposition and is extremely advantageous for real-time implementation. Thescheme for modifying the coefficient is derived from Bayesian estimation and approximatesa range of optimal non-linear functions. Training to deduce parameter values is conductedby contaminating several nominally noise-free images with various realisations of grain noise.Using real and synthetically generated grain noise demonstrated an improvement of objectiveand visual qualities of the image. The ability of the technique to adapt with respect to imageand noise characteristics is also clear.� 2005 Elsevier Inc. All rights reserved.

Keywords: Film grain; Synthesis; Reduction; Wavelet transform; Non-linear filtering

1047-3203/$ - see front matter � 2005 Elsevier Inc. All rights reserved.

doi:10.1016/j.jvcir.2005.06.002

* Corresponding author. Fax: +44 0 2380593190.E-mail address: [email protected] (A. De Stefano).

mailto:[email protected]

164 A. De Stefano et al. / J. Vis. Commun. Image R. 17 (2006) 163–182

1. Introduction

When manipulating film images, as in the post-production industry, one needs totake considerable care to ensure the integrity of the grain structure on the final im-age. Potential problems arise when one seeks to seamlessly mix computer graphics(CG) elements and film, as required in many modern cinematic effects. To success-fully install CG elements in to film, grain needs to be added to the CG element;and similarly when extracting film elements for manipulation using CG tools oneneeds to first remove the film grain. This paper presents two tools for adding andremoving film grain in an efficient manner and with a minimum of user intervention.

Film grain arises because of the silver halide crystals that clump together duringthe development in the processed negative. In this paper, we address the problem ofthe construction of artificial film grain and propose a general technique to reducefilm grain.

There are numerous occasions when film grain needs to be applied to a digital se-quence, for example, artificial grain must be added to a CG element in order for it toblend in with existing film footage. Film grain is most often modelled as an additivenoise process with a variance that depends on the image intensity [1]. It is necessaryto model the grain separately in the red, green, and blue colour channels, and applydifferent levels of film grain to each channel. Due to the chemical processes involved,the blue channel is often found to contain the highest levels of grain. This paper isbased on such an additive, signal-dependent model of film grain.

Various methods for artificial grain synthesis have been proposed. As the struc-ture of film grain varies considerably with film stock, the user needs to specify thegrain to be replicated. This is most easily achieved by manually selecting a plain re-gion of the source image. One basic approach is to simply take the grain in this patchand manufacture a synthetic grain image by tiling this sample. This is rarely used be-cause the resulting image has an obvious repetitive structure. An alternative ap-proach [2] is to isolate the grain signal from a frame of film. This grain image canthen be superimposed onto a clean image. The technique requires the processingof an image the same size as, or bigger than, the clean image and for sequencesone needs to process the whole grainy sequence to avoid producing stable, unnaturallooking grain. Imperfections at the separation phase may also lead to cross-contam-ination between the two original frames. Many successful algorithms for texture rep-lication have been recently published [3,4]. These can also be used for synthesisinggrain; however, they all rely on duplicating regions of the source grain in some formand so can suffer from visible repetitions especially if the source region is small. Themost commonly used approach, in practice, is to filter a white Gaussian randomfield. The characteristics of the filter are selected by a compositor who uses theirjudgement to determine when the resulting output is acceptable. This operation isboth time consuming and highly subjective. Further, this method assumes that thegrain noise has a Gaussian distribution, an assumption that is difficult to test giventhe signal-dependent character of the problem.

The approach adopted in this research provides a method that avoids the needfor subjectivity and an assumption of Gaussianity. Our method again exploits a

A. De Stefano et al. / J. Vis. Commun. Image R. 17 (2006) 163–182 165

user-defined plain area to specify the nature of the film grain to be replicated. Thegrain within this patch is analysed and the synthesis routine generates grain withmatching spatial structure. Importantly, the approach ensures that the resultant syn-thetic grain has an identical probability distribution function (pdf) to the original,without making any assumptions regarding the original pdf.

This technique involves determining two filters: one that whitens the grain sampleand a second that reconstructs the grain sample from the whitened image. The designof these filters is achieved using standard methods. The crux of the technique is torecognise that the whitened image can be readily expanded to an arbitrary size byrandomly re-sampling pixels from the original whitened patch. The random natureof the sampling avoids repetitive structures whilst guaranteeing an identical pdf.The expanded grain image is constructed by application of the second filter to there-sampled image. The amplitude of the grain image is adjusted to compensate forchanges in image intensity before the final grain field is added to the clean sequence.The modulation used to adjust the grain amplitude is derived from the filmic re-sponse curves given in [1].

A solution to reduce the grain size is to use emulsions having very small fine grain;the major disadvantage of this solution is that small fine grain do not have muchlight capturing power and therefore in films the speed must be limited [5]. A compro-mise between grain size and film size is then in general required.

An alternative approach to reducing film noise is to process the image after it hasbeen captured. Several different film grain noise reduction techniques have been dis-cussed previously. Some authors [6–8] have employed the Maximum a Posteriori(MAP) estimation assuming image and noise having Gaussian distributions and esti-mating the general or local statistics of the image. The drawbacks of this method interms of real-time implementation are the assumption of correct estimation of the im-age statistics and the heavy computational procedure of MAP evaluation. Anothergroup of authors applied Wiener [9] and MMSE [10] filters referring, respectively, toimages gathering a large amount of light from the scene and to a specific and knownKodak film. A more sophisticated technique [11] was implemented to reduce grainnoise using spatial andmotion compensated automatic algorithms in which non-linearspatial filters and temporal spectral analysis are used to remove automatically grainnoise whilst preserving details. Sharpening weighted median algorithms [12,13] isanother solution recently tried. Signal-dependent weights are used to tune the sharpen-ing level insensible to compression and background noise. The technique has been ap-plied and validated using images contaminated by background grain noise and JPEGcompressed. A recent approach [14,15] reduced the noise contaminating images cor-rupted by both additive white and film grain noises using the linear combination of anumber of estimates derived from several transforms using local statistics. The trans-forms are performed using an adaptive criterion for the selection of the window size.Even though these methods seem very promising in terms of quality and generalityof the noise reduction, its real-time implementation appears to be problematical.

The grain noise reduction method described in this paper utilises one of themost popular classes of image noise reduction algorithms. These we refer to as trans-form-based schemes and are generic in nature. Initially, the image is decomposed


into a set of coefficients; then these coefficients are modified and, finally, the modifiedcoefficients are inverse transformed to yield the output image. The modification seeksto suppress coefficients that are suspected of being heavily contaminated with noise,whilst coefficients that are relatively unaffected by noise are retained in their originalform. These algorithms are characterised by the choice of transformation and themethod of coefficient modification employed.

The choice of transform to be used in the noise reduction algorithms is almostunlimited, the only formal requirement being that the transform is invertible. Howev-er, this does not imply that all transforms produce equally good results. The trans-form should concentrate the image information into relatively few coefficients,whilst the effect of the noise should be spread uniformly throughout the coefficients.Further, the transform should be such that the changes introduced at the coefficientmodification stage should create image artefacts that are perceptually acceptable. Thewavelet transform is well suited to the task in hand primarily because the introducedartefacts are perceptually more acceptable than those introduced by Fourier-basedmethods and is, like the Fourier method, computationally efficient. The multi-scalenature of the wavelet transform avoids these artefacts. The wavelet transforms forreducing additive image noise were proposed in a series of papers by Donoho [16–20]. The algorithm employed there exploited an orthogonal discrete wavelet trans-form. The use of orthogonal transforms is attractive since they are efficient represen-tations (only requiring as many coefficients as pixels in the image). However, one oftheir drawbacks relates to their lack of shift invariance. Non-redundant orthogonalwavelet transform has been recently also employed to reduce film grain noise duringthe procedure of compression [21,22]. The algorithm operates in the transformeddomain concurrently with quantization. This has been demonstrated to result inartefacts appearing close to edges in the image [23]. Latter, the use of redundant,non-orthogonal transforms has been proposed to mitigate the effects of these artefacts[16,17,24–29]. The transforms employed by these methods generate many more coef-ficients in the image representation than pixels present in the original image. Practi-cally, these methods require significantly more memory and computational load.

The second aspect that characterises a wavelet noise reduction technique is themethod of modifying the coefficients. The most widely used technique is to applya non-linear function to the coefficients. The non-linear function used is selectedfrom a parameterised family of functions, termed a scheme. The most widely usedschemes are the soft- and hard-thresholding functions [18–20,30,31], even if otherschemes have been described in the literature [27–29,31–34]. The value of parametersused to select the member of the scheme should depend on the statistics of signal andnoise.

The method for image noise reduction proposed here contains various attractiveelements. The transform used is a simple bi-orthogonal wavelet transform in whichsub-sampling does not take place. The transform used has a trivial inverse, which ismore advantageous in several ways for the real-time implementation of the algo-rithm. The scheme used for modifying the transform coefficient is more complexthan the classical soft- and hard-thresholding schemes. This scheme can approximatea range of optimal non-linear functions. The parameter values defining the member


of the scheme to be used are selected by training over a test set of images. The train-ing is conducted by contaminating the nominally noise-free image with various real-isations of grain noise. The training then ‘‘teaches’’ the algorithm the optimalparameters for the non-linear functions. The algorithm is trained and tested on dif-ferent images to demonstrate the generalisation ability of the scheme.

The optimisation requires a quantitative measure, but the only true measures ofimage quality are subjective. One of the advantages of this algorithm is that throughthe suitable choice of system parameters one can achieve a limited degree of percep-tual relevance to the metrics used in the training. The system parameters vary as afunction of the level of noise corruption, so in practice one needs to estimate thestandard deviation of the noise.

2. Film grain synthesis

2.1. Theory

The basic assumption on the structure of film grain is that it can be modelledusing a causal auto-regressive (AR) model [35]. Specifically, the grain image satisfiesthe equation:

XL

p¼0

XL

q¼0

ap;qxðm� p; n� qÞ ¼ eðm; nÞ; ð1Þ

where x (m,n) is the film grain, ap,q are the AR coefficients, with a0,0 = 1, L dictatesthe filter order (the restriction to a square aperture is made solely for the sake of sim-plicity), and e (m,n) is the innovations process. Eq. (1) defines a causal AR filter inthat e (m,n) depends only on passed (assuming a raster scan) grain values. It is as-sumed that e (m,n) is spatially uncorrelated and homogeneous, so that

E½eðm; nÞeðp; qÞ� ¼ r2 m ¼ p; n ¼ q;

0 elsewhere.

�ð2Þ

Note that the only assumption made with regard to the probability density function(pdf) of the innovations sequence is that it has a finite variance.

To estimate the AR coefficients from the data, one normally employs the correla-tion coefficients, defined as:

rðp; qÞ ¼ E½xðm; nÞxðm� p; n� qÞ�. ð3ÞA linear system of equations involving these coefficients can be obtained by multiply-ing (1) by x (m,n) and taking expectations; the resulting system of equations areknown as the Yule–Walker equations [35]. In practice, the correlation coefficientsare estimated from a finite block of data, of size M by M, using the biased estimator:

8rðp; qÞ ¼ 1

M2

XM�1

m¼p

XM�1

n¼q

xðm; nÞxðm� p; n� qÞ. ð4Þ


The correlation coefficients estimated via (4) can then be substituted into the Yule–Walker equations, which can then themselves be solved to obtain estimates of theAR coefficients. Finally, one can construct an estimate of the innovations sequence,eðm; nÞ, by application of (1) to the image. Further, one can invert the filter (1) toallow one to reconstruct the grain image from the innovations sequence. There area variety of techniques that can be used to accomplish this inversion, but we chooseto construct a reconstruction filter of the form:

8xðm; nÞ ¼

XN=2

p¼�N=2

XN=2

q¼�N=2

bp;qeðm� p; n� qÞ; ð5Þ

where the reconstruction filter uses a square aperture of dimension N + 1. The coef-ficients of this filter can be obtained by minimising the mean squared error, W1, overthe analysis block, with respect to the coefficients bp,q:

W1 ¼XM�1

m¼0

XM�1

n¼0

xðm; nÞ �8

xðm; nÞ½ �2. ð6Þ

This minimisation once again merely requires one to solve a system of linearequations.

Application of Eq. (5) to the innovations sequence allows one to create a grainimage from a spatially white innovations field. Thus, to generate a synthetic grainimage we first generate synthetic innovations data. There are several methods toachieve this goal: one could model the pdf of the innovations sequence and drawsamples from that pdf. However, this entails making assumptions about the formof the innovations pdf. The approach adopted to generate a new innovations fieldis to randomly draw values from eðm; nÞ, in a manner akin to that used in bootstrapmethods [36]. Assuming values of eðm; nÞ are drawn with equal probability then thesimulated field is guaranteed to have the same pdf as the original field. The resultingsynthetic grain field will be spatially independent, an assumption somewhat stricterthan (2). This leaves the possibility of some higher order spatial properties of the syn-thetic and actual grain fields being mismatched. In practice, we have not observedthat this lead to a visually perceptible difference.

In summary, the method consists of first estimating the filter coefficients forobtaining the innovations process from the grain data. The inverse of this filter isthen computed along with an estimate of the innovations field. The new grain field(of arbitrary size) is formed by creating a synthetic innovations field, via re-sampling,and filtering this field, using the inverse filter (5).

2.2. Implementation

In order to implement the method outlined in the previous subsection, one needsto obtain a sample of the grain field. This is achieved by asking the user to select aplain area of the image. This area can be a relatively small area, but it does need tocapture a representative sample of the grain. Once a plain area has been identifiedany underlying trends need to be removed. For areas in which the background


colour is truly constant then this requires one to subtract the mean from the region.In practice, the selected areas are often found to exhibit some general trends. Onecan apply a high pass filter to the area or fit and subtract off a higher order model,e.g., a plane. Some care has to be exercised during this de-trending process to ensurethat this process does not remove significant grain data. Indeed, the use of a grainremoval algorithm, as discussed in the next section, can make this process more ro-bust. Specifically, if one examines the error image between the original and imageafter grain removal typically contains large areas where only grain is evident.

Although we have found this process to very accurately replicate the statistics ofthe source grain, occasionally, users may still wish to manually fine tune the grainproperties. This is particularly the case if the scale of the grain has changed due toresizing either the source or reference image. Under these circumstances the scaleof the grain structure can be altered by appropriate interpolation of the reconstruc-tion filter b (p,q).

3. Film grain reduction

The details of the procedure used here to reduce film grain have been described inother works [27–29]; therefore in this section, we provide just an overview of themethod emphasising its key elements. The method is based on a wavelet decompo-sition of the image and on grain reduction on the components using parametersextracted during a preliminary training stage. In this section, we describe the funda-mental elements of the procedure: wavelet decomposition, film grain reduction onthe wavelet components, and the training process.

3.1. Wavelet-based image decomposition

There are several ways in which the wavelet transform (WT) can be applied toimages for noise reduction. The filter bank implementation is a widely used ap-proach using two pairs of filters, called the analysis and synthesis filters, to imple-ment the wavelet transform and its inverse. In a classical decimated filter bankimplementation, the wavelet coefficients are obtained by down-sampling the outputof the analysis filters. The inverse transform is computed by up-sampling and thenapplying the synthesis filters. Undecimated wavelet transforms do not employchanges in the sample rate. In the absence of the up and down samplers, a shiftin scale is achieved by inserting new zeros between each coefficient of the filterat every level of the decomposition. This is the algorithm a trous [38,39]. In bothdecimated and undecimated algorithms, this process is applied in a tree structure togenerate multiple levels of decomposition corresponding to information at differentscales.

In the wavelet decomposition of an image, the above procedure is most commonlyapplied first to the rows of the image and then to its columns, generating four com-ponents for each level of decomposition. In the case of colour images, this is repeatedon each colour component.


The analysis and synthesis filters are designed so that perfect reconstruction [37] isachieved. A range of classes of filters have been identified that satisfies this condition(e.g. [14]). In the case of decimated wavelets, the constraints imposed during this pro-cess are rather restrictive and severely limit the choice of filter. The use of undecimat-ed transforms provides greater freedom when designing the analysis and synthesisfilters, but also results in an increase in the amount of data stored during the decom-position. An additional benefit of using an undecimated transform is that it preservesthe shift invariance of the wavelet transform.

The particular filters used in this paper are:

H 0 zð Þ ¼ 1þ 2z�1 þ z�2� �

=4; G0 zð Þ ¼ 1;

H 1 zð Þ ¼ �1þ 2z�1 � z�2� �

=4; G1 zð Þ ¼ 1; ð7Þ

where the H (z) are the analysis filters and G (z) are the synthesis filters. This filter sethas the perfect reconstruction property and is well suited to real-time implementa-tion. It is also particularly useful that the final synthesis stage only requires one tosum the components. The decomposition procedure is applied to the three compo-nents of the RGB colour images allowing one to optimise the algorithm both interms of decomposition level and colour component.

3.2. Film grain reduction on the wavelet components

The wavelet decomposition generates a set of images (the wavelet components)that represent the image content at various scales and orientations. Film grain reduc-tion is then applied to the wavelet components via a non-linear operation, typically ageneralised form of thresholding, with the final output image being formed by sum-ming these components. It is a convenient and commonly used assumption that allthe thresholding functions applied to the components belong to one thresholdingscheme. A thresholding scheme is defined as a parameterised family of (non-linear)functions, fh (x), with h defining the parameters that specify the shape of a particularmember of the scheme. Thresholding schemes can be derived from various theoret-ical bases [30–32,34,40]. If one retains the assumption of Gaussianity on the noisebut considers various distributions for the underlying image, one can define variousoptimal thresholding schemes. Such schemes commonly have rather complex analyt-ical forms.

In order to construct a practical algorithm in our method, we employ a piece-wiselinear scheme as an approximation to these optimal schemes. This approximation is:

fh xð Þ ¼h2=h1ð Þx xj j < h1;

x� sign xð Þh1½ � h3�h2h3�h1

� �þ sign xð Þh2 h3 P xj j P h1;

x xj j > h3.

8>><>>:

ð8Þ

This is a piece-wise linear function that, in the first quadrant, passes through the ori-gin and the two points, (h1,h2) and (h3,h3). The evident disadvantage of using athresholding scheme with more controlling parameters is that the problem of

Fig. 1. General structure of the noise reducing scheme.


selecting suitable parameter values becomes more involved. The methodology weadopt to select the parameters is based on training on a representative set of imagesand noise.

Fig. 1 summarises the application of the thresholding scheme to one colour com-ponent of the image. This figure contains several features that should be brought tothe reader�s attention. A different member of the thresholding scheme is applied toeach component; the parameters specifying the thresholding function for the kthcomponent are contained in the vector hk. Since each level of decomposition gener-ates three new components, the total number of components generated to decom-pose a single colour component using L levels is 3L + 1. The final componentcontains the energy in the lowest frequency band. This residual image does not passthrough the thresholding function since it contains DC components of the image.This means that, in this component, the noise will not necessarily correspond tosmall coefficients. Fig. 1 also emphasises that the image synthesis only requiresone to sum the individual components of the same colour.

3.3. Algorithm training

Recall that in the system we seek to train, the parameters to be found are thosedefining the thresholding scheme. If the thresholding scheme depicted in Fig. 1 isused, a total of 9L parameters need to be defined for each colour component in orderto specify the noise reduction scheme. This large number of parameters raises twopotential problems. First, the optimisation task takes longer and, second, one runsthe risk of the system becoming specific to the particular image, or set of images, thatare used for training.

The training scheme involves taking a test image, I (m,n), (nominally grain noise-free), contaminating it with grain noise with prescribed characteristics. The noisy im-age, In (m,n), is then applied to the noise reduction algorithm and the error (differ-ence) between the output of the noise reducer and the original image is formed.The error signal is filtered, using a low-pass weighting function, in order to providea frequency-dependent weighting that approximates the perceptual importancein the high frequency region. The filtered error signal is denoted, e (m,n). The cost


function, W2, for training is the sum of the squared filtered errors and is a formweighted mean squared error (wmse):

W2 ¼XMm¼1

XNn¼1

eðm; nÞ2. ð9Þ

The above summation extends over the complete image. The weighting function canbe set to unity, in which case the cost function is equivalent to the mean squared er-ror (mse). The effectiveness of the training scheme clearly depends on the ability tocreate realistic fill grain, in this case on the effectiveness of the algorithm described inSection 2.

The cost function W2 needs to be minimised with respect to the parameters of thethresholding scheme and is not quadratic or convex nor can the optimum be foundanalytically. To achieve this optimisation, various techniques have been considered[27–29] including a Newton method and a genetic algorithm but simulations havedemonstrated that a Nelder–Meade [41] algorithm provided robust and rapidconvergence.

The optimisation is simplified by the choice of decomposition. The fact that thesynthesis is conducted using a summation means that the parameters are moredirectly related to the error signal. In a wavelet filter bank which includes separatesynthesis filters, the output of the non-linear elements is filtered prior to the compu-tation of the error, in which case the effect of each parameter is partially mixed by theaction of the filter and makes the action of changing a parameter more convoluted.

As already noted, our thresholding scheme is rather complex and leads to a highdimensional optimisation. In order to facilitate a reduction in the number of param-eters, various methods for limiting the number of parameters have been considered.The algorithm presented here exploits a specific form for the parameters at differentscales. If one defines the parameters for the kth component through the parametervector, hk, defined as:

hk ¼ ½h1k h2k h3k�. ð10Þ

Then our approximation entails the use of the parameter vector:

hk ¼ ½h1k ah1k bh1k�; ð11Þ

where the parameters a and b are scalars that are independent of the component.This leads to a threshold regime that requires one to only determine 3L + 2 param-eters for each colour component.

4. Results

4.1. Film grain synthesis

Figs. 2 and 3 show the results of grain replication on real film stocks. Figs. 2A and3A are samples of size 146 by 146 pixels of grain taken from test plates using Kodak

Fig. 2. (A) Kodak 320T film stock. (B) Enhanced brightness and contrast version of (A) to emphasisegrain. (C) Artificially synthesised grain from (A) with enhanced brightness and contrast.

Fig. 3. (A) Kodak 500T film stock. (B) Enhanced brightness and contrast version of (A) to emphasisegrain. (C) Artificially synthesised grain from (A) with enhanced brightness and contrast.


320T and Kodak 500T film. These images are repeated in Figs. 2B and 3B with in-creased contrast and brightness to emphasise the grain structure. Figs. 2C and 3Cshow the synthesized grain created using a 15 point filter, based on the 40 pixelsquare from the bottom left-hand corner of the source figures. The results are shownwith increased brightness and contrast. In order to illustrate the effectiveness of thescheme, the bottom left corner of Figs. 2C and 3C contains the original pixels andare not synthesised. Thus, there are adjacent areas of original and synthesised grainmimicking a CG element composited over film. In both cases, the result looks nat-ural and contiguous.

Figs. 4A and B show the result of applying the synthesized grain to a 195 by 195pixel image, the original form of which contained no film grain. The successful rep-lication of different styles of grain can be seen, with the more grainy character of theKodak 500T relative to 320T clearly evident.

4.2. Film grain reduction

In the first experiment, to evaluate the qualitative and quantitative performancesof the noise reduction technique presented herein we used two images artificially con-taminated by synthetically generated grain noise, as described above, simulating twodifferent Kodak film stocks: 320T and 500T. In Figs. 5 and 6, one can see the original

Fig. 4. (A) Synthesised Kodak 320T grain applied to tree and (B) synthesised Kodak 500T grain.


clean, 320 by 230 pixel image along with the grain contaminated images and the de-grained images obtained by applying the technique. The images were automaticallyprocessed using the technique previously presented without any human interventionfor tuning the parameters. The noise estimation was conducted using a plain area inthe blue background of the image. The parameters for the grain reduction algorithmwere determined by training on a set of six images [27] (this training set does not in-clude any of images used in this paper). Table 1 reports the noise reduction expressedin terms of percentage MSE reduction for each colour component and for the com-plete RGB image.

The qualities of the two film makes are different: the Kodak 500T introduces alarger amount of grain noise than the 320T and the percentage of MSE reductionis proportional to the amount of contamination introduced by the film. The noisegrain removal using our technique is visible in both cases even without any enlarge-ment and this produces a reasonable improvement in picture quality. It is noticeablethat for both the makes of film the percentage of grain removal is significantly small-er for the blue component than for the other components. This is because both imag-es contain a blue background, which has been utilised as a plane area to generate anestimate of the noise variance. Due to the high signal level in this region the grainnoise in this particular region is large, leading to an over estimate of the grain noisein the blue channel, which consequently results in poor performance in this channel.Manual tuning of the algorithm parameters can readily improve this performance,but in only some applications is such intervention acceptable.

In order to test the efficiency of the technique in real conditions, in the secondexperiment we used another two colour images (346 by 292 and 313 by 263 pixels)contaminated by real grain noise from Kodak Vision 320T and 500T. Figs. 7 and8 show sections (not enlarged) of the noise contaminated images (a), the images afternoise grain removal (b), and the images differences (c). In order to emphasize theamount of grain noise removed the difference images have been magnified by a factorof 10 and the negative images have been displayed.

Fig. 5. Original image (A), image artificially contaminated by grain noise (B), and de-grained image (C)for film make Kodak 320T.


The comparison between grain contaminated (a) and processed (b) images illus-trate that, again, the grain noise reduction is visible and the visual quality of theimages is significantly improved. On the other hand, the image differences (c) showthat a small amount of detail has been removed together with the noise in all threecases. This effect appears more evident where the amount of noise was higher.

Fig. 6. Original image (A), image artificially contaminated by grain noise (B), and de-grained image (C)for film make Kodak 500T.


The noise reduction over plain areas is estimated assuming that in these areas thenoise is highly predominant. The mean square error (MSE) has been used to com-pare quantitatively the performances of the technique in the two experiments. Table2 shows the total amount of MSE over plain areas in terms of standard deviationsfor each colour component and for the complete RGB image. Table 2 lists the per-centages of MSE reduction for each colour component and for the complete RGBimage.

Table 1MSE reduction over total images contaminated with grain noise synthetically generated and processed(Experiment 1)

MSE reduction % Red Green Blue RGB image

Kodak 320T 11 12.3 0.5 7.6Kodak 500T 21.8 25.2 13 20


The levels of MSE contamination and the percentages of MSE reduction for thefull RGB images in Table 2 demonstrate that, in general, the higher is the noise thelarger is the percentage of noise reduction obtained using the technique herein pre-sented. The tables also illustrate that in the two experiments the techniques per-formed differently (Table 2) depending on the distribution of the grain noiseover the colour components (Values in brackets). In the first experiment, the tech-nique performed almost uniformly over the R, G, and B colour components whilein the second experiment the technique adapts itself in order to tune the parame-ters processing each colour components depending on the level of the noise con-taminating that component.

5. Conclusions and discussion

This paper presented a method for synthesising and automatically reducing grainnoise contaminating colour images.

The automated technique for synthesising film grain which exactly matches a sam-ple swatch of grain has been detailed. The method requires only that the user select aplain area of image from which to sample the grain. The algorithm will then produceunlimited quantities of new grain that has a similar appearance to the original. Thistechnique is quicker than manually shaping noise to look like grain.

The technique to reduce the film grain is based on an undecimated wavelet trans-form and on the application of a thresholding scheme. The parameters characterisingthe thresholding scheme are obtained by training the system on a set of images thatare artificially corrupted with grain and a good choice of parameters is identifiedthrough the use of an optimisation algorithm.

The method has been tested on images contaminated by synthetically generatedgrain noise and on images contaminated by real grain noise. The results indicate thatthe algorithm provides visible improvement of the visual quality of the image evenwhen these are not enlarged. The noise reduction has been measured in terms of per-centage of MSE reduction, and the results demonstrate the ability of the technique toadapt itself depending on the characteristics of image and noise.

Another significant characteristic of the method presented in this work is that thisprocedure can be easily adapted to several conditions (class of images to be pro-cessed and type of contaminating noise) and requirements (enhancement type) usingdifferent filter banks, thresholding schemes, cost function, and class of images de-scribed by the training set.

Fig. 7. Image contaminated by grain noise (A), de-grained image (B), and image difference for film makeKodak 320T (C).


Fig. 8. Image contaminated by grain noise (A), de-grained image (B), and image difference for film makeKodak 500T (C).


Table 2Comparison of MSE reductions over the plain area of the images for the two experiments

MSE reduction % Red Green Blue RGB image

Experiment 1

Kodak 320T 38.4 (3.6) 44.6 (4.7) 48.2 (4.9) 43.7 (4.4)Kodak 500T 44.9 (4.3) 51.8 (5.4) 54.5 (5.4) 50.4 (5.0)

Experiment 2

Kodak 320T 45.4 (4.5) 43.8 (4.4) 21.3 (2.2) 35.2 (3.7)Kodak 500T 34.7 (3.5) 31.5 (3.2) 67.6 (5.7) 45.3 (4.1)

Values in brackets show the standard deviations of the original image.


References

[1] L. Stroebel, J. Compton, I. Current, R. Zakia, Photographic Material and Processes, Focal Press,1986.

[2] J. Yan, P. Campisi, D. Hatzinakos, Signal-dependent film grain noise generation using homomorphicadaptive filtering, IEE Proc. Vision Image Signal Process. 147 (3) (2000).

[3] A. Ashikhmin, Synthesizing natural textures, in: Symposium on Interactive 3D Graphics, 2001.[4] A.A. Efros, W.T. Freeman, Image quilting for texture synthesis and transfer, in: ACM SIGGRAPH

01, 2001.[5] A. Sharma, Digital noise, film grain, digital photo techniques, Nov/Dec 2001, pp. 62–64, 2001.[6] B.B. Hunt, Bayesian methods in non-linear digital image restoration, IEEE Trans. Comput. C-26 (3)

(1977) 219–229.[7] A.S. Tavildar, H.M. Gupta, S.N. Gupta, Maximum a posteriori estimation in presence of film grain

noise, Signal Process. 8 (3) (1985) 363–368.[8] R. Kasturi, J.F. Walkup, T.F. Krile, Maximum a posteriori image estimation in signal dependent

noise, in: Proceedings of the 7th Conference on Pattern Recognition, Cat. No. -84CH2046-1, vol. 1,1984, pp. 148–150.

[9] D.G. Falconer, Image enhancement and film grain noise, Opt. Acta 17 (9) (1970) 693–705.[10] A.S. Tavildar, H.M. Gupta, S.N. Gupta, Linear MMSE filtering for restoration of images degraded

by film grain noise, Signal Process. 6 (3) (1984) 225–234.[11] I. McLean, S. Witt, Telecine Noise reduction, IEE Seminar Ref.-No. 01/049, vol. 2, 2001, pp. 1–6.[12] M. Fischer, J.L. Paredes, G.R. Arce, Image sharpening using permutation weighted medians, in:

Proceedings of the 10th European Signal Processing Conference, vol. 1, Tampere, Finland, 2000, pp.1–4.

[13] M. Fischer, J.L. Paredes, G.R. Arce, Weighted median image sharpeners for World Wide Web, IEEETrans. Image Process. 11 (7) (2002) 717–727.

[14] K. Egiazarian, V. Katkovnik, J. Astola, Local transform based image de-noising with adaptivewindow size selection, Proceedings of the SPIE, The International Society for Optical Engineering4170 (2000) 13–23.

[15] K. Egiazarian, V. Katkovnik, J. Astola, Adaptive window size image denoising based on ICI rule, in:IEEE Proceedings Cat.-No. 01CH37221, vol. 3, 2001, pp. 1869–1872.

[16] R.R. Coifmann, D.L. Donoho, Translation-invariant de-noising, in: A. Antoniadis (Ed.), Waveletsand Statistics, 1995.

[17] A. Bruce, D. Donoho, H. Gao, Wavelet analysis, in: IEEE Spectrum, February 1996, pp. 26–35.[18] D.L. Donoho, I.M. Johnstone, Wavelet shrinkage: Asymptopia? J. R. Stat. Soc. 57 (2) (1995) 301–

369.[19] D.L. Donoho, I.M. Johnstone, Ideal spatial adaptation by wavelet shrinkage, Biometrika 81 (3)

(1994) 425–455.[20] D.L. Donoho, De-noising by soft-thresholding, IEEE Trans. Inform. Theory 41 (3) (1995) 613–627.


[21] R. Oktem, K. Egizarian, A wavelet method for coding film grain noise corrupted images, in: IEEEProceedings Cat.-No. 00CH37100, vol. 4, 2001, pp. 2063–2066.

[22] R. Oktem, K. Egizarian, Transform domain algorithm for reducing effect of film grain noise on imagecompression, Electron. Lett. 35 (21) (1999) 1830–1831.

[23] S. Mallat, S. Zhong, Characterisation of signals from multiscale edges, IEEE Trans. Pattern Anal.Mach. Intell. 14 (7) (1992) 710–732.

[24] L.L. Schumaker, G. Webb, Recent Advances in Wavelet Analysis, Boston Academic Press, 1994.[25] H.L. Resnikoff, Wavelet Analysis: The Scalable Structure Information, Springer, New York, 1998.[26] N. Kingsbury, Complex wavelet and shift-invariance, in: IEE Colloquium, Section 5, 2000, pp. 1–10.[27] A. De Stefano, Elimination video noise, Ph.D. Thesis, ISVR University of Southampton,

Southampton, UK, 2000.[28] A. De Stefano, P.R. White, W.B. Collis, An innovative approach for spatial video noise reduction

using wavelet based frequency decomposition, in: ICIP 2000 IEEE Conference, Vancouver, 2000.[29] A. De Stefano, P.R. White, W.B. Collis, Selection of thresholding scheme for video noise reduction on

wavelet components using Bayesian estimation, in: Proceedings of the 5th IMA Conference on Signalprocessing, Warwick, UK, 2000.

[30] M. Jansen, A. Bultheel, Geometrical priors in a Bayesian approach to improve wavelet thresholdprocedures, SPIE Proc. Series 3813 (1999) 580–590.

[31] J.M. Lina, B. MacGibbon, Non-linear shrinkage estimation with complex Daubechies wavelets, SPIEProc. Series 3169 (1997) 67–79.

[32] F. Abramovich, Y. Benjamini, Thresholding of wavelet coefficients a multiple hypotheses testingprocedure, in: A. Antoniadis (Ed.), Wavelets and Statistics, 1995.

[33] P. Moulin, J. Liu, Analysis of multiresolution image denoising schemes using generalised Gaussianand complexity priors, IEEE Trans. Inform. Theory 45 (1999) 909–919.

[34] D. Wei, C.S. Burrus, Optimal wavelet thresholding for various coding schemes, IEEE Trans. ImageProcess. 1 (1995) 610–613.

[35] S. Kay, Modern Spectral Estimation: Theory and Application, Prentice-Hall, Englewood Cliffs, NJ,1987.

[36] R. Lepage, L. Billard, Exploring the Limits of Bootstrap, Wiley, New York, 1992.[37] M.J. Shensa, The discrete wavelet transform: wedding the a trous and Mallat algorithms, IEEE Trans.

Signal Process. 40 (10) (1992) 2464–2482.[38] P. Duttilleux, An implementation of the algorithm �a trous� to compute the wavelet transform, in:

Wavelet, Time-Frequency Methods and Phase Space, IPTI, Berlin, 1989, pp. 298–304.[39] I.Y. Soon, S.N. Koh, C.K. Yeo, Noisy speech enhancement using discrete cosine transform, Speech

Commun. 24 (3) (1998) 249–257.[40] P.M.B. van Roosmalen, S.J.P. Westen, J. Biemond, Noise reduction for image sequences using

oriented Pyramid thresholding Technique, IEEE Int. Conf. Image Process. 1 (1996) 375–378.[41] W.H. Press, B.P. Flannery, S.A. Teukolsky, W.T. Vettering, Numerical Recipes in C, Cambridge

University Press, Cambridge, 1988.

Antonio De Stefano is Distance Learning Coordinator for the Master TrainingPackages in Biomedical Signal Processing at the ISVR in the University ofSouthampton, UK. He received the Electronics Engineering Degree, the special-ization in Biomedical Sciences and the CEng qualification from the UniversityFederico II in Naples (Italy), and Ph.D. from the ISVR in the University ofSouthampton (UK). His research interests include wavelet-based noise reductionand image enhancement, biomedical signal processing, implementation of distancelearning packages for biomedical subjects, techniques for EMG analysis duringwalking of pathological children, and mechanical models for speech design. Hehas over 30 publications in journals and international conferences.


Dr. William Collis has been Algorithms Engineer at The Foundry since December2000. Prior to joining the Foundry, Bill led the algorithms team at Snell & Wilcoxfor 5 years working on standards conversion, motion estimation, archive restora-tion, and filter design. During this period he also developed the Flo-Mo re-timingsoftware used in the pioneering �bullet time� sequences in the film, The Matrix. Billgraduated with a first in Electrical Engineering from Southampton University, andwent on to complete a Ph.D. in non-linear signal processing at the Institute ofSound and Vibration Research, where he still holds the position of AssociateFellow. Bill is the author of a book, over 30 research papers, and five patents.

Paul White is currently a senior lecture in the Institute of Sound and Vibration
Research in the University of Southampton. He attained his degree in AppliedMathematics from Portsmouth Polytechnic in 1985, whereupon he joined ISVR tostudy for his Ph.D. In 1988 he was made a lecturer within ISVR, finally completinghis Ph.D. in 1992. In 1998 he was made a senior lecturer. The basic signal pro-cessing techniques that have formed the basis of this work include time-frequencyanalysis, non-linear systems, adaptive systems, detection and classification algo-rithms, higher order statistics, and independent component analysis. The appli-cation areas to which he has considered applying these include image processing,
underwater systems, condition monitoring, and biomedical applications. He has published in excess of 130papers in the field, approximately 35 of these appearing in referred Journals or as chapters in books. He isa member of the Editorial board of the Journal of Condition Monitoring and Diagnostic EngineeringManagement (COMADEM) and is a member of the IEEE.

Synthesising and reducing film grain

Documents

Transcript of Synthesising and reducing film grain