Processing of I mages B ased on B lind E valuation of N oise T ype and C haracteristics

1

Vladimir Lukin 31/08/2009

Processing of Images Based on Blind Evaluationof Noise Type and Characteristics

Processing of ImagesBased on Blind Evaluation

of Noise Type and CharacteristicsVladimir V. Lukina, Sergey K. Abramova, Nikolay N. Ponomarenkoa,

Mikhail L. Ussa,b, Benoit Vozelb, Kacem Chehdib,Jaakko Astolac

a National Aerospace University, 61070, Kharkov, Ukraine;b University of Rennes I, 22 305 Lannion cedex, BP 80518, France; c Tampere University of Technology, Institute of Signal Processing,

P.O. Box-553, FIN-33101, Tampere, Finland

2


Contents

Contents

• Introduction• A priori Knowledge on Noise Statistics, Its Use in Image Processing• Examples of False Preliminary Assumptions on Noise Type• General Aspects of Image Processing• Methods for Blind Determination of Noise Type• Methods for Blind Determination of Noise Statistical Characteristics• Edge Detection Example• Requirements to Methods of Blind Determination of Noise Statistical Characteristics• Image Filtering Examples• Noisy Image Compression• Automatic Procedure for Lossy Compression• Accuracy of Blind Estimation Methods• Conclusions

3


Introduction

Applications: Systems for single- or multi-channel remote sensing (RS) data processing

Goal: Carrying out analysis of mixed noise influence for three particular operations of image processing: edge detection, filtering and compression of noisy images

Additivenoise

Influence on: edge detection, filtering and compression

Reasons: transmitting and encoding/decoding

errors

Multiplicative noise

Reason: circuit and atmospheric

noise

Blind noise parameters evaluation→ inaccurate estimates may occur

Impulse noise

Reasons: insufficient observation interval,

speckle effects

Image processing

Blind noise type determination→ false decisions may occur

RS image

4


A priori Knowledge on Noise Statistics, Its Use in Image Processing

Characteristics of noise:

1. Noise type (additive, signal-dependent, multiplicative, impulse, mixed, etc.);

2. PDF, variance, relative variance, probability of impulse noise, etc.;

3. Is noise i.i.d. or spatially correlated; 2D ACF or spectrum in the latter case.

Examples:

1. Noise PDF is needed to design optimal edge detectors and to set thresholds for most known edge detectors;

2. Noise statistics is used in image filtering (wavelets, DCT, sigma, Lee, Frost, Kuan) and reconstruction (Katsaggellos et al);

3. Lossy compression of noisy images.

Peculiarities:

There are no universal methods for coping well with any type and level of noise.

Specialized methods that exploit available information on noise type and its characteristics in full extent as a rule provide the most efficient processing.

5


A priori Knowledge on Noise Statistics, Its Use in Image Processing

Statement:Information on noise type and statistics is to be available or it should be

retrieved for data (image) at hand.

Otherwise, a used method has to exhibit certain robustness, i.e. ability to perform well enough in conditions of some unknown parameters and characteristics of noise.

Fortunately, some methods for blind noise model identification and estimation of its parameters exist and they are rapidly developing*.

Goal:Carrying out analysis of these methods for three particular operations of image

processing, namely, edge detection, filtering and compression of noisy images.

* B. Vozel, S. Abramov, K. Chehdi, V. Lukin, N. Ponomarenko, M. Uss, Blind methods for noise evaluation in multi-component images, book chapter in “Multivariate image processing”, ISTE Ltd, France, 2009 (in press).

6


Examples of False Preliminary Assumptions on Noise Type: Optical Images

The noisy (dominant Poisson) test image (left) and this image after Gamma-correction (γ = 0.6) (right)

Scatter-plots of local estimates of variance for original noisy (left) and corrected images (right)

2ˆloc

l̂ocI

2ˆloc

l̂ocI

7


Examples of False Preliminary Assumptions on Noise Type: Radar Images

A real life K-band SLAR image

Scatter-plot of local estimation pairs (variance and squared mean) with two approximation

lines fitted by expert (green line) and automatically (red line)

Other examples: a model of fully developed speckle, noise in ultrasound images, noise in sub-band

images of hyperspectral remote sensing data.

2ˆloc

2l̂ocI

8


General Aspects of Image Processing

1. Noise is a phenomenon inherent for any imaging and RS system;

2. Noise presence is often a dominant factor that prevents solving final tasks for which obtained data are intended for;

3. These tasks could be: object detection and localization, sensed terrain classification, segmentation, edge detection, image filtering, segmentation, compression, classification, etc.

4. There are some methods of image processing that do not exploit information on noise type and characteristics (like standard median filter) but methods that use a priori information on noise type and characteristics are able to perform better;

5. There is a lot of different types of noise and their combinations.

9


Methods for Blind Determination of Noise Type

An automatic system to identify the nature of the degradation affecting an image and select the appropriate algorithm of its processing has been designed*.

The system is intended for identifying four sources of degradation made of three different noises: additive, multiplicative and impulse noise and a defocusing blur.

Thirteen observation models have been deduced from these four sources: each noise type separately and different possible combinations.

The system has been tested for different simulated data and provided appropriately high probability of correct identification of noise/distortion type.

* Carton-Vandecandelaere, M.-P., Vozel, B., Klaine, L., Chehdi, K., Application to Multispectral Images of a Blind Identification System for Blur, Additive, Multiplicative and Impulse Noises. Proceedings of EUSIPCO, III, 2002, pp. 283-286.

10


Methods for Blind Determination of Noise Statistical Characteristics

There are methods for blind estimation of: additive noise variance*;

mixed noise parameters***.

* V. Lukin, S. Abramov, B. Vozel, K. Chehdi, J. Astola, Segmentation-based method for blind evaluation of noise variance in images, SPIE Journal on Applied Remote Sensing, Vol. 2, Aug. 2008, 15 p.

** Lukin V.V., Abramov S.K., Ponomarenko N.N., Vozel B., Chehdi K., Methods for blind evaluation of noise variance in multichannel optical and radar images, Telecommunications and Radio Engineering, Vol. 65 (6), 2006, pp. 509-537.

*** A. Foi, Pointwise Shape-Adaptive DCT Image Filtering and Signal-Dependent Noise Estimation: Thesis for the degree of Doctor of Technology, Tampere University of Technology, Tampere, Finland, Dec. 2007.

They are based on different principles: Curve fitting to scatter-plots;

Analysis of local estimate histograms;

Data analysis in spectral domain;

Bit-plane analysis, etc.

multiplicative noise variance**;

11


Edge Detection Example

Noise free (a) and noisy (mixed multiplicative and impulse noise, ) (b) test images; edge maps obtained by Sobel (c) and normalized quasirange (d) edge detectors

Principle:For most of edge detectors, a local parameter is calculated for each pixel

using image values in a given pixel and its neighbors. Then this local parameter is compared to a preset threshold that takes into account noise type and statistics.

2 0.003, 0.01P а b c d

Example:

12


Edge Detection

Requirement to accuracy:It is desirable to provide within the limits from till .

Procedure: to estimate local mean; to calculate ; to determine the corresponding local threshold; to obtain a detector output; to compare it to the local threshold; to undertake a decision.

2ˆn 20.9 n 21.1 n

Signal-dependent noise case:

If a dependence of local variance on local mean is known in advance or pre-estimated, then a threshold for an edge detector should be set locally as

or

2 ( )loc locf I

1loc locT 22loc locT

2 ˆˆ ( )loc locf I

13


Requirements to Methods of Blind Determination of Noise Statistical Characteristics

1. To produce unbiased estimates;

2. To provide appropriately accurate estimates (characterized by small variance of noise parameter estimation);

3. To be applicable to images of different content and to noise of different level and with different spatial characteristics (a wide range of noise parameters’ variation can be observed in practice);

4. To be fast enough.

Note: if an image is multichannel, processing is component-wise and it can be parallel.

A question is what are appropriately accurate estimates?

14


Image Filtering Examples:Test Images and Quantitative Criteria

Goal:Consider how filtering efficiency depends upon accuracy of additive noise

variance estimation.

Test images: Quantitative criteria: An error of additive noise variance

estimation:

Variation of filtering efficiency

2 2 2 2ˆ ˆ( ) /n n n

2 2 2ˆ( ( ) ( )) / ( )n n nMSE MSE MSE

Where is MSE provided under condition that is set as the parameter of a used filter.

2( )nMSE 2n

Test01 Test02

15


Image Filtering Examples:Modified Sigma Filter

The plots vs for the test images for 100 and 2252 n

Preliminary conclusion:For the image Test01, smaller output MSE can be reached if when better

edge/detail/texture preservation is provided.

The opposite situation is observed for the image Test02. If , smaller output MSE is produced. Thus, does not necessarily lead to the best efficiency of noise suppression (according to output MSE criterion).

If is within the limits from 0.8 to 1.2, then it is almost guaranteed that one is in the neighborhood of minimal output MSE. This is observed for other filters as well as for multiplicative noise.

2 2ˆn n

2̂

2 2ˆ n n 2 2ˆ n n

2 2ˆn n

2̂

The noise variance was estimated in automatic manner robust to the presence of impulses. The resulting estimate is 24.93.

16


Image Filtering Examples:Blind Noise Type and Variance Retrieval

The original NOAA image (a), filtered image (b), and the obtained impulse noise map (c)

2ˆ n

Real-life image:

a

b

c

Identification result:

Image is identified as corrupted by mixed additive and impulse noise.

Impulse noise is specific, spatially correlated, where several neighboring image pixels in rows are corrupted.

Estimation result:

17


Image Filtering Examples:Blind Noise ACF Retrieval

Original SAR image (left) and the obtained filtered image (right)

* V. Lukin, N. Ponomarenko, K. Egiazarian, J. Astola, Adaptive DCT-based filtering of images corrupted by spatially correlated noise, Proc. SPIE Conference Image Processing: Algorithms and Systems VI, Vol. 6812, 12 p., 2008.

If fluctuative noise type is determined correctly and its variance is estimated with high enough accuracy, it becomes possible to blindly evaluate spatial correlation properties of noise as well*.

These estimates can be further exploited in DCT based filtering. Taking into account noise spatial spectrum leads to PSNR increasing by 1...3 dB.

18


Image Filtering Examples:Fully Automatic Procedure

Stages:1. Blind determination of noise type, additive or multiplicative (if there is no a

priori information);

2. Estimation of its statistical characteristics (variance);

3. Estimation of spatial correlation properties of noise;

4. Image filtering with taking into account the obtained estimates of noise characteristics.

Features:This procedure is also applicable if both additive and multiplicative noise

components have been identified in a given image.

Note that DCT based filters can be easily modified to mixed additive and multiplicative or other signal-dependent noise.

Similar modifications are not a problem for the family of sigma filters and hard-switching locally adaptive filters.

19


Image Filtering Examples:3D Processing of Multichannel Images

a b (PSNR= 22.87 dB)

c (PSNR= 25.17 dB) d (PSNR= 30.01 dB)

Automatic procedures based on blind estimation described above can be used for both component-wise and 3D processing of multichannel images.

Improvement of PSNR due to 3D processing is 0.6 dB in comparison to component-wise DCT based processing.

The noise-free test image Goldhill (a), noisy image ( ) (b), intermediate output image (c), resulting output image (d)

Preliminary conclusion:

2 225, 0.01 n P

Noisy Image Compression:Quantitative Criteria

20


The standard measures to characterize a compressed image quality

- , where is the decompressed image; 2

1 1

( ) /( 1)I J

d nor ij ij

i j

MSE G G IJ

dijG

It is more reasonable to characterize a compressed image quality by quantitative measures calculated with respect to the corresponding noise-free image (MSEnf, PSNRnf) rather than to the original noisy one (MSEor, PSNRor).

- - for 8 bits image representation.21010log (255 / )or orPSNR MSE

Alternative measures to characterize a compressed image quality

- , where is the noise free image; 2

1 1

( ) /( 1)I J

dnf ij ij

i j

MSE G G IJ

ijG

- .21010log (255 / )nf nfPSNR MSE

Noisy Image Compression:Optimal Operation Point

21


0

50

100

150

200

250

2 3 4 5 6 7 8 9 10 11

QSn

MS

En

f

Dependences MSEnf (QSn) for the noisy test gray-scale image Lena for different additive

noise levels

Optimal operation point (OOP): The argument of the curves MSEnf (CR), MSEnf (bpp) or MSEnf (QS) for which these

curves reach theirs minima have been called optimal operation point (OOP): CROOP , bppOOP or QSOOP..

OOP is observed and commonly occurs to be more “obvious” for less complex content images and/or for rather intensive noise.

Main idea:It is worth compressing a noisy image in

the neighborhood of OOP.

Main problem:In practice, noise-free image is not at

disposal.

σ2n = 400 σ2

n = 100 σ2n = 50

n nQS QS n OOPQS

Noisy Image Compression:Peculiarities

22


Why lossy (not lossless) compression?

1) Lossy compression is able to provide considerably larger CRs (compared to lossless coding) without degrading image resolution and introducing disturbing artefacts;

2) A positive effect of image filtering can be observed due to lossy compression if introduced losses mainly relate to noise removal and useful image content is preserved.

The RS (Helsinki region) image corrupted by additive Gaussian noise with σ2

n = 100The decoded lossy compressed image

(bpp = 0.75)

Noisy Image Compression:Real-life Test Images

23


The compressed image (HBC strategy,AGU coder with QSA=35)

The automatic procedure for attaining OOP have been also designed for pure multiplicative and Poisson noise.

The corresponding homomorphic transform is to be applied (for making noise additive) before compression.

Inverse homomorphic transform is to be carried out after decompression.

The noisy real-life test image Frisco

24


Automatic Procedure for Lossy Compression

Stages:1. Blind determination of noise type: additive, Poisson or multiplicative;

2. Estimation of noise statistical characteristics;

3. Carrying out proper forward homomorphic transform if noise is Poisson or multiplicative; calculation or estimation of variance after homomorphic transform (if needed);

4. Carrying out lossy compression with providing OOP by either setting QS according to above recommendations or by finding proper bpp (for JPEG2000 or SPIHT coders, see details in the paper*).

* N. Ponomarenko, V. Lukin, M. Zriakhov, K. Egiazarian, J. Astola, Lossy compression of images with additive noise, Proceedings of International Conference on Advanced Concepts for Intelligent Vision Systems, Antwerpen, Belgium, September, 2005, pp. 381-386.

Note that are to be within the limits from till . .2n 20.8 n 21.2 n

Accuracy of Blind Estimation Methods: TID2008 (Tampere Image Database)

25


Test noise-free color images of TID2008 (http://www.ponomarenko.info/tid2008.htm)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 2520

40

60

80

100

120

140

160

180

200

220

Test image index

^ n 2

Blue component

Green componentRed component

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 2510

20

30

40

50

60

70

80

90

100

110

Test image index

^ n 2

Blue component


26


Accuracy of Blind Estimation Methods:Spatial Domain

Noise variance estimates obtained by the methods* (left) and** (right) with nonoverlapping 5x5 pixels block size for red, green and blue components for the test image set corrupted by spatially

uncorrelated additive noise with =65

* Lukin V.V., Abramov S.K., Vozel B., Chehdi K. Improved minimal inter-quantile distance method for blind estimation of noise variance in images, Proceedings of SPIE/EUROPTO Symposium on Satellite Remote Sensing, Florence, Italy, Sept 2007, SPIE Vol. 6748, 12 p.

** V. Lukin, S. Abramov, B. Vozel, K. Chehdi, J. Astola, Segmentation-based method for blind evaluation of noise variance in images, SPIE Journal on Applied Remote Sensing, Vol. 2, Aug. 2008, 15 p. (open access paper)

2n

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 2520

40

60

80

100

120

140

160

Test image index

^ n 2

Blue component


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 250

10

20

30

40

50

60

70

80

Test image index

^ n 2

Blue component


27


Accuracy of Blind Estimation Methods:Wavelet Domain

Noise variance estimates obtained by the method* for red, green and blue components for the test image set corrupted by spatially uncorrelated (left) and spatially correlated (right) additive noise with

= 65

* L. Sendur, I.W. Selesnick, Bivariate shrinkage with local variance estimation, IEEE Signal Processing Letters, Vol. 9, No 12, 2002, pp. 438-441 (see also http://taco.polv.edu/WaveletSoftware/index.html)

2n

Be careful with spatially correlated noise !!!

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 250

50

100

150

200

250

Test image index

^ n 2

Blue component


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 250

20

40

60

80

100

120

140

160

180

200

Test image index

^ n 2

Blue component


28


Accuracy of Blind Estimation Methods:Spatial Domain & Spatially Correletad Noise

Noise variance estimates obtained by the method* with nonoverlapping 5x5 (left) and 7x7 (right) pixels block size for red, green and blue components for the test image set corrupted by spatially

correlated additive noise with =65

* Lukin V.V., Abramov S.K., Vozel B., Chehdi K. Improved minimal inter-quantile distance method for blind estimation of noise variance in images, Proceedings of SPIE/EUROPTO Symposium on Satellite Remote Sensing, Florence, Italy, Sept 2007, SPIE Vol. 6748, 12 p.

2n

Summary: It is better using 7x7 blocks to have less problems with (possibly) spatiall correlated noise.

29


Conclusions

1. It is possible to design multi-stage automatic procedures of image processing based on blind determination of noise type and estimation of its characteristics.

2. The accuracy of such estimates is commonly enough for carrying out efficient edge detection, filtering, and lossy compression in automatic manner.

3. Currently only quite simple situations like identified pure additive or pure multiplicative noise are studied well enough; design and performance analysis for other noise environments are far from completeness.

4. Special attention should be paid to multichannel and hyperspectral images for which, on one hand, necessity to make processing automatic is the most actual and, on the other hand, the situation with such design is the most complex.

Processing of I mages B ased on B lind E valuation of N oise T ype and C haracteristics

Documents

Transcript of Processing of I mages B ased on B lind E valuation of N oise T ype and C haracteristics