Presentation v3.2

Click here to load reader

  • date post

  • Category


  • view

  • download


Embed Size (px)

Transcript of Presentation v3.2

  1. 1. INNOVATIVE METHODS FOR THE RECONSTRUCTION OF NEW GENERATION SATELLITE REMOTE SENSING IMAGES November 29th, 2012 PhD Student: Luca Lorenzi Ph.D. thesis defense Advisors: Farid Melgani Grgoire Mercier
  2. 2. Introduction General Problem Missing data in VHR optical image; Mainly due to acquisition conditions, e.g., the presence of: clouds: partially or completely missing data; shadows: partially missing data. 2 MODIS: Black see GeoEye-1: Doha Stadium, Qatar
  3. 3. Introduction General Solution A common solution approach: 1. Pre-process the image (co-registration, calibration); 2. Detect the location of the contaminated regions; 3. Attempt to restore the missing areas. 3 Missing Area Detection Missing Area Reconstruction Original Image Image Pre- Processing Restored Image
  4. 4. Objective Propose new methodologies for the reconstruction of missing areas for new generation satellite remote sensing images. Missing areas due to the presence of: Clouds Shadows 4
  5. 5. Cloud-Contaminated Images Contribution 1: different solutions based on the inpainting approach. Three strategies: local image properties; isometric transformations; multiresolution processing scheme. Contribution 2: to improve the reconstruction process by integrating both radiometric and spatial information, through a specific kernel. Contribution 3: new methods based on the Compressive Sensing theory. Three strategies: Orthogonal Matching Pursuit (OMP); Basis Pursuit (BP); An alternative solution based on Genetic Algorithms (GAs). 5
  6. 6. Shadow-Contaminated Images Contribution 4: a novel approach to solve both problems of detection and reconstruction. Shadow detection is performed through a hierarchical supervised classification scheme, while the proposed reconstruction relies on a linear prediction function, which exploits information returned by the classification. Contribution 5: to try to answer to the following question: Is it possible to know a priori if a shadow area can be well recovered? Eight different criteria. A fuzzy logic combination is explored. 6
  7. 7. two contributions in detail In the next Slides
  9. 9. Problem formulation In I(1), any pixel can be expressed as: We have to evaluate: From I(1) : In I(2) : 9 2,1,)()()( iI iii )1()1( x )1()1()1( ,, lkx )1()1( , xf )2()2( x f ? source area missing area
  10. 10. Compressive sensing (CS) Compressive sensing theory [1] Idea: exploit redundancy in signals Signals like images are sparse many coefficients close to zero To enforce the sparsity constraint, CS finds a vector which minimizes: D is a dictionary with a predefined number of atoms; x is the original pixel, expressed by a linear combination of atoms; Eq. (1) represents a NP-hard problem computationally infeasible to solve. Cands and Tao [2], reduce the Eq. (1) in a relatively easy linear programming solution: under some reasonable assumptions: 10 Dxsubject tomin 0 0:#0 ii (1) [1] D. L. Donoho, Compressed Sensing, IEEE Trans. Inf. Theory, vol. 52, no. 4, pp. 1289-1306, Apr 2006. [2] E. J. Cands and T. Tao, Decoding by Linear Programming, IEEE Trans. Inform. Theory, vol. 51, no. 12, pp. 4203-4215, Dec. 2005. 01 minmin
  11. 11. Orthogonal Matching Pursuit (OMP) 12 [3] Y. C. Pati, R. Rezaiifar and P. S. Krishnaprasad, Orthogonal Matching Pursuit: Recursive Function Approximation with Applications to Wavelet Decompositions, in Proc. 27th Asilomar Conf. on Sig., Sys. and Comp., Nov. 1-3, 1993. Orthogonal Matching Pursuit (OMP) [3] finds the atoms which has the highest correlation with the signal: where dictionary D is a collection of atom vectors and R(m) is a residual. It updates the coefficients of the selected atoms at each iteration (adopting a least-squares step), so that the resulting residual vector R is orthogonal to the subspace spanned by the selected atoms. )( 1 m m i dd Dd dd Rx ii Ddd
  12. 12. Orthogonal Matching Pursuit (OMP) 13 OMP pseudo code i=0: x(0)=0 , R(0)=x and D(c0)={} i=k: i=m: ,)( 1 )( m m i dd m Rx ii )(m RR Step 1: find which ; add to the set of selected variables; update . Step 2: let denote the projection onto the linear space spanned by the elements of update Step 3: compute s.t. Step 4: if ||R(k)||
  13. 13. Basis Pursuit (BP) To solve Eq. (1), we may adopt the Basis Pursuit principle [4]: convexification from L0 to L1; Thanks to that, it becomes a support minimization problem; Eq. (2) can be reformulated as a linear programming (LP) problem, and solved using the Simplex methods. Given that, it is possible to rewrite L1 norm in Eq. (2) as: where If we substitute it in Eq. (2), it allows to perform a linear minimization problem. 14 [4] S. S. Chen, D. L. Donoho and M. A. Saunders, Atomic Decomposition by Basis Pursuit, SIAM J. on Sci. Comp., vol. 20, pp. 33-61, 1999. Dxsubject tomin 1 (2) i iii i vu1 00, 00, iiii iiii ifuv ifvu
  14. 14. Genetic Algorithm (GA) To cope with complex optimization problems, there exist metaheuristic techniques, like the evolutionary algorithms. Genetic Algorithms (GAs) [5] are: inspired by evolutionary biology; general purpose randomized optimization techniques; based on simple rules. Their basic idea is to evolve iteratively a population of chromosomes, each representing a candidate solution to the considered problem. In complex problems requiring the simultaneous optimization of multiple objectives, GAs are particularly indicated for they deal simultaneously with a population of solutions. Advantages: little information about the problem required; robust to local optima; optimization of real, integer and binary problems. 15 [5] L. Chambers, The Practical Handbook of Genetic Algorithms. New York: Champan & Hall, 2001.
  15. 15. GA Principal Steps 1. Initial population of M chromosomes is generated; 2. The goodness of each chromosome is evaluated according to predefined fitness functions; 3. Successively, the GA favors the selection of the best chromosomes and removes the others; 4. In the next step a new population is represented by adopting genetic operators, e.g., crossover and mutation; 5. All these steps are iterated until a predefined condition is satisfied. 16 Condition satisfied? Nextiteration Y N
  16. 16. GA Setup Idea: exploit the GA capabilities for solving the L0 norm problem. The NP-hard problem is: Chromosome structure: The number of chromosomes M must be fixed; i is a chromosome which contains w genes with real values; The length w of the chromosome is thus equal to the one of the dictionary D. 17 Gene 1 Gene 2 Gene w i1 i2 iw Chromosome i Fitness functions: Multiple objective optimization [6] 01 min f 2 22 min xDf NSGA-2 Dxsubject tomin 0 (1) [6] K. Deb, Multi-Objective Optimization Using Evolutionary Algorithms. Chichester, U.K.: Wiley, 2001.
  17. 17. Experimental Dataset Aim: to compare the results obtained by the CS reconstructions with: Multiresolution Inpainting (MRI) [7] Contextual Multiple Linear Prediction (CMLP) [8] In order to quantify the reconstruction accuracy: 1. consider I(1) cloud-free 2. simulate a presence of clouds by obscuring partly I(2) 3. compare the reconstructed I(2) with its original one Evaluate the sensitivity to two aspects: Test 1: kind of ground cover obscured Test 2: size of the contaminated area 20 [7] L. Lorenzi, F. Melgani and G. Mercier, Inpainting Strategies for Reconstruction of Missing Data in VHR Images, IEEE Geosci. Remote Sens. Letters, vol. 8, no. 5, pp. 914-918, Sep. 2011. [8] F. Melgani, Contextual Reconstruction of Cloud-Contaminated Multitemporal Multispectral Images, IEEE Trans. Geosci. Remote Sens., vol. 44, no. 2, pp. 442455, Feb. 2006.
  18. 18. Multiresolution Inpainting (MRI) Based on the Region-based inpainting (RBI) from Criminisi et al [9] 21 [7] L. Lorenzi, F. Melgani and G. Mercier, Inpainting Strategies for Reconstruction of Missing Data in VHR Images, IEEE Geosci. Remote Sens. Letters, vol. 8, no. 5, pp. 914-918, Sep. 2011. [9] A. Criminisi, P. Perez and K. Toyama, Region filling and object removal by exemplar-based image inpainting, IEEE Trans. on Image Process., vol. 13, no. 9, pp. 1-14, Sep 2004. In [7], we proposed a processing scheme which recursively injects multiresolution information for a better reconstruction of the missing area.
  19. 19. Contextual Multiple Linear Prediction Contextual Multiple Linear Prediction (CMLP) [8] : Training: Prediction: 22 A simple solution is based on the minimum square error pseudoinverse technique. [8] F. Melgani, Contextual Reconstruction of Cloud-Contaminated Multitemporal Multispectral Images, IEEE Trans. Geosci. Remote Sens., vol. 44, no. 2, pp. 442455, Feb. 2006.
  20. 20. Test 1 Contamination of Different Ground Cover We suppose multiple land cover contamination, namely different areas of the image are missing Each mask is composed by ~2,000 pixels 23
  21. 21. Test 1 Contamination of Different Ground Cover Dictionary D is composed by pixels regularly sub- sampled from region: 316 for DS1 402 for DS2 24 D Sub-sampling
  22. 22. Test 1 Datasets 1 & 2 26 Mask A Mask B Mask C PSNR Complexity Time [s] PSNR Complexity Time [s] PSNR Complexity Time [s] MRI 22.54 - 2856 16.05 - 2517 33.77 - 2898 CMLP 20.99 1 1 20.11 1 1 24.05 1 1 OMP 23.96 3 4 20.60 3 4 31.97 3 4 BP 22.22 294 66 24.74 168 59 30.67 301 60 GA 23.78 148 68621 23.15 95 26312 32.01 138 43193 Mask A Mask B PSNR Complexity Time [s] PSNR Complexity Time [s] MRI 24.27 - 2995 29.54 - 3614 CMLP 24.61 1 1 27.69 1 1 OMP 26.36 3 5 30.43