GIAnT tutorial course  

Yosuke Aoki & Teguh Purnama Sidiq Earthquake Research Institute,

University of Tokyo

Email: yaoki@eri.u-tokyo.ac.jp teguh@eri.u-tokyo.ac.jp  

20-21 February 2014 ERI, Univ. Tokyo  

1

Wi-Fi  

ESS-ID: ERI_WLAN

WPA-PSK (TKIP): ERI_WLAN_2013_WPA_TKIP  

2

For rental PC users  

ID: pixel

Password: pixel00  3

GIAnT web sites  

GIAnT home page http://earthdef.caltech.edu/projects/giant/wiki!

GIAnT user manual http://earthdef.caltech.edu/attachments/download/15/GIAnTdoc.pdf!

(or click 3. User Manual (Most recent version) on the GIAnT home page

GIAnT sample data http://earthdef.caltech.edu/attachments/download/17/GIAnT_synth.tgz!

(or click 4. GIAnT test data on the GIAnT home page)

4

What GIAnT does  Generic InSAR Analysis Toolbox performs time-series analyses by various methods from given InSAR results either by ROI_PAC, DORIS, ISCE, GMTSAR, or GAMMA.

Python-based. No MATLAB or whatever commercial softwares required.

1. Small BASeline subset algorithm (SBAS) Berardino et al. (TGRS, 2002)

2. NSBAS Lopez-Quiroz et al. (J. Appl. Geophys, 2009) Doin et al. (FRINGE2011)

3. TimeFun Hetland et al. (JGR, 2012)

4. Multiscale InSAR Time Series (MInTS) Hetland et al. (JGR, 2012)

5

What SBAS does  

In a given pixel, The observation equation is

: Phase of the interferogram combining acquisitions i and j

: Phase increment between acquisitions n and n+1

Cumulative phase can be obtained by solving a linear least squares problem.

Solves only for pixels with complete dataset, i.e. all interferograms and acquisitions are available.

Φij

δϕn

Φij =j−1�

n=i

δϕn

6

NSBAS is similar to SBAS but it 1)  estimates DEM errors as well and 2) allows missing observations

The observation equation is

The deformation part takes a functional form with a default

The function needs to be specified in userfn.py.

What NSBAS does  

0 =k�

n=1

δϕn − f(∆tk) + eBkperp(k = 2, 3, · · · N)

f(t) = at2 + bt + c

Φij =j−1�

n=i

δϕn

DEM error Deformation

7

TimeFun is an extension of NSBAS to allow incorporating various functions, e.g., seasonal oscillations, polynominals, step functions, etc.

The observation equation for each pixel is

: k-th function

: Coefficient associated with the k-th function.

If regu in sbas.xml is true, the regularization is applied so that the temporal oscillation is suppressed (see p.66-67 of the User Manual for details).

What TimeFun does  

Φij =�

k

αk

�fk(ti)− fk(tj)

�+ eBij

perp

fk(t)

αk

8

What MInTS does  

MInTS is an extension of TimeFun in that:

The interferometric phase data is transformed into wavelet domain. It means that it considers spatial correlation rather than processing each pixel independently.

This processing takes much more time (>10 hours in my old Macintosh) other analyses.

9

Installing pre-requisites (1/9): Python  

Ubuntu: !>> apt-get install python2.7 python2.7-dev!

Mac OS: !>> port install python 27 !>> port select python27!(installation through MacPorts)  

10

Installing pre-requisites (2/9): NumPy  

A tool for array manipulation, FFT, linear algebra routines.

Ubuntu: !>> apt-get install python-numpy!

Mac OS: !>> port install py27-numpy!

The Manual reccomends >> port select py27-numpy +atlas +gcc45!but it does not work on my (old) Macintosh.

11

Installing pre-requisites (3/9): SciPy  

A tool for linear algebra, FFT, optimization, sparse matrices.

Ubuntu: !>> apt-get install python-scipy!

Mac OS: !>> port install py27-scipy!

The Manual reccomends >> port select py27-numpy +atlas +gcc45!but it does not work on my Macintosh.

12

Installing pre-requisites (4/9): Cython  

A tool to write C extensions for the Python language.

Ubuntu: !>> apt-get install cython!or >> easy_install cython!

Mac OS: !>> port install py27-cython or >> easy_install-2.7 cython!

13

Installing pre-requisites (5/9): Matplotlib  

2D plotting libraries.

Ubuntu: !>> apt-get install python-matplotlib!

Mac OS: !>> port install python-matplotlib

14

Installing pre-requisites (6/9): h5py  

NumPy interface to Hierarchial Data Format 5 (HDF5) memory mapped files.

Ubuntu: !>> apt-get install python-h5py!

Mac OS: !>> port install py27-h5py

15

Installing pre-requisites (7/9): pygrib  

The interface for reading GRIB-format weather data files in Python.

Ubuntu: !>> apt-get install zlib1g zlib1g-dev!>> apt-get install libpng12-0 libpng12-dev!>> apt-get install libjasper1 libjasper-dev!>> apt-get install libopenjpeg2 libopenjpeg-dev!>> apt-get install libgrib-api-1.9.9 libgrib-api-dev libgrib-api-tools!>> apt-get install python-mpltoolkits.basemap!>> apt-get install pyproj!>> east_install pygrib!

Mac OS: !>> port install py27-pygrib

16

Installing pre-requisites (8/9): pywavelets  

Wavelets for spatial decomposition of unwrapped interferograms in MInTS.

Ubuntu: !>> apt-get install python-pywt!

Mac OS: !>> port install py27-pywavelets!or >> easy_install-2.7 pywavelets

17

Installing pre-requisites (9/9) LXML  

A tool to read XML files.

Ubuntu: !>> apt-get install python-lxml!

Mac OS: !>> port install py27-lxml!

18

Installing optional packages (1/8): ffmpeg or mencoder  

A tool to make movies.

Ubuntu: !>> apt-get install ffmpeg mencoder!

Mac OS: !>> port install ffmpeg!

19

Installing optional packages (2/8): pyresample  

A tool for geocoding. Because pyproj is already installed,

Ubuntu: !>> easy_install pyresample!

Mac OS: !>> easy_install-2.7 pyresample!

should work. 20

Installing optional packages (3/8): HDFview  

A tool to explore HDF files.

Ubuntu: !>> apt-get install hdfview!

Mac OS (10.7 or later): !http://www.hdfgroup.org/products/java/release/download.html!

Mac OS (10.6)!http://www.hdfgroup.org/ftp/HDF5/releases/HDF-JAVA/hdf-java-2.9/bin/!

21

Installing optional packages (4/8): iPython  

Interactive Python.

Ubuntu: !>> apt-get install ipython!

Mac OS: !>> port install py27-ipython!

22

Installing optional packages (5/8): bpython  

An interface to the python interpreter. Needed only when iPython is not available.

Ubuntu: !>> apt-get install bpython!

Mac OS: !>> port install py27-bpython!

23

Installing optional packages (6/8): pykml  

A Python library for manipulating KML files.

Ubuntu: !>> easy_install pykml!

Mac OS: !>> easy_install-2.7 py27-bpython!

24

Installing optional packages (7/8) ImageMagick  

A tool to transform image formats.

Ubuntu: !>> apt-get install imagemagick!

Mac OS: !>> port install imagemagick!

25

Installing optional packages (8/8): xmlcopyeditor  

An editor for XML file. Available at (for example)

http://xml-copy-editor-sourceforge.net!

or search on the Internet.

Probably you do not necessarily need this.

26

Installing GIAnT (1/4): Downloading GIAnT  

Sign up at http://earthdef.caltech.edu/ (click register on the top right) if you have not done so, then

>> cd /Users/aoki/GIAnT-1.0 !>> svn co http://earthdef.caltech.edu/svn/giant!>> cd giant/GIAnT!>> svn co http://earthdef.caltech.edu/svn/pyaps!

If you are asked to provide an ID and password, use those registered when you signed up for the GIAnT website.

27

Installing GIAnT (2/4): Setting path to GIAnT  

On csh, >> setenv GIANT “/Users/aoki/GIAnT-1.0/giant/GIAnT:/Users/aoki/GIAnT-1.0/giant/GIAnT/SCR:/Users/aoki/GIAnT-1.0/giant/GIAnT/solver”!>> setenv PYTHONPATH “${GIANT}”!>> set path=($path $GIANT)!

On bash, >> export GIANT=/Users/aoki/GIAnT-1.0/giant/GIAnT:/Users/aoki/GIAnT-1.0/giant/GIAnT/SCR:/Users/aoki/GIAnT-1.0/giant/GIAnT/solver!>> export PYTHONPATH=$GIANT!:$PYTHONPATH!>> export PATH=$PATH:$GIANT 28

Installing GIAnT (3/4): Building GSVD (1/2)  

GSVD (Generalized Singular Value Decomposition) is used for L2 norm regularlized inversion in GIAnT.

>> cd /Users/aoki/GIAnT-1.0/giant/GIAnT!>> setenv CC gcc (on csh) or >> export CC=gcc (on bash) You need to specify the C compiler on Macintosh.

>> python setup.py config!Check setup.cfg to check if the Fortran compiler set here is the same as the one used to build SciPy and NumPy.

fcompiler = gnu95 (gfortran) !fcompiler = gnu (g77)

>> python setup.py build_ext!29

Installing GIAnT (4/4): Building GSVD (2/2)  

To test whether it is build properly, >> cd /Users/aoki/GIAnT-1.0/giant/GIAnT/solver/gsvd!>> python test.py !

If the build is done properly, you will get the output like: GERR = 1.33226762955e-15!HERR = 5.55111512313e-16!UERR = 6.66133814775e-16!VERR = 2.22044604925e-16!

If you get an error message, it is often due to the lack of lapack. Make sure that liblapack.a and libblas.a exists. !

30

Working with the sample data: (1/12) Extracting the dataset  

The sample dataset is at http://earthdef.caltech.edu/attachments/download/17/GIAnT_synth.tgz!or click 4. GIAnT test data on the GIAnT home page.

>> cd /Users/aoki/GIAnT-1.0!>> tar zxvf GIAnT_synth.tar.gz !

This creates a folder synthetic where we are going to work. !

31

Working with the sample data: (2/12) Pre-requisite files  

  Unwrapped interferograms in radar coordinates produced by ROI_PAC, DORIS, GMTSAR, GAMMA or ISCE (we use ROI_PAC outputs in this case). They are in /Users/aoki/GIAnT/synthetic/data/*unw !

  The corresponding coherence files. They are in /Users/aoki/GIAnT/synthetic/data/*cor

  A list of interferograms with Bperp and the sensor code name. /Users/aoki/GIAnT/synthetic/ifg.list!

  example.rsc (ROI_PAC), interferogram.out and master.res (DORIS), insarProc.xml (ISCE), image.PRM and unwrap.grd (GMTSAR), or date.mli.par (GAMMA) files.

See p.31-32 of the user manual for details. 32

Working with the sample data: (3/12) Creating XML files  

>> cd /Users/aoki/GIAnT-1.0/synthetic!

>> python prepxml.py!

This creates data.xml, sbas.xml, and mints.xml. !

data.xml: A XML file for the dataset (see p. 36-39 for details).

sbas.xml: A XML file for the SBAS, NSBAS, or TimeFun analysis (see p. 40-41 for details). !

mints.xml: A XML file for the MInTS analysis (see p. 41-43 for details). 33

Working with the sample data: (4/12) data.xml  

Digging into prepxml.py, you will find lines

g = ts.TSXML('data')!g.prepare_data_xml('example',xlim=[0,500], ylim=[0,500], rxlim=[30,50], rylim=[50,70], latfile='', lonfile='', hgtfile='', inc=21., cohth=0.1, chgendian='False', masktype='f4', unwfmt='FLT', corfmt='FLT')!

‘example’: Name of the rsc file from the ROI_PAC output

Generation of data.xml is mandatory for all analyses.

See p.36-38 of the User Manual for details.

34

Working with the sample data: (5/12) sbas.xml  

Digging into prepxml.py, you will find lines

g = ts.TSXML('params')!g.prepare_sbas_xml(nvalid=100, netramp=True, atmos='', demerr=False, uwcheck=False, regu=True, masterdate='19920604', filt=0.4)!

sbas.xml is used in SBAS, NSBAS, and TimeFun analyses.

sbas.xml controls network deramping, GPS deramping, atmospheric corrections, DEM error estimations, regularization of time functions, etc.

See p.39-40 of the User Manual for details.

35

Working with the sample data: (6/12) mints.xml  

Digging into prepxml.py, you will find lines

g = ts.TSXML('params')!g.prepare_mints_xml(netramp=True, atmos='', demerr=False, uwcheck=False, regu=True, masterdate='19920604')!

mints.xml is used for the MInTS analysis.

mints.xml controls network deramping, GPS deramping, atmospheric corrections, DEM error estimations, regularization of time functions, spatial smoothing, name of wavelets, etc.

See p.41-43 of the User Manual for details.

36

Working with the sample data: (7/12) Preparing the interferogram stack  

>> cd /Users/aoki/GIAnT-1.0/synthetic!>> PrepIgramStack.py !

Outputs are: !

  ./Stack/RAW-STACK.h5!with variables Jmat, bperp, cmask, dates, igram, tims, and usat!

  ./Figs/Igrams/*png!View interferograms by, for example, >> xv I001-031228-031019-ENV.png!

37

Working with the sample data: (8/12) Removing orbit and tropospheric effects  

This part removes the orbital ramps and tropospheric artifacts. See Chapter 5 (especially Figure 5.1) for details on what it does.

>> cd /Users/aoki/GIAnT-1.0/synthetic!>> ProcessStack.py !

Outputs are: !

  ./Stack/PROC-STACK.h5!with variables Jmat, bperp, cmask, dates, figram, ramps, and tims!

  ./Figs/Ramp/*png! 38

Working with the sample data: (9/12) SBAS analysis  

>> cd /Users/aoki/GIAnT-1.0/synthetic!>> SBASInvert.py!

Uncertainties are estimated by >> SBASxval.py !

To view the result >> plotts.py -f ./Stack/LS-PARAMS.h5!>> plotts.py –f ./Stack/LS-xval.h5 -e!

39

Working with the sample data: (10/12) NSBAS analysis  

>> cd /Users/aoki/GIAnT-1.0/synthetic!>> NSBASInvert.py!

Codes for estimating uncertainties are not ready yet.

To view the result >> plotts.py -f ./Stack/NSBAS-PARAMS.h5!

40

Working with the sample data: (11/12) TimeFun analysis  

>> cd /Users/aoki/GIAnT-1.0/synthetic!>> TimefnInvert.py!

Uncertainties are estimated by >> Timefnxval.py !

To view the result >> plotts.py -f ./Stack/TS-PARAMS.h5!>> plotts.py –f ./Stack/TS-xval.h5 -e!

41

Working with the sample data: (12/12) MInTS analysis  

It takes a lot of time (more than 10 hours by my old Macintosh) to complete the analysis.

>> cd /Users/aoki/GIAnT-1.0/synthetic!>> DatatoWavelet.py!>> InvertWaveletCoeffs.py!>> WavelettoData.py!

Uncertainties are supposed to be estimated by >> DatatoWavelet.py (You do not need to execute twice.) >> InvertWaveletCoeffs_xval.py!>> WavelettoData.py –i WAVELET-INV-xval.h5!but it does not work on my Macinotsh.

To view the result >> plotts.py -f ./Stack/WS-PARAMS.h5!

42

Visualization  

GIAnT provides several visualization tools. See Chapter 8 of the User Manual for details.

Interactive visualization >> plotts.py (-h) (-e) –f FNAME –i TIND –m MULT –y YINF YSUP –ms MSIZE –mask MASKFILE MASXML (-raw) (-model) (-zf)!

Making movies >> make_movie.py (-h) –i FNAME –o ONAME !

Generating KML files >> make_kml.py (-h) –i FNAME –x XNAME –o ONAME –nslice NSLICE –y YINF YSUP –win GWIN (-model) (-trans) !

43

Working with the real data (1/7): Downloading the Izu-Oshima dataset  

The dataset is at http://www.eri.u-tokyo.ac.jp/yaoki/Izu-Oshima_test.tar.gz!

Download to wherever you like and extract it. >> cd /Users/aoki/Izu-Oshima/GIAnT!>> tar zxvf Izu-Oshima_test.tar.gz!

This creates a folder Izu-Oshima_test.!

Interferograms are created with GAMMA.

Read /Users/aoki/Izu-Oshima/GIAnT/Izu-Oshima_test/bashScriptGamma/ManualStepGiantGamma.txt!for how to set up files and perform the time-series analyses. !

44

Working with the real data: Copying GAMMA files  

1.  Copy the GAMMA offset parameter file (DATE1_DATE2.off) and rename it to PROJECT.off (Izushima.off in this case).

2.  Copy GAMMA ISP MLI parameter file (DATE1.mli.par) and rename it to PROJECT.par (Izushima.par in this case).

3.  Copy DEM of radar coordinate (DATE1_DATE2.rdc_hgt) and rename it to PROJECT.rdc_hgt (Izushima.rdc_hgt in this case).

All of these above are already done in the dataset we provide.

45

Working with the real data: Creating ifg.list  

46

Make a text file with 4 columns, master date, slave date, Bperp in meters, and type of satellite like:

20060911 20070614 33.0356 ALOS!20060911 20070730 551.757 ALOS!20060911 20070914 661.701 ALOS!20070614 20070730 513.425 ALOS!20070614 20070914 623.66 ALOS !20070614 20071030 1162.21 ALOS!20070614 20090619 -520.82 ALOS!…..

Note: no blank space after each line or at the end of the file.

Working with the real data: Creating the rsc file  

47

Create PROJECT.rdc_hgt.rsc (Izushima.rdc_hgt.rsc in this case) by referring the GAMMA DATE1_DATE2.off like

WIDTH 1000!FILE_LENGTH 1166!LAT_REF1 34.6!LON_REF1 139.3!LAT_REF2 34.8!LON_REF2 139.5!LAT_REF3 34.6!LON_REF3 139.3!LAT_REF4 34.8!LON_REF4 139.5!AZIMUTH_PIXEL_SIZE 18.889452!RANGE_PIXEL_SIZE 9.344204!

Working with the real data: Creating userfn.py  

48

Create userfn.py. A good starting point is just to copy userfn.py from the sample dataset (/Users/aoki/GIAnT-1.0/synthetic/userfn.py).

Edit the following lines (already done in this dataset).

dirname = './data’!iname = '%s/%s_%s.diff_int.sm.unw'%(dirname,dates1,dates2)!cname = '%s/%s_%s.diff_int.cc'%(dirname,dates1,dates2)!

Working with the real data: Editing prepxml.py  

49

Copy and edit prepxml.py. See p.35-43 of the User Manual for parameterization.

Working with the real data: Ready to go!  

50

Generating XML files >> python prepxml.py!

Preparing the interferogram stacks >> PrepIgramStack.py !

Deramping and atmospheric correction >> ProcessStack.py !

SBAS analysis (see slide 39) >> SBASInvert.py !>> SBASxval.py!

NSBAS analysis (see slide 40) >> NSBASInvert.py!

TimeFun analysis (see slide 41) >> TimefnInvert.py !>> Timefnxval.py !

MInTS analysis (see slide 42) >> DatatoWavelet.py !>> InvertWaveletCoeffs.py !>> WavelettoData.py!>> InvertWaveletCoeffs_xval.py!>> WavelettoData.py –f WAVELET-INV-xval.h5!