Catalog on the fly satellite images

Luiz.motta@ibama.gov.brhttp://www.ibama.gov.br

Luiz Motta FOSS4G 2015 - Seul

MMA - Ministry of the Environment – Brazil

IBAMA - Brazilian Institute of Environment and Renewable Natural Resources

DIPRO - Directorship of Environmental Protection

CGMAM – General Coordination of Monitoring

COTIG – Coordination of Geospatial Technology *

Luiz Motta:● Academic education

● Forestry Engineer● Msc. Forestry Science (Optimization with GIS) – 1995

● Profissional experience (using GIS):EMBRAPA (Corn zoning), IEF-MG(monitoring of vegetation), …IBAMA (since 2003 – first tender for career)Amazon: SIPAM, Terra Legal Project, …FOSS4G: since 2010(contribution for QGIS).

Example of goal: Deforestation of Amazon

Example of goal: Illegal mining

● Satellite images used by DIPRO.

● Change in the use of satellite images.

● Demand of images for shares of DIPRO.

● Catalog on the fly● Server – Generation Geotif and TMS - Products ready● Client – QGIS

● Planet Labs Catalog – Explorer Program.

Presentation plan

● Free images● CBERS 2B (Brazil e China)

● 20m(CCD) e 2.7m(HRC)● Until 2010● Popularizing the use of images- INPE: Start 2004

“Experts”

● ALOS/Amazon Project (IBAMA, DPF and JICA)● ALOS/PALSAR: 100m● 2010 – 2011

● LANDSAT: Ortho rectified (temporal series + Landsat 8 ~ 4.5K)

● Purchased:● Rapideye: 3 covers of all Brazil (8,516,000 km2) ~57k

Satellite images used by DIPRO

Satellite images used by DIPRO

LANDSAT(2001- 2015)

LANDSATRAPIDEYE

Change in the use of satellite images

● High availability of higher spatial resolution● Effect “Google”, “Bing”,...● Reduced need for specialist.● Image classification x visual interpretation

● IBAMA:● Deforestation interpretation on each pass Landsat(16

days)● Detect changes based on temporal series temporal

series.● Support with images higher spatial resolution

Change in the use of satellite images

LANDSAT 8: 2015-08-28

1:10,500

BING

1:10,500

GOOGLE

1:10,500

1:50,000 1:10,500

2015-08-05

Change in the use of satellite images

RAPIDEYE 2014-06-19

Change in the use of satellite images

RAPIDEYE 2014-06-19 - R3G5B2

Landsat 8(2014-07-24) R6G5B4 Rapideye 5(2014-06-19) R3G5B2

Improvement for interpretation

Scale: 1:10,000

Demand of images for shares of DIPRO

Detect new deforestation fronts.

Temporal and spatial availabilityl:Example: Sentinel-2(10m) + Landsat

2015-05-10 2015-06-11

CR = Clear-cut

Catalog on the fly

● Objective: Obtain images automatically from a target

● Need – Server/local:● Organize images acquis (Landsat, RapidEye, ...)● End Product (RGB composition)

Geotiff and TMS● Footprint images (Catalog layer ) - Address of images

● Need – Client(QGIS):● Plugin for identify the images in the map extension● Recognize the type of source (local or server)● Automatically add images(Layer Group)● Order by date

● Plugin p/ cruzamento da grade com a extensão do mapa

Catalog on the fly

Use the original name + RGB

Processing:● Create RGB: 2_rgb.sh LC82270632013140LGN01.tif 6 5 4● Convert for 8bits:

16b_2_8b_convert.sh LC82270632013140LGN01_r6g5b4.tif● Change original image.● Scale: Minimum → 0 and Maximum → 255

Scripts: https://github.com/lmotta/scripts-for-gis

Catalog on the fly

Create TMS and GDAL_WMS files

mk_tiles.shLC82270632013140LGN01_r6g5b4.tif2 15/images/tms/landsat/imagens/png/landsathttp://10.1.25.66/imagens/tms/landsat

* gdal_tiler.py (tilers-tools 3.2.0)

Create new TAG: TargetWindow

Use for “Zoom to Layer”

Catalog on the flyBash example:nohup parallel mk_tiles.sh {} 2 17 ./png ./tms url < images.lst

Catalog on the flyCreative example: Not could use this PC, it is of private project. No problem, boot by portable HD (Xubuntu)

Catalog on the fly

Steps for footprint images (catalog layer)● Create footprint for each image● Smoothing for footprint● Add all fooprint for shapefile

Catalog on the flyCreate footprint – footprint.sh [image]

● gdal_calc.py:-A $in_img --A_band 1 --type Byte --calc "A>0" --outfile $zero_one_img* 16 bits

● gdal_sieve.py-q -st 100 -4 $zero_one_img -nomask $sieve_img

● gdal_edit.py:-a_nodata 0 $sieve_img

● gdal_polygonize.py:$sieve_img -q -b 1 -f "GeoJSON" $footprint_geojson

● Sed: add image and path namesssed="s|{ \"DN\": 1 }|{ \"path\": \"$dir_img\", \"image\": \"$basename_img\" }|"sed -i "$ssed" "$footprint_geojson"

Catalog on the fly

Smoothing for footprintfootprint_convexhull.py [geojson]● Create convex hull of geometry● Add suffix: _convexhull

Add all fooprint for shapefilefootprint_append_shp.sh [geojson] [shapefile]● Add feature from geojson in shapefile● ogr2ogr -update -append -t_srs EPSG:4674 $shapefile $footprint_geojson

Bash Example :

Catalog on the fly

Catalogs layers

● Created for all Rapideye images and put in database.● Landsat 8: Added directly in database● Fields:

● Path: network address(directory of Geotif)● Image: name of image(RGB)● TMS: Address of XML in HTTP server (GDAL_WMS)● Quicklook: Address of PNG in HTTP server● Date: create from name of image

● Plugin: Catalog on the fly [1]

● Search for catalog layer in current project● Fields: Date(not mandatory) and address of each image*Address: local image or GDAL_WMS for Internet

● Geometry: Footprint of image (polygon layer)

● Get images where intersect with extent map canvas. Use the features geometry of catalog layer for query.

Catalog on the fly

[1] https://plugins.qgis.org/plugins/catalogotf_plugin/

Catalog on the fly

It is not mandatory.The field is date type or Text (yyyy-mm-dd)

Local file or Internet by address for GDAL_WMS, prefix = http and suffix = .xml

● Check the layer, to search images where intersect with map canvas.

● Create a “group” for add new images (name layer - Catalog)

Reverse order by date or name image

Catalog on the fly

● Buttons for seach features in map canvas:

● Run for all.

● Run for selected.

● Cancel

● Waiting cancel

Catalog on the fly

● Create contex menu for each image in catalog group:● Zoom to● Highlight

WMS_GDAL need have the TAG <TargetWindow>, this TAG is not standard.TargetWindow: Extent of image.

Catalog on the fly

Catalog on the fly

Using with “Auxiliary windows” Plugin

Catalog on the fly

Quickly catalog from local images

56 files

Image Boundary Plugin (Extent)

Catalog on the fly

Quickly catalog from local imagesExpressions for virtual fields:● Source of file● Date from name image

Planet Labs Catalog

Presentation:http://pt.slideshare.net/LuizMotta3/planetlabs-explorer-qgisplugin

Planet Labs Catalog

Planet Labs Catalog

Planet Labs Catalog

Date: 2015-08-29Date: 2015-08-29

Acknowledgements

I thank all the people who share their knowledge openly.

Special thanks to the committees of OSGEO and FOSS4G 2015.

Luiz.Motta@ibama.gov.brIBAMA