ICEDS – Creating a flexible geospatial information service

Jeremy Morley, Prof. Jan-Peter Muller, Nuno Gil*, Cristiano Giovando+

Dept. Geomatic Engineering, UCL

Oliver Greening, Rob ScottESYS plc

* - now with ESRI Portugal

+ - now at UCSD

Original Motivation

• CEOS Landsat and SRTM Project (CLASP)– NASA has provided SRTM (shuttle Radar Topographer Mapper) and

3 sets of orthorectified Landsat data to UN – The UN wish to maximise access to these data to all of its agencies– CEOS (Committee on Earth Observation Satellites) agencies offered

to assist the UN and NGOs in providing developing countries with internet access to global mosaics of Landsat and SRTM data through the provision of a number of integrated but distributed data servers. (part of the CEOS EO Data Portal)

– Exploitation of Open Geospatial Consortium interoperability standards– ‘Continental’ datasets could eventually be hosted by different CEOS

member space agencies using Landsat and/or SRTM data as a raster background

On-line GIS

• Fast developing area• New distributed processing technologies:

– Web services components (SOAP)– Grid processing

• Aided by standards– Data transfer formats– Data access methods– OpenGeospatial Consortium central + ISO TC/211

• ICEDS: Integrated CEOS European Data Server– Demonstrator sponsored by British National Space

Centre International Co-operation Programme (ICP-2)

ICEDS:Integrated CEOS European Data Server

http://iceds.ge.ucl.ac.uk

ICEDS Phase 1&2 Achievements

• Promoting Web access to large datasets– Europe (to Urals) and Africa– SRTM & Landsat mosaics

• Prepared OGC-compliant:– Web Map Service (WMS)– Web Coverage Service (WCS)– Portal website

• Provided detailed documentation on data preparation process

• Exploited open access (i.e. open source / freeware / free to download) software– MapServer: http://mapserver.gis.umn.edu/

Software• UCL GE Dept. software:

– ENVI 4.1 (SRTM and Landsat processing and editing)– ArcView 3 and ArcGIS 8.3 (coverages and vector data processing)

• Free software:– DLGV32 Pro NASA release (SRTM colourised hillshading)– MicroDEM

• Libraries and tools– GDAL libraries: GDALtileindex (shapefile indexing)– Global Mapper (full version of DLGV32 – low cost)– Free libraries for use with MapServer (including GDAL)

• Map servers tested– MapServer 4.4.1– IONIC RedSpiderWeb 3.1 (WMS setup and website design)

• Various IONIC tools (indexers, editors, pyramid-creator)– DeeGree v.1

• Deegree tools: runtileimage2

JPL Landsat 5 Africa mosaic

• 15 DVD-ROMs– 13 DVD-ROMs holding 30m mosaic– Other two holding subsampled versions, e.g. 300m

• Full-res. data stored as 5º x 5º tiles– 1 file per band; 3 bands (Landsat 1,2,3)– 324MB per file– GeoTIFF format– Not contrast stretched, beyond mosaicing

• Reduced res. data: all bands available, stretched.

JPL Landsat 5 Africa mosaic contrast enhancement & retiling

• Retiling operation was performed into– 5ºx5º contrast stretched composite tiles (WMS)– 0.5ºx0.5º tiles in separate bands (WCS)

• Performed with DLGV32 / Global Mapper using a text batch script file produced with a perl routine

• Pyramidal structure for WMSCurrent resolutions are:– 1 arcsec/pixel from scale 1:1 to 1:1.000.000 (14302 tiles)– 10 arcsec/pixel from scale 1:1.000.000 to 1:8.000.000 (1020 tiles)– 100 arcsec/pixel from scale 1:8.000.000 to max (1 tile)

• Each level has its own index file to allow the map-server application to find and get the right files

JPL Landsat 5 Africa mosaic –retiling & pyramidal structure

SRTM retiling & hill shading

• For portrayal in WMS, preferred portrayal for SRTM was colourised & hill shaded.– DLGV32 or Global Mapper preferred software– Handles SRTM HGT; can take user-defined colour table

• Unedited SRTM data contains elevations over ocean – these masked out in hill shaded images

• An additional issue: raw HGT files from EDC contain additional overlap pixels

• Overlap pixels need to be removed for processing in Global Mapper / DLGV32

SRTM data preparationMasking

The background colour (red) shows voids in the SRTM data

Hill shading

A colour table has been defined reflecting atlas-like colour ranges

SRTM tiles intersecting coastline

DEM mosaicked in 4 regions (NW, NE, S, Azores)

Azores included in Europe and Africa – political versus geographic affiliations

Inland tiles do not need masking

Smart processing

SRTM retiling & hill shading

SRTM pyramidal structure

• Current resolutions are:– 0.000834º/pixel from scale 1:1 to 1:3.000.000

(1-16 tiles in view)– 0.00834º/pixel from scale 1:3.000.000 to max

(1-18 tiles in view)

• Both levels have their index file to drive the map-server application to get the right files

ICEDS Phase 3 goals (by12/05)

• To extend Web access to global datasets– SRTM & Landsat mosaics– GTOPO30/SRTM30 to extend beyond area of SRTM– Web map services (transportation, built settlements)– Internal WMS/WCS as well as attach new layers on the fly– MODIS WMS products where available

• To develop an on-line map annotation and output– To allow users to add simple on-line annotations (e.g. text)– To allow users to take a snapshot of their map and output this as a

high-quality PDF

• To develop new visualisation capabilities– Use of transparency, flicker, colour LUTs to combine data

• To promote use in Africa and encourage other space agencies to set up their own CLASP servers

Additional datasets

• SRTM3 DEM coverage extended to full coverage• SRTM30 DEM coverage of world• JPL Landsat 5 mosaic extended to Europe

– currently a missing column of tiles as supplied!– band 4 available in addition to 1-3 (WCS only)

• DMSP night-time lights layer (radiance)• Additional cascaded layers, e.g.

– near-real time IR cloud map (CustomWeather)– NASA Aster DEM coverage (from USGS)– SIGAFRIQUE (from BRGM)

ICEDS map client

• Purely OGC-based services• Hence separation of client and server(s)• Increasing range of ‘off-the-peg’ OGC clients, e.g.

– ka-Map! (http://ka-map.maptools.org/)– Chameleon

(http://www.dmsolutions.ca/technology/chameleon.html)

• ICEDS client based on Ionic’s GAF client– Javascript only– Good browser compatibility: PC & Mac IE,

Mozilla/Firefox, Mac Safari

ka-Map!

Interface upgrades

• Download source data from ICEDS WCS via icon on web interface

• Colour scale for SRTM hill-shading– updated for global height range

• Solved PNG transparency problems– compatibility improved: tests on IE & Mozilla/Firefox on

PC & Mac; Safari on Mac

• Testing a sketch annotation method (part of a separate project)

• Additional layer list controls

Overall functionalities

• Connection to real-time and near-real time services

• Drill-down to anywhere on the planet to scales of 1:25 000

• Explore change (e.g. Landsat 5 to 7) and context (e.g. rivers, transportation networks)

• Explore geographical relationships at the continental and global scale (e.g. sea-level rise impact of global population)

Current portal

Drill down to anywhere

and overlay digital map data on SRTM

At any scale with Landsat5

Current portal - Night-time lights

Displaying natural hazard locations

Displaying global NRT clouds

Displaying geological Maps

And download data

And download data…

GeoTIFF file,current zoom.

Limited area.

Source data (e.g. DEM, not hillshade)

Latest interface upgrades

• Partial layer opacity• Flicker function

Partial layer opacity

SIGAFRIQUE geology over hill-shaded SRTM DEM

Partial layer opacity

NASA ASTER DEM footprints on hill-shaded SRTM DEM (red = SRTM gaps)

Layer flicker tool

Launch movie

Connecting to the service

• Simple client based on the client/server architecture developed with IDL– Code is easily portable via an IDL virtual machine which is freely

available

• Allows on-line connection to the

ICEDS WCS server– Signed 16-bit data retrieval– Display 3D perspective views of

SRTM DEM– Vertical exaggeration control

parameter– Possibility to choose desired

area by BBox

WCS client

WMS Connection from ArcGIS

WMS connection from Google Earth

WMS connection from Google Earth

WMS connection from osgPlanet

WMS connection from osgPlanet

WCS Connection from Cadcorp SIS

Comsine: WCS demonstration - orthorectification

wcs_url = 'http://iceds.ge.ucl.ac.uk/cgi-bin/wcs?map=wcs.map&SERVICE=WCS&REQUEST=GetCoverage&COVERAGE=srtm&FORMAT=GEOTIFFINT16' res = max(tif_pixsize_x, tif_pixsize_y) wcsRequest = wcs_url + "&CRS=EPSG:" + str(epsg) + "&BBOX=" + bbox + "&RESX=" + str(res) + "&RESY=" + str(res) logger.info("WCS REQUEST " + wcsRequest) if (dem == None): demFilename = str(time.clock()) + ".tif" urllib.urlretrieve(wcsRequest, filename) cleanup = 1

GeoConnections Canada

Conclusions

• Implementations of continental-scale data serving using OGC technologies

• Providing data & portrayal (WCS/WMS) services• User-selectable data freely and “instantly”

available at full resolution• Guidelines for setting up your own server available• Scripts/software available• Capability to add any cascaded WMS layers, such

as EA-lidar and orthorectified IKONOS/Quickbird

Future Work

• Final debugging of flicker function for release• Addition of CIESIN GRUMP global population data

• Cartographic output to PDF• Annotation functionality• Modularisation for interface internationalization

• Integration with JAXA / RESTEC• Feedback from users!

JISC Interoperability Project

• ICEDS client / Ionic GAF provides a very flexible framework

• Within JISC Interop. Project, Digimap OS data with seamless scale-dependent datasets

• Also, feature digitisation• Digitisation will send features to Ian Turton’s XIMA

server– if time, retrieval of previous features to client

Interoperability client (i)

Interoperability client (ii)

Interoperability client (iii)

Interoperability client (iv)

To be submitted to Ian Turton’s XIMA server…

Interoperability client (v)

ICEDS:Integrated CEOS European Data Server

http://iceds.ge.ucl.ac.uk

Jeremy Morley

jmorley@ge.ucl.ac.uk