ICEDS – using open source software to serve data and maps of the globe at full resolution Jeremy...

ICEDS – using open source software to serve data and maps of the globe at full resolution

Jeremy Morley, Jan-Peter Muller,Nuno Gil*, Cristiano Giovando**, Iain Willis^

Oliver Greening, Rob Scott^(ESYS plc)

*now with ESRI Portugal; **now at SDSU; ^ now at Qinetiq

Department ofGeomatic Engineering

http://iceds.ge.ucl.ac.uk

Some OGC Implementation Specifications used now and in the future in ICEDS (Integrated CEOS European Data Server)

• WMS – Web Map Service– Provides rendered images (pictures) of maps

• WCS – Web Coverage Service– Provides raster data (coverages) on demand– Restricted to locally held US-sourced data (SRTM, Landsat 7 & 5)

• WFS – Web Feature Service– Provides vector data (features) on demand

• GML – The Geography Markup Language– Used as an interoperable standard for transmitting geographic data


ICEDS Phases 1, 2 & 3 (2004-6)

• Prepared OGC-compliant:– Web Map Service (WMS)– Web Coverage Service (WCS)– Portal website (based on Ionic Red Spider GAF

interface)

• Provided detailed ‘Guidelines’ document on data preparation process

• Exploited open access (i.e. open source / freeware / free to download) software– MapServer: http://mapserver.gis.umn.edu/


ICEDS Phases 1, 2 & 3 (2004-6)

• Provided Web access to global datasets– SRTM & Landsat 5 and 7 mosaics– GTOPO30/SRTM30 to extend beyond area of SRTM– Web map services (transportation, built settlements)– MODIS WMS products where available

• Developed new visualisation capabilities– Use of transparency, flicker, colour LUTs to combine

data

• Tested interfaces to Google Earth• Encouraged other CEOS members to interoperate


Meteorological Information Resource for Africa (MIRA) extension (March 2006)

• Aim to tie together interoperable components from BNSC ICP2 projects with a focus on real-time.

• Serving meteorology-related data (surface wind field from GFS model; Meteosat-8 Near Real-Time cloud imagery)

• Chain of services: Comsine Ltd. hosting WMS• Implies new temporal selection capability in ICEDS client• Cannot offer free operational service due to the restrictive

conditions of access to EUMETSAT data which forbid Internet broadcasting of any images of derived Metoesat data for any purpose (including educational)


ICEDS system

• RAID Server:– 1.7TB capacity;

dual Athlon MP 2400+ processors; 1 Gb of RAM

• Mandrake Linux 9• Apache Web Server• University of Minnesota MapServer

– version 4.4.1 (current is 4.8.3)– (Mandrake 9 difficult to upgrade MapServer)


Datasets hosted on ICEDS• Landsat 5 mosaic

– Africa: bands 1,2,3 (WMS/WCS)– Europe: bands 1,2,3 (WMS/WCS) + band 4 (WCS)

• SRTM DEM– version 1 (unfilled)– colourised hill-shaded, gap map (WMS)– Elevations with gaps indicated (WCS)– SRTM30 (merge with GTOPO30, etc.) for globe

• SRTM water bodies mask– developed by NGA during SRTM DEM v2 production– held in original shapefile tiles, rendered on the fly

• DMSP Night-time Lights• MODIS Blue Marble

– version 1– principally for back-drop


Cascaded WMS layers

• CustomWeather:– Near-real time global IR cloud map (within 3 hours)– Weather forecast symbols

• NASA Aster DEM coverage (from USGS)• SIGAFRIQUE (from BRGM)

– African geological maps (1:10M scale)

• JPL OnEarth:– SRTM amplitude mosaic– Global Landsat 7 mosaic– Multiple styles for each layer


Data processing

• Data processing & loading fully documented in downloadable Guidelines document

• Revised methodologies cf. phases 1 & 2– more efficient processing schemes– expanded hill-shading colour table for Himalayas!

• Scripts used in preparation also downloadable


SRTM data preparationMasking

The background colour (red) shows voids in the SRTM data

Hill shading

A colour table has been defined reflecting atlas-like colour ranges

SRTM tiles intersecting coastline

DEM mosaicked in 4 regions (NW, NE, S, Azores)

Azores included in Europe and Africa – political versus geographic affiliations

Inland tiles do not need masking

Smart processing


Pyramid layers

• Key for access speed is pyramid layers• Landsat 5 mosaic:

– 1 arcsec/pixel from scale 1:1 to 1:1 M (14302 tiles)– 10 arcsec/pixel from scale 1:1 M to 1:8 M (1020 tiles)– 100 arcsec/pixel from scale 1:8 M to max (1 tile)

• Hill-shaded SRTM:– 3 arcsec/pixel from scale 1:1 to 1:1.7 M (14621 tiles)– 30 arcsec/pixel from scale 1:1.7 M to 1:20 M (434 tiles)– 180 arcsec/pixel from scale 1:20 M to max (8 tiles)


ICEDS portal• Purely OGC-based services• Hence separation of client and server(s)• Increasing range of ‘off-the-peg’ OGC clients, e.g.

– ka-Map! (http://ka-map.maptools.org/)– Chameleon

(http://www.dmsolutions.ca/technology/chameleon.html)

• ICEDS portal based on a client from Ionic– HTML + Javascript only– Good browser compatibility: PC & Mac IE,

Mozilla/Firefox/Seamonkey, Mac Safari & Camino– NO need for separate plug-in or standalone programme


ka-Map!


Interface features

• Download source data extracts from ICEDS WCS via icon on web interface

• Polygon annotation demonstration– only on beta site at present– WMS for displaying annotations

• Colour scale for SRTM hill-shading• Browser compatibility

– tests on IE & Firefox / Mozilla / Seamonkey on PC & Mac; Safari on Mac

– Javascript object models, PNG display are main issues


Interface features – layer controls

• Added capabilities for layer overlay:– partial layer opacity– ‘swipe’ function– ‘flicker’ function

• WMS layer style selection– e.g. OnEarth Landsat 7 band combinations


Interface features – temporal selection


Interface features – temporal selection

Thumbnails are live & updated to match zoom/ pan of map


Chained Met-8 service cascaded

EUMETSAT Met-8 feed

LogicaCMG:

Processing and orthorectification service

Comsine:

Databasing & WMS service

WMS

UCL:

ICEDS client


Near Real-time (every 15 minutes) Meteosat-8 SEVIRI data.


ICEDS portal - demonstration


Interoperability

• Examples:– SRTM hill-shading served into BRGM SIGAFRIQUE– New agreement with JAXA to ‘swap’ layers

• Demonstrations of other clients connecting to ICEDS WMS & WCS


WMS Connection from ArcGIS


WMS connection from osgPlanet


WCS + WMS Connection from Cadcorp SIS


Comsine: WCS demonstration - orthorectification wcs_url = 'http://iceds.ge.ucl.ac.uk/cgi-bin/wcs?map=wcs.map&SERVICE=WCS&REQUEST=GetCoverage&COVERAGE=srtm&FORMAT=GEOTIFFINT16' res = max(tif_pixsize_x, tif_pixsize_y) wcsRequest = wcs_url + "&CRS=EPSG:" + str(epsg) + "&BBOX=" + bbox + "&RESX=" + str(res) + "&RESY=" + str(res) logger.info("WCS REQUEST " + wcsRequest) if (dem == None): demFilename = str(time.clock()) + ".tif" urllib.urlretrieve(wcsRequest, filename) cleanup = 1


GeoConnections Canada


Google Earth – live connection Uses a PHP script at ESA to translate from WMS to KML for each refresh request


Site access stats. – IP addresses per day


Site access stats1st – 25th November 2005

• IP addresses:– accessing server: 471

– making a WMS request: 302– making a WMS request from outside ICEDS portal: 133

– making a WCS request: 62– making a WCS GetCoverage request: 23


Conclusions

• Open Source software solution (MapServer) appears to be very robust & effective

• Growing interest in ICEDS service– WCS as well as WMS

• A great deal can be achieved in creating a Web portal simply within the Web browser environment

• Plan to add X-SRTM WMS strips in the near future• Collaboration on OGC demonstration projects• ICEDS provides a good foundation for further OGC-related

developments– e.g. service into OGC GEOSS demonstrators

ICEDS – using open source software to serve data and maps of the globe at full resolution Jeremy...

Documents

Transcript of ICEDS – using open source software to serve data and maps of the globe at full resolution Jeremy...