Online GeoSpatial Processing (OLGP): An Experimentation With UMN-MapServer

Online GeoSpatial Processing (OLGP): An Experimentation With UMN-MapServer

Ranga Raju VatsavaiSRG, Department of Computer Science and

EngineeringRSL, Department of Forest Resources

University of Minnesota

Plan-B Presentation on March 26, 2003Committee:

Prof. Shashi ShekharProf. Jaideep Srivastava

Prof. Thomas E. Burk

Outline

Motivation

Contributions

Relate Work WebGIS Architectures

Our Approach Architecture GeoProcessing Design Issues

Validation (Case Study) NRAMS and kNN Applications

Conclusions

Motivation

Proper use and monitoring of environmental resources requires

Timely and accurate data on land use Regular availability

Remote Sensing and GIS Invaluable input for natural resource analysis and mapping

Problem? Lack of an efficient and easy-to-use delivery mechanism.

Internet Web has became popular as a vehicle for information

distribution and client/server applications. GISes are Standalone Web offers a convenient way to share complex multimedia data

Background

ForNet One of 18 Remote Sensing Database (RSD) programs

funded by NASA MapServer and ImageServer

MapServer Monolithic CGI program Map Creation, Simple feature query, Feature annotation,

feature classification, on-the-fly projection, etc. MapScript: Rapid prototyping of web applications with

server-side scripting languages. Beyond WebMapping (OLGP extension to MapServer)

GeoSpatial Analysis and Native DBMS support

Motivation Background

Contributions

Beyond Web Mapping Online GeoSpatial Processing (OLGP)

Efficient Implementation Load Balancing Client/Server Approach, Fine-

tuning Integration of disparate data sources

Remote Sensing, Sampling Data (FIADB) Extending Queries

Arbitrary region of interest (ROI) Integration with RDBMS Innovative Applications – From theory to

practice

MotivationContributions

WebGIS Architectures

WWW Intelligent mix of protocols –

Client/Server handshaking and HTTP HTML and XML

Popularity can be attributed to – User-friendly Web Clients (Netscape, IE, ..) Recent Advancements in development environments

WebGIS Main Components

The Client, The Server and the Network Limitations – No support for geographic elements Initial developments – Visualization (MapViewer from Xerox

PARC)

ContributionsRelated Work

Related Work - WebGIS Architectures

Initial Focus – Map Visualization (e.g. MapViewer –Xerox) First noticeable WebGIS – GRASSLinks (UC, Berkeley) Industry’s Initial response

CGI Wrappers to their standalone GIS This resulted in “thin-client/fat-server” systems Limitations

Server is overburdened with data access and analysis As the number of requests increases, server performance

decreases Benefits

Users does not need any additional resources Recent Advances in Internet Development Environments

Applets, ActiveX Controls and Extendible Web Clients (“plug-ins”)

Client-side GIS Resulted in “thick-client/thin-server” systems

ContributionsRelated Work

Architecture

#####

## ##

##### #

## #########

## #

######## #

## #####

## #### ###### ##

### ## ####### #

##

###

#

##

### ####

### #### #

###

## ## #

###### ####

### ###

##### ##

######

##

####

##

### ##

##### #### #

### ######## #

### #### ## # #

#### ### #

##### ######

####

#### ##

#### ##### ### #

### ## #####

#### #####

#### ### #

## ####

## ####### ### ##

##

#######

### ###

## ###

#####

######

#

####

#

###### ##

# ### ##

## ##

## ####

#

###

###

##### #####

## ###### #

##

#####

######

## ## #

##

#

# #

##

### ###

#

#

### ##### ### ##

##### #####

###

##

#

###

#

#

#### ###

###### ## #

#####

# ###

# ## ######

#

## #### #

####### #

####### ###

######

# # ## #

##

#####

#####

# ####

# ###

# #

## ###

####

#####

###

###

## ######

###

##

##

##

######### ##

## #### ##

#

######

##

#

# #

# # ##

##

#

##

#####

####

#

#

##

## #

## #

###

# ###

##

###

## #

# ### #

##

#

###

#

#

##

#

#

# #### #

##

###

#

####

#

##

##

## ##

## ###

##

#

# #

###

##

#

####

##

###

## ## #

#

###

###### ##

#####

#

##

##

#

###

## #

### ## #

#

#

## #

# ###

#

### #

# #

## #

# ##

# ### #

##

#

###

# ##

# ##

#

##

####

## ###

#

# ##

###

#

### #

#

###

#

##

## ##

# #

##

#

## #### ## #

# ####

#

##

#

# ## ##

# ####

#

##

#

## #

##

### ##

##

# ##

#

#

#

#

##

# ###

# #

#

#

#

#

##

#

#

#

#

#

#

#

##

#

# #

# # ####### ###

###### ## ####

# ##### #

#

###

##

# ##

### ####

## #####

##### ######

######

#

#

####### ## ###

### #######

## ### ##

#

### ## ##

## # ## ####### # #

### #

##

#

#

###

# #### # ### ###

### ### ## ##

### ##

## ###

### ##

#

###

# ### #

#### ### # #

# ###### ####

### ###### ### #

### ### ##

#

##

## ## ### ##

## # ## ## #### #

### ## ##

##### ## ## ## #

##

##

#

#

##

###

#

#

##

#

##

#

#

#

Internet

Geospatial Database Access Layer

RDBMS

Geospatial Analysis Layer

Server SideComponents

Client SideComponents

Pro

files

Geo

stat

istic

sH

isto

gram

s...

......

.

Com

mu

nica

tion

Laye

r

MapServer (Mapping, Query, ...)

CG

I

(WEBSAS)

Web Server

Web Clients

DatabaseSpatial

Related WorkOur Approach

Architecture

WEBSAS - Architecture “Balanced Client/Server” paradigm 3-tier Architecture Tier 1: Client Tier 2: Application Server

CGI Module (+MapServer) GeoSpatial Analysis System Communication System

Tier 3: GeoSpatial Database Access System Generic Image support (BIL/BSQ/BIP) Native RDBMS support (MySQL, Oracle etc).


GeoSpatial Analysis

Availability of multi-temporal AVHRR imagery made it possible –

Plant phenology Quantitatively describe NPP patterns in time and space Monitor and Map natural resources at regional and global scales.

NDVI and NDVI Profiles A temporal profile is a graphical plot of sequential NDVI

observations against time. These profiles quantify the remotely sensed vegetation’s

seasonality and dynamics ROI and Polygon based queries

Change Detection


GeoSpatial Analysis – Spatial Interpolation Queries

Given a set of sample plots (locations) and a set of corresponding attributes a set of spatial database layers (RS,..) user specified arbitrary region of interest (ROI)

Find Estimates for each location inside the ROI

i.e. Generalize queries over space Constraints

Non-numerical attributes, auto-correlation Objective

Minimize error




Given

Query Window

Find estimates at each cell

FIADBSURVEY(..,statecd,cycle,subcycle,…)

#12COUNTY(..,statecd,unitcd,countycd,..)

#6PLOT(..,statecd,cycle,unitcd,plot,..) #28SUBPLOT(..,statecd, ..,subp, ..) #13COND(..,statecd,..,conid,..)

#48TREE(..,statecd,..,tree,..) #61SEEDLING(..,statecd,..,spcd,..) #12


1) Extract FIA plot-id and coordinates from FIADBplot-id[], x[], y[] <-

SELECT p.plot, p.lat, p.lonFROM Plot pWHERE p.countycd = ‘137’ and …

2) Coordinate transform :: lattitude/longitude into UTM (meters)

ima_x[], img_y[] <- geo_to_utm(p.lat[], p.lon[])


Algorithm


3) For each plot, compute mean of 3x3 windowSignature[][] <- mean(p.plot, DN[][]).

4) For each pixel vector (scan each line, and each pixel in a line)

pixel[] <- Dni, where i = 1,2, .., #of channels


p.plot

Signature[][]

.. ..

203 [21,38,24,…]’

204 [28,31,22,…]’

.. ..Algorithm


5) Compute Euclidean-distance between pixel[] and each spectral signature

Distance[plotid[]] <- euc_dist(pixel[], signature[][])


Algorithm

Signature[][]

Pixel[]

0 255

255

Red

IR


6) Assign the closest FIA plot-id to the output pixel(x,y)

Opixel(x,y) <- min(dist[])7) Repeat (for all pixels)


Algorithm


Generic FIA Query Calculate the total number of all live white pine

trees on timberland in the state of Michigan

SELECT SUM(p.expcurr * t.tpacurr)FROM plot p, cond c, tree tWHERE p.statecd = 26 AND

(joint conditions ..) Limitations?

How about estimates for a census bloc?



Extract plot-id’s from plot-id image Generate plot-id histogram ( {<plot-id, frequency>, …}) Formulate Query (on-the-fly)

SELECT p.plot, p.expcurr, t.tpacurrFROM plot p, cond c, tree tWHERE p.statecd = 26 AND (join conditions ..)

ANDp.plot in (plot-id-list)

Results = SUM (frequency[p.plot] * p.expcurr * t.tpacurr)


Solution

plot Expcurr ..

.. ..

203 1.88

... …

308 2.11FCC ImagePlotid Image (Integration)

Design Choices – System Level Performance Issues

Communication – Amount of data to be transferred Increasing speed of internet connection Decreasing the amount of data to be transferred

Progressive Vector Transmission – M.J. Egenhofer et. al. Efficient Spatial Data Transmission in WEBGIS – Z.-K. Wei

Computation – GeoSpatial Analysis Functions Designing efficient algorithms Efficient data structures

Our Approach Load Balancing Fine Tuning Partial Materialization System Configuration - Ease of Use


Design Choices


ClientRequest

WebServer MapServer

Mapping

LB=>C or S ?

ConfigurationFg

Server

Client

Hard-coded

Pre-compute

Database

Presentation

Template Based

Static

Global Optimization

Global + Local

Load Balancing Client/Server

Where should fg be computed Choices – Server, Client, Pre-realization

Our Approach Based on amount of data to be transmitted over

network– “Output(fg) < Input(fg)”

Based on Response Time. fg On Server

If ((tf < tc) && ( dp di ) || (dp << di))

otherwise fg on clientIn Server case

Data to be transmitted = Output(fg) ( di)In Client case

Data to be transmitted = Input(fg)

Related WorkOur Approach Design Choices

Fg Level Fine Tuning

Pre-realization An important criteria we have adopted in the

development of geospatial database

CriteriaApply fg first and populate the geospatial database

If the operation is computationally intensive &&

Parameters are fixed (output is same) &&Size (output) Size(input)

Else apply fg on-the-fly in WEBSAS


Fg Level Fine Tuning

Multi-temporal Image Organization –Band Interleaved by Pixel (BIP).


Given images

BIP

BSQ

Partial Materialization

Partial-materialization Division of fg into sequence of

sub-tasks If possible pre-compute one of

the compute-intensive subtask Example – kNN in queries

involving interpolation


Storage Cost

No materialization

Partial materialization

Full materialization

Query

Response

Time

NM – 484 MB (St. Louis)

PM – 48 MB

FM – 96 MB/attribute

NM – >10 H

PM – (25W-15s, 365W-4m)

FM – (25W-1s, 365W-1m)

System Configuration

Application is configured using server-side configuration files

Map Object Label Object Layer Object Feature Object Web Object Process Object

Front-end Standard HTML elements Java-Script


System Configuration

Map Object MAP

NAME Application’s NameSTATUS On/OffIMAGECOLOR R G BUNITS Meters/FeetFONTSET Fonts file nameMARKERSET Filename

(Shade/Line)SIZE X Y

LayersScale…..

END

Layer Object Layer

NAME Layer NameGROUP NameDATA File NameSTATUS ON/OFFTYPE Annotation|Point|Line|

…MINSCALE NCLASSITEM ColumnCLASS …LABELITEM Column

END


Validation

Case Studies Natural Resource Analysis and Mapping System

(NRAMS) – A WebGIS application for Land Managers.

Efficient Interpolation Queries (kNN Application) AVHRR/MODIS Download Facility

Application is on Web Since March 1999. Usage Statistics

Public Domain Software Mailing Lists Over 600 users

Our ApproachValidation

NRAMS-Frontend

Analysis Numerical

Window Histogram Plot Spectral Plot Save Image

Window Size Mapping

Browse Map Query

Data Layers

NRAMS – Numerical Window and Histograms


NRAMS – Spectral Profiles


Conclusions

Visualization, Query and Beyond Online GeoSpatial Analysis (OLGP)

Various Design Choices To Improve Performance Load balancing Client/Server Architecture Fine tuning, Pre/Partial materialization Template Based System Configuration

Open Source, Documentation, Mailing Lists, Trusted User Base, Dynamic, Innovative applications

600 users, 10 developers, universities, public, private

Avg. 10,000 Data products/month Future Directions

Caching – Both server side and client side Persistent database connections Web Coverage Standard

ValidationConclusions

Acknowledgements

http://terrasip.gis.umn.edu/projects/NASA Funding

Prof. Shekhar, Prof. Tom Burk, Prof. Jaideep, Steve Lime (DNR), Perry, Jamie, Mark Hansen,SRG members, RSL members.

Online GeoSpatial Processing (OLGP): An Experimentation With UMN-MapServer

Documents

Transcript of Online GeoSpatial Processing (OLGP): An Experimentation With UMN-MapServer