Science Environment for Ecological Knowledge Bertram Ludäscher San Diego Supercomputer Center...

Science Environment Science Environment for Ecological for Ecological Knowledge Knowledge

Science Environment Science Environment for Ecological for Ecological Knowledge Knowledge

Bertram Ludäscher

San Diego Supercomputer CenterUniversity of California, San Diego

http://seek.ecoinformatics.org

UC Santa Barbara

UC San Diego

U New Mexico

U Kansas

Vermont, Napier, ASU, UNC

http://www.sdsc.edu/


SEEK Overview, 3/2004 2

Architecture Overview Architecture Overview

• Analysis & Modeling System– Design and execution of

ecological models and analysis

– End user focus– application-/upperware

• Semantic Mediation System– Data Integration of hard-

to-relate sources and processes

– Semantic Types and Ontologies

– upper middleware• EcoGrid

– Access to ecology data and tools

– middle-/underware

•Plus Working Groups:

– Knowledge Representation (SEEK-KR)

– Classification and Nomenclature (TAXON)

– Biodiversity and Ecological Analysis and Modeling (BEAM)

(cf. GEON + Cyberinfrastructure)



SEEK EcoGridSEEK EcoGrid

• Goal: standardize interfaces (using web and grid services)– We have standardized data via EML– Integrate diverse data networks from ecology, biodiversity, and

environmental sciences

• Grid-standardized interfaces– Uniform interface to:

• Metacat, SRB, DiGIR, Xanthoria, etc.• Anyone can implement these interfaces• Hides complexity of underlying systems

• Metadata-mediated data access– Supports multiple metadata standards– EML, Darwin Core as foci

• Computational services– Pre-defined analytical services– On-the-fly analytical services



Grid versus Web ServicesGrid versus Web Services

• Grid Services are Web Services– Add authentication, lifecycle management, notification, etc.– Globus Toolkit 3: Implements Open Grid Services Architecture

(OGSA)

• Implications for use– Write a normal web service extending GridService base class– When deployed within GT3, you get these extra functions for

‘free’– Supports distributed computation via proxy authentication

• Problems– Complex system to understand– GT3 can be difficult to deploy– Proposals to incorporate grid services within the Web services

community (Web Services Resource Framework [WSRF])



EcoGrid client interactionsEcoGrid client interactions

• Modes of interaction– Client-server– Fully distributed– Peer-to-peer

• EcoGrid Registry– Node discovery– Service discovery

• Aggregation services– Centralized access– Reliability– Data preservation



Building the EcoGridBuilding the EcoGrid

AND

LUQ

HBR

NTL

Metacat node

Legacy system

LTER Network (24) Natural History Collections (>> 100)Organization of Biological Field Stations (180)UC Natural Reserve System (36)Partnership for Interdisciplinary Studies of Coastal Oceans (4)Multi-agency Rocky Intertidal Network (60)

SRB node

DiGIR node

VCR

VegBank node

Xanthoria node



Kepler: Scientific WorkflowsKepler: Scientific Workflows

EML provides semi-automated data binding

Scientific workflows represent knowledge about the process; Kepler captures this knowledge

Query EcoGrid to find data

Archive output to EcoGrid



GARP Invasive Species ModelGARP Invasive Species Model

Training sample (d)

GARPrule set (e)

Test sample (d)

Integrated layers

(native range) (c)

DiGIRSpecies

presence &absence points(native range)

(a)

EcoGridQuery

EcoGridQuery

LayerIntegration

LayerIntegration

Sample

+A3+A2

+A1

DataCalculation

Map Validation

User

ValidationMap

SRBEnvironmental layers (invasion

area) (b)

Integrated layers

(invasion area) (c)

Invasionarea

prediction map (f)

DiGIR Species presence &absence points

(invasion area) (a)

Native range

predictionmap (f)

Model qualityparameter (g)

SRBEnvironmental layers (native

range) (b)

Model qualityparameter (g)

Slide from D. Pennington

Scientific workflows represent knowledge about the process; AMS captures this knowledge



Kepler Team, Projects, Sponsors Kepler Team, Projects, Sponsors

• Ilkay Altintas SDM • Chad Berkley SEEK • Shawn Bowers SEEK• Jeffrey Grethe BIRN• Christopher H. Brooks Ptolemy II • Zhengang Cheng SDM • Efrat Jaeger GEON • Matt Jones SEEK • Edward A. Lee Ptolemy II • Kai Lin GEON• Bertram Ludäscher BIRN, GEON, SDM, SEEK• Steve Mock NMI• Steve Neuendorffer Ptolemy II • Jing Tao SEEK• Mladen Vouk SDM • Yang Zhao Ptolemy II • …

Ptolemy IIPtolemy II



Kepler Understands EML Data Kepler Understands EML Data (Chad Berkley, SEEK)(Chad Berkley, SEEK)



Kepler: Ecological ModelingKepler: Ecological Modeling(Chad Berkley, SEEK)(Chad Berkley, SEEK)



Database Access Database Access (Efrat Jaeger, GEON)(Efrat Jaeger, GEON)

Note: EML descriptions of relational sources would allow automated data ingestion



Mineral Classification with Kepler … Mineral Classification with Kepler … (Efrat Jaeger, GEON)(Efrat Jaeger, GEON)



… … inside the Classifierinside the Classifier



Standard BrowserUI: Client-Side Standard BrowserUI: Client-Side SVGSVG



SWF Reengineering SWF Reengineering (Ilkay, SDM; Ashraf, Efrat, Kai, GEON)(Ilkay, SDM; Ashraf, Efrat, Kai, GEON)



DataMapper Sub-WorkflowDataMapper Sub-Workflow



Result launched via BrowserUI Result launched via BrowserUI actoractor

(coupling with ESRI’s ArcIMS)(coupling with ESRI’s ArcIMS)



Distributed Workflows in Distributed Workflows in KEPLERKEPLER

• Web and Grid Service plug-ins– WSDL (now) and Grid services (stay tuned …)– ProxyInit, GlobusGridJob, GridFTP, DataAccessWizard– SSH, SCP, SDSC SRB, OGS?-???… coming

• WS Harvester– Import query-defined WS operations as Kepler actors

• XSLT and XQuery Data Transformers– to link not “designed-to-fit” web services

• WS-deployment interface (planned)



Web Service Actor Web Service Actor (Ilkay Altintas, (Ilkay Altintas, SDM)SDM)

Given a WSDL and the name of an operation of a web service, dynamically customizes itself to implement and execute that method.

Configure - select service operation



Set Parameters and CommitSet Parameters and Commit

Set parameters and commit



SpecializedSpecialized WS Actor WS Actor (after instantiation)(after instantiation)



Web Service Web Service Harvester Harvester (Ilkay Altintas, SDM)(Ilkay Altintas, SDM)

• Imports the web services in a repository into the actor library.• Has the capability to search for web services based on a keyword.



Kepler: Grid Services AccessKepler: Grid Services Access(Steve Mock, NMI)(Steve Mock, NMI)



An (oversimplified) An (oversimplified) Model of the Model of the GridGrid

• Hosts: {h1, h2, h3, …}

• Data@Hosts: d1@{hi}, d2@{hj}, …

• Functions@Hosts: f1@{hi}, f2@{hj}, …

• Given: data/workflow:• … as a functional plan: […; Y := f(X); Z := g(Y); …] • … as a logic plan: […; f(X,Y)g(Y,Z); …]

• Find Host Assignment: di hi , fj hj for all di , fj

… s.t. […; d3@h3 := f@h2(d1@h1), …] is a valid plan

f gX Y Z



Shipping & Handling Algebra Shipping & Handling Algebra (SHA)(SHA)

f@a

x@b y@c

f@a

x@b y@c

f@a

x@b y@c

f@a

x@b y@c

plan Y@C = F@A of X@B =

1. [ X@B to A, Y@A := F@A(X@A), Y@A to C ]

2. [ F@A => B, Y@B := F@B(X@B), Y@B to C ]

3. [ X@B to C, F@A => C, Y@C := F@C(X@C) ]

Logical view

Physical view: SHA Plans

(1)

(3)

(2)



Grid-Enabling PTII: Grid-Enabling PTII: HandlesHandles

A B

GA GB

1. AGA: get_handle2. GAA: return &X3. AB: send &X4. BGB: request &X5. GBGA: request &X6. GA GB: send *X7. GBB: send done(&X)

Example: &X = “GA.17”

*X =<some_huge_file>

Candidate Formalisms:• GridFTP• SSH, SCP• SDSC SRB• OGS?-??? … WSRF?

1 2

3

4

5

6

7

Kepler space

Grid space

Logical token transfer (3) requires get_handle(1,2); then exec_handle(4,5,6,7) for completion.



Homogeneous Data IntegrationHomogeneous Data Integration

• Integration of homogeneous or mostly homogeneous data via EML metadata is relatively straightforward



Heterogeneous Data Heterogeneous Data integrationintegration

• Requires advanced metadata and processing

– Attributes must be semantically typed– Collection protocols must be known– Units and measurement scale must be known– Measurement relationships must be known

• e.g., that ArealDensity=Count/Area



• Label data with semantic types• Label inputs and outputs of analytical components with

semantic types

• Use reasoning engines to generate transformation steps– Beware analytical constraints

• Use reasoning engine to discover relevant components

Semantic MediationSemantic Mediation

Data Ontology Workflow Components



Ecological ontologiesEcological ontologies

• What was measured (e.g., biomass)• Type of measurement (e.g., Energy)• Context of measurement (e.g., Psychotria limonensis)• How it was measured (e.g., dry weight)

• SEEK intends to enable community-created ecological ontologies using OWL– Represents a controlled vocabulary for ecological metadata



ExtensionsExtensions: Semantic Types: Semantic Types

• Take concepts and relationships from an ontology to “semantically type” the data-in/out ports

• Application: e.g., design support: – smart/semi-automatic wiring, generation of “massaging

actors”

m1

(normalize)p3 p4

Takes Abundance Count

Measurements for Life StagesReturns Mortality Rate Derived

Measurements for Life Stages



Semantic TypesSemantic Types

• The semantic type signature– Type expressions over the (OWL) ontology

m1

(normalize)p3 p4

SemType m1 ::

Observation & itemMeasured.AbundanceCount &

hasContext.appliesTo.LifeStageProperty

->

DerivedObservation & itemMeasured.MortalityRate &

hasContext.appliesTo.LifeStageProperty



Extended Type System Extended Type System (here: OWL (here: OWL Semantic Types)Semantic Types)

SemType m1 :: Observation & itemMeasured.AbundanceCount & hasContext.appliesTo.LifeStageProperty DerivedObservation & itemMeasured.MortalityRate & hasContext.appliesTo.LifeStagePropertySubstructure association:

XML raw-data =(X)Query=> object model =link => OWL ontology



Semantic Types for Scientific Semantic Types for Scientific WorkflowsWorkflows



Deriving Data Transformations Deriving Data Transformations from Semantic Service from Semantic Service

RegistrationRegistration

[Bowers-Ludaescher,DILS’04]



Structural and Semantic MappingsStructural and Semantic Mappings

[Bowers-Ludaescher,DILS’04]



• Fundamental improvements for researchers

– Global access to ecologically relevant data– Rapidly locate and utilize distributed computation– Capture, reproduce, extend analysis process

SEEK ImpactSEEK Impact



AcknowledgementsAcknowledgements

This material is based upon work supported by:

The National Science Foundation under Grant Numbers 9980154, 9904777, 0131178, 9905838, 0129792, and 0225676.

PBI Collaborators: NCEAS, University of New Mexico (Long Term Ecological Research Network Office), San Diego Supercomputer Center, University of Kansas (Center for Biodiversity Research)

Kepler contributors: SEEK, Ptolemy II, SDM/SciDAC, GEON


Science Environment for Ecological Knowledge Bertram Ludäscher San Diego Supercomputer Center...

Documents

Transcript of Science Environment for Ecological Knowledge Bertram Ludäscher San Diego Supercomputer Center...