NASA SensorWeb: From v2.0 to v3.0 Workflows and WCPS for ... · NASA SensorWeb: From v2.0 to v3.0...

NASA SensorWeb: From v2.0 to v3.0Workflows and WCPS for Space

ApplicationsApplications

Pat Cappelaere (Vightel)

D M dl (GSFC)

Open Geospatial Consortium TC Meeting

W kfl & C W ki G Dan Mandl (GSFC)

Stu Frye (SGT)Workflow & Coverages Working GroupsJune 2010, Silver Spring MD

1

https://ntrs.nasa.gov/search.jsp?R=20100028151 2020-04-21T21:56:32+00:00Z

AgendaAgenda

• Our Standard Compliance Evolution

• v2 0 Accomplishments & Lessons Learnedv2.0 Accomplishments & Lessons Learned

• v3.0 Architecture Migration– Towards A Unified Interface

– Examplesp

22

SatellitesSatellitesInt’lInt’l

UsersUsers

Int’lInt’lDisastersDisasters NASANASA

SensorWebSensorWebSensorWebSensorWebProjectProject

3

Big Software Project: Cost is Big Software Project: Cost is f(codef(code))

Cos

tC

ost

Lines of CodeLines of Code

4

Our Reality...Our Reality...

Fund

ing

Fund

ing

2005 ... 20102005 ... 2010

FF

5

But we have made some good progress...But we have made some good progress...

ties

ties

Cap

abili

tC

apab

ilit

2005 ... 20102005 ... 2010

6

Number of OGC Standards Has Increased SignificantlyNumber of OGC Standards Has Increased Significantly

ndar

dsnd

ards

OG

C S

tan

OG

C S

tan

which wouldwhich wouldincrease our cost!increase our cost!

2005 ... 20102005 ... 2010

# O

# O

7

S O OGC C li H Si ifi tl D dS O OGC C li H Si ifi tl D d

ance

ance

So Our OGC Compliance Has Significantly DecreasedSo Our OGC Compliance Has Significantly Decreased

C C

ompl

iaC

Com

plia

2005 ... 20102005 ... 2010

OG

CO

GC

8

v2.0 Architecture

9

NASA SensorWeb 2.0Orbit/STKOrbit/STKSPS

GetFeasibilitiesGetFeasibilities

SubmitTaskSubmitTask

EO-1

Task

EO 1ASTER

FORMOSATSPOT...

Sensor Planning Service10




EO-1

Task

You’ve Got Data EO 1ASTER

FORMOSATSPOT...

WNSWNSYou ve Got Data

Web Notification Service11




EO-1

Task

You’ve Got Data EO 1ASTER

FORMOSATSPOT...

OPSBOPSBYou ve Got Data

Publish/Subscribe Basic Service12




EO-1

Task

ASTERFORMOSAT

SPOT...OPSBOPSB

Raw Data (L1R, L1G)SOSSOS

DataData

Sensor Observation Service13




EO-1

Task

ASTERFORMOSAT

SPOT...OPSBOPSB

Raw Data (L1R, L1G)GeoTorrentGeoTorrent

DataData

GeoTorrent Service14




EO-1

Task

EO 1ASTER

FORMOSATSPOT...

OPSBOPSB

GeoTorrentGeoTorrentDataData

Derived Products CompositeCompositeCompositeCompositeDerived Products

WPS

pp

BurnscarBurnscar......

(JPL NG )

pp

WPS(JPL, NG...)

Web Processing Service15

NASA SensorWeb 2.0SPS



EO-1

Task

ASTERFORMOSAT

SPOT...OPSBOPSB


Execute Workflow

Data Atom Feed

WfCS

Create/Visualize Workflows

FlowsFlowsFlowsFlowsFlowsFlowsFlowsFlowsFlowsFlows

CompositeCompositeCompositeComposite


WPS

pp

BurnscarBurnscar......

(JPL NG )

pp

WPS (JPL, NG...)

Workflow Chaining Service16




EO-1

Task

ASTERFORMOSAT

SPOT...OPSBOPSB


Execute Workflow

Data Atom Feed FlowsFlowsFlowsFlowsFlowsFlowsFlowsFlowsFlowsFlowsCreate/Visualize Workflows

WfCS

CompositeCompositeBB

CompositeComposite


WPSpp

BurnscarBurnscar......pp

WCPSAccess Raster Data

WPSWPS WCSWCS

Execute Algorithm Access Raster Data

AlgosAlgosAlgosAlgos


AlgosAlgosAlgorithmsAlgorithmsCustom Products

Web Coverage Processing ServiceCreate/Visualize Algorithms

17




EO 1

Task

EO-1HyspIRITestbed

OPSBOPSB


Execute Workflow


WfCS


CompositeComposite


GCC GCC Compiler/Compiler/

LinkerLinker

Dynamic SwamoAgent Upload

WPSpp


WCPS

LinkerLinker

AMQP

WPSWPSParser /Parser /CodeCode

GeneratorGenerator

Execute AlgorithmAlgosAlgos



AlgorithmsAlgorithmsCustom Products ENVI/IDLENVI/IDL ENVI/IDLENVI/IDL

WCPS With a TwistCreate/Visualize Algorithms

ENVI/IDLENVI/IDL ENVI/IDLENVI/IDL

18

NASA SensorWeb 2.0

WCPS: A SQL-like Query Language

for c in ( NIR )return

encode((char) ( ((c.0 / ((float)c.0 + c.1))-(c.1 / ((float)c.0 + c.1)))> 0.6 ) * 255,

"png" )

WCPS

WPSWPS




AlgorithmsAlgorithmsCustom Products

WCPS LanguageCreate/Visualize Algorithms

19




Task

Visual EditorT t EditOPSBOPSB

GeoTorrentGeoTorrent

define name: flow1

Text Editor

Execute Workflow

Data Atom Feed FlowsFlowsFlowsFlows

FlowsFlowsFlowsFlows

FlowsFlowsCreate/Visualize Workflows

WfCSdefine name: flow1task_eo1process_alinotify_user


CompositeComposite

Create/Visualize Workflows y_end

WPSBurnscarBurnscar......

WCPSBPMN 1.0

WPSWPS





WfCS LanguageCreate/Visualize Algorithms

20

v2.0 Lessons Learned

Too Many Ways to Do Similar Things

Execute Task, Process, Workflow, Algorithm...

Hard Learning Curve on Users and ImplementorsHard Learning Curve on Users and Implementors

Two Languages: g g

For Workflows and Algorthms Developments

What About Web Service Security?

21

NASA SensorWeb v3.0SPS



EO 1

Task

EO-1HyspIRITestbed

OPSBOPSB


Execute Workflow


WfCS


CompositeComposite



LinkerLinker


WPSpp


WCPS

LinkerLinker

AMQP


GeneratorGenerator





v 3.0 Simplification #1Create/Visualize Algorithms


22




EO 1

Task

EO-1HyspIRITestbed

OPSBOPSB


Execute Workflow


WfCS


CompositeComposite



pp

WCPS

SEXPWPSWPS




AlgosAlgosCustom Products

SEXP

Create/Visualize Algos23

SEXP?SEXP?

24

S ES-Exp or S b li E iSymbolic Expressions

Think LISP or Parse TreesThink LISP or Parse Trees(= 4 (+ 2 2))

25




EO 1

Task GenericGenericOnboard SWAMO Onboard SWAMO

AgentAgent

EO-1HyspIRITestbed

OPSBOPSB


Execute Workflow

Data Atom Feed FlowsFlowsFlowsFlows

FlowsFlowsFlowsFlows

FlowsFlowsCreate/Visualize Workflows

WfCS


CompositeComposite



WCPS

WPSWPS





SEXP

Create/Visualize Algorithms26

EO 1

GenericGenericOnboard SWAMO Onboard SWAMO

AgentAgent

EO-1HyspIRITestbed

TextText VisualVisualCreate/Update

TextText VisualVisual

Domain

SEXP or JSONExecute

FlowsFlows

SpecifcLanguage (DSL) AlgorithmsAlgorithms

SEXP

RMagickRMagick

APIAPI

SecuritySecurity RubyRuby

GDALGDAL

gg

DataDataSecuritySecurity

2-factor Auth27

DSL E lDSL Example

define name:‘ali_l1g_visible’, author:‘Pat C.’, revision:1.0scene asset:EO1, instrument:‘ALI’, data:‘L1g’b5 = band(5)b4 = band(4)b3 = band(3)mask = b5 > 0composite_image red:b5, green:b4, blue:b3, alpha:mask

t t h t 2stretch percent:2compression quality:50 # 0 to 100, 100% being highest qualityencode type:‘jpg‘ #could do tif, png, kmz

endendend

end

28

DSL E lDSL Example

define name:‘ali_ndvi’, author:‘Pat C.’, revision:1.0scene asset:EO1, instrument:‘ALI’, data:‘L1g’b7 = band(7)b5 = band(5)mask = b7 > 0 and b5 > 0normalized_difference_ratio first:b7, second:b5

stretch percent:2l i ‘ i ’colorize map:‘prism’

compression quality:50 # 0 to 100, 100% being highest qualityencode type:‘kmz‘ #could do tif, png, jpg, kmz

endendend

end

29




EO 1

Task

EO-1HyspIRITestbed

OPSBOPSB


Execute Workflow


WfCS


CompositeComposite



LinkerLinker


WPSpp


WCPS

LinkerLinker

AMQP


GeneratorGenerator





v3.0 Simplification #1Create/Visualize Algorithms


30

NASA SensorWeb v3.0OPSBOPSB


Notifications

Data Atom Feed

EO 1Execute Task

Feature ServerFeature ServerProfileProfile

GetMetada

DataDataGetFeasibilitiesGetFeasibilities


EO-1HyspIRITestbed

Execute Workflow

TaskingTaskingProfileProfile

GetFeasibilities

Data Atom Feed FlowsFlowsFlowsFlowsFlowsFlowsFlowsFlowsFlowsFlows

WorkflowsWorkflowsProfileProfile

CompositeCompositeBurnscarBurnscar......

CompositeComposite

Execute ProcessProcessingProcessing

ProfileProfileCustom Products

AlgosAlgosAlgosAlgosAlgosAlgosAlgosAlgosAlgosAlgosAlgorithmsAlgorithmsCustom Products

Coverage Coverage ProcessingProcessing

ProfileProfile

Execute Algos

Dynamic Upload

Create/Visualize Flows and Algos

31

NASA SensorWeb v3.0OPSBOPSB


Notifications

Data Atom Feed

GetFeasibilitiesGetFeasibilities EO 1

DataDataExecute Task

GetMetada Feature ServerFeature ServerProfileProfile



EO-1HyspIRITestbed

Execute Workflow

GetFeasibilities TaskingTaskingProfileProfile

Data Atom Feed FlowsFlowsFlowsFlowsFlowsFlowsFlowsFlowsFlowsFlows

WorkflowsWorkflowsProfileProfile


CompositeComposite

Execute ProcessProcessingProcessing

ProfileProfileCustom Products

CoverageCoverage AlgosAlgosAlgosAlgosAlgosAlgosAlgosAlgosAlgosAlgosAlgorithmsAlgorithmsCustom Products

Coverage Coverage ProcessingProcessing

ProfileProfile

Execute Algorithms OtherOtherNodesNodes

32

Security / Security / OpenIDOpenID / / OAUthOAUth SecuritySecurity

Distributed Security Providers

2-Factor Auth

NASA SensorWeb v3.0

GetFeasibilitiesGetFeasibilities EO 1

DataDataResources



EO-1HyspIRITestbed

Service Document

FlowsFlowsFlowsFlowsFlowsFlowsFlowsFlowsFlowsFlowsGet, Post, Put, DeleteRESTFul /

SecureAPI


CompositeCompositeAtom Feeds

API

AlgosAlgosAlgosAlgosAlgosAlgosAlgosAlgosAlgosAlgosAlgorithmsAlgorithms

OtherOtherNodesNodes

33

Security / Security / OpenIDOpenID / / OAUthOAUth SecuritySecurity

Distributed Security Providers

2-Factor Auth

NASA SensorWeb: From v2.0 to v3.0 Workflows and WCPS for ... · NASA SensorWeb: From v2.0 to v3.0...

Documents

Transcript of NASA SensorWeb: From v2.0 to v3.0 Workflows and WCPS for ... · NASA SensorWeb: From v2.0 to v3.0...