Services for Science
description
Transcript of Services for Science
![Page 1: Services for Science](https://reader035.fdocuments.net/reader035/viewer/2022070406/5681420c550346895dae06b1/html5/thumbnails/1.jpg)
Ian Foster
Computation Institute
Argonne National Lab & University of Chicago
Services for Science
![Page 2: Services for Science](https://reader035.fdocuments.net/reader035/viewer/2022070406/5681420c550346895dae06b1/html5/thumbnails/2.jpg)
2
Thanks!
DOE Office of Science
NSF Office of Cyberinfrastructure
National Institutes of Health
Colleagues at Argonne, U.Chicago, USC/ISI, OSU, Manchester, and elsewhere
![Page 3: Services for Science](https://reader035.fdocuments.net/reader035/viewer/2022070406/5681420c550346895dae06b1/html5/thumbnails/3.jpg)
3
ScientificCommunication, ~1600
Brahe Kepler
![Page 4: Services for Science](https://reader035.fdocuments.net/reader035/viewer/2022070406/5681420c550346895dae06b1/html5/thumbnails/4.jpg)
4
![Page 5: Services for Science](https://reader035.fdocuments.net/reader035/viewer/2022070406/5681420c550346895dae06b1/html5/thumbnails/5.jpg)
5
Scientific Communication, ~2000
Data ArchivesData Archives
User
Analysis toolsAnalysis tools
Gateway
Figure: S. G. Djorgovski
Discovery toolsDiscovery tools
Service-Oriented Science
![Page 6: Services for Science](https://reader035.fdocuments.net/reader035/viewer/2022070406/5681420c550346895dae06b1/html5/thumbnails/6.jpg)
6
caBIG: sharing of infrastructure, applications, and data.
DataIntegration!
Services& Cancer Biology
Globus
![Page 7: Services for Science](https://reader035.fdocuments.net/reader035/viewer/2022070406/5681420c550346895dae06b1/html5/thumbnails/7.jpg)
7
RaffaeleMontella,U. NapoliParthenope
Services& Environmental Science
![Page 8: Services for Science](https://reader035.fdocuments.net/reader035/viewer/2022070406/5681420c550346895dae06b1/html5/thumbnails/8.jpg)
82
Network for Earthquake Engineering Simulation
Field Equipment
Laboratory Equipment
Remote Users
Remote Users: (K-12 Faculty and Students)
High-Performance Network(s)
Instrumented Structures and Sites
Leading Edge Computation
Curated Data Repository
Laboratory Equipment (Faculty and Students)
Global Connections(fully developed
FY 2005 –FY 2014)
(Faculty, Students, Practitioners)
![Page 9: Services for Science](https://reader035.fdocuments.net/reader035/viewer/2022070406/5681420c550346895dae06b1/html5/thumbnails/9.jpg)
9
Service-Oriented Science
People create services (data, code, instr.) …
which I discover (& decide whether to use) …
& compose to create a new function ...
& then publish as a new service.
I find “someone else” to host services, so I don’t have to become an expert in operating services & computers!
I hope that this “someone else” can manage security, reliability, scalability, …
!!“Service-Oriented Science”, Science, 2005
![Page 10: Services for Science](https://reader035.fdocuments.net/reader035/viewer/2022070406/5681420c550346895dae06b1/html5/thumbnails/10.jpg)
10
Creating Services
People create services (data, code, instr.) …
which I discover (& decide whether to use) …
& compose to create a new function ...
& then publish as a new service.
I find “someone else” to host services, so I don’t have to become an expert in operating services & computers!
I hope that this “someone else” can manage security, reliability, scalability, …
!!“Service-Oriented Science”, Science, 2005
![Page 11: Services for Science](https://reader035.fdocuments.net/reader035/viewer/2022070406/5681420c550346895dae06b1/html5/thumbnails/11.jpg)
11
Anatomy of a Service
op1 opN (meta)data
Implementation(s)
Clients RegistryManagement
Clients
…
Service
Service
AttributeAuthorityAttributeAuthority
Persistence
![Page 12: Services for Science](https://reader035.fdocuments.net/reader035/viewer/2022070406/5681420c550346895dae06b1/html5/thumbnails/12.jpg)
12
Exposing Service State
State Interfaces
Get resource property
Get multiple RPs
Set RP state
Query a set of RPs
Subscribe: notifications
Manage termination time
Immediate destruction
Resourceproperty
Resource
Service GetRP
GetMultRPs
SetRP
QueryRPs
Subscribe
SetTermTime
Destroy
EPREPR
EPR
State representation Resource, resource property
State identification Endpoint Reference
![Page 13: Services for Science](https://reader035.fdocuments.net/reader035/viewer/2022070406/5681420c550346895dae06b1/html5/thumbnails/13.jpg)
13
Apache Tomcat
Service Container
Globus ToolkitWeb Services Container
RPs
Resource
ServiceGetRP
GetMultRPs
SetRP
QueryRPs
Subscribe
SetTermTime
Destroy
EPREPR
EPR
ResourceHome
RPs
Resource
ServiceGetRP
GetMultRPs
SetRP
QueryRPs
Subscribe
SetTermTime
Destroy
EPREPR
EPR
ResourceHome
RPs
Resource
ServiceGetRP
GetMultRPs
SetRP
QueryRPs
Subscribe
SetTermTime
Destroy
EPREPR
EPR
ResourceHome
PIP
PDP
WorkManager DB Conn Pool JNDI Directory
Security
Persistence
Management
State
Authorization
Globus Toolkit Version 4: Software for Service-Oriented Systems, LNCS 3779, 2-13, 2005
Globus
![Page 14: Services for Science](https://reader035.fdocuments.net/reader035/viewer/2022070406/5681420c550346895dae06b1/html5/thumbnails/14.jpg)
14
Creating Services (~2005)
“This full-day tutorial provides an introduction to programming Java services with the latest version of the Globus Toolkit version 4 (GT4). The tutorial teaches how to build a Java Service that makes use of GT4 mechanisms for state management, security, registry and related topics.”
![Page 15: Services for Science](https://reader035.fdocuments.net/reader035/viewer/2022070406/5681420c550346895dae06b1/html5/thumbnails/15.jpg)
15
ApplnService
Create
Index service
StoreRepository ServiceAdvertize
Discover
Invoke;get results
Introduce
Container
Transfer GAR
Deploy
Argonne/U.Chicago & Ohio State University
Creating Services in 2008Introduce and gRAVI
Introduce Define service Create skeleton Discover types Add operations Configure security
Grid Remote Application Virtualization Infrastructure Wrap executables
Globus
![Page 16: Services for Science](https://reader035.fdocuments.net/reader035/viewer/2022070406/5681420c550346895dae06b1/html5/thumbnails/16.jpg)
Creating Services
Introduce + gRAVIShannon Hastings
Scott OsterDavid Ervin
Stephen Langella
Kyle ChardRavi Madduri
![Page 17: Services for Science](https://reader035.fdocuments.net/reader035/viewer/2022070406/5681420c550346895dae06b1/html5/thumbnails/17.jpg)
17
Discovering Services
People create services (data or functions) …
which I discover (& decide whether to use) …
& compose to create a new function ...
& then publish as a new service.
I find “someone else” to host services, so I don’t have to become an expert in operating services & computers!
I hope that this “someone else” can manage security, reliability, scalability, …
!!“Service-Oriented Science”, Science, 2005
![Page 18: Services for Science](https://reader035.fdocuments.net/reader035/viewer/2022070406/5681420c550346895dae06b1/html5/thumbnails/18.jpg)
18
The ultimate arbiter?
Types, ontologies
Can I use it?
Billions of services
Discovering Services
Assume success
Semantics
Permissions
Reputation
A B
![Page 19: Services for Science](https://reader035.fdocuments.net/reader035/viewer/2022070406/5681420c550346895dae06b1/html5/thumbnails/19.jpg)
19
Discovery (1):Registries
Globus
![Page 20: Services for Science](https://reader035.fdocuments.net/reader035/viewer/2022070406/5681420c550346895dae06b1/html5/thumbnails/20.jpg)
20
Discovery (2):Standardized Vocabularies
Core Services
Grid Service
Uses TerminologyDescribed In
Cancer DataStandards
Repository
EnterpriseVocabularyServices
ReferencesObjects
Defined in
Service Metadata
Publishes
Subscribes toand Aggregates
Queries Service
Metadata Aggregated In
Registers To
Discovery Client API
IndexService
Globus
![Page 21: Services for Science](https://reader035.fdocuments.net/reader035/viewer/2022070406/5681420c550346895dae06b1/html5/thumbnails/21.jpg)
21
![Page 22: Services for Science](https://reader035.fdocuments.net/reader035/viewer/2022070406/5681420c550346895dae06b1/html5/thumbnails/22.jpg)
22
Discovery (3): Tagging& Social Networking
GLOSS: Generalized
Labels Over Scientific data Sources
(Foster, Nestorov)
![Page 23: Services for Science](https://reader035.fdocuments.net/reader035/viewer/2022070406/5681420c550346895dae06b1/html5/thumbnails/23.jpg)
23
Discovery (3): Tagging& Social Networking
David de Roure, Carole Goble,
et al.
![Page 24: Services for Science](https://reader035.fdocuments.net/reader035/viewer/2022070406/5681420c550346895dae06b1/html5/thumbnails/24.jpg)
24
Composing Services
People create services (data or functions) …
which I discover (& decide whether to use) …
& compose to create a new function ...
& then publish as a new service.
I find “someone else” to host services, so I don’t have to become an expert in operating services & computers!
I hope that this “someone else” can manage security, reliability, scalability, …
!!“Service-Oriented Science”, Science, 2005
![Page 25: Services for Science](https://reader035.fdocuments.net/reader035/viewer/2022070406/5681420c550346895dae06b1/html5/thumbnails/25.jpg)
26
Composing Services
Globus
![Page 26: Services for Science](https://reader035.fdocuments.net/reader035/viewer/2022070406/5681420c550346895dae06b1/html5/thumbnails/26.jpg)
Composing Services
Taverna + GT4Taverna team
Wei TanRavi Madduri
![Page 27: Services for Science](https://reader035.fdocuments.net/reader035/viewer/2022070406/5681420c550346895dae06b1/html5/thumbnails/27.jpg)
28
Publishing Services
People create services (data or functions) …
which I discover (& decide whether to use) …
& compose to create a new function ...
& then publish as a new service.
I find “someone else” to host services, so I don’t have to become an expert in operating services & computers!
I hope that this “someone else” can manage security, reliability, scalability, …
!!“Service-Oriented Science”, Science, 2005
![Page 28: Services for Science](https://reader035.fdocuments.net/reader035/viewer/2022070406/5681420c550346895dae06b1/html5/thumbnails/28.jpg)
29
Publishing Services
Description Syntax, semantics
State Availability, load, …
Policies Who, what, when, …
Hosting Location, scalability, …
![Page 29: Services for Science](https://reader035.fdocuments.net/reader035/viewer/2022070406/5681420c550346895dae06b1/html5/thumbnails/29.jpg)
30
Defining Community: Membership and Laws
Identify VO participants and roles For people and services
Specify and control actions of members Empower members delegation Enforce restrictions federate policy
A
1 2
B
1 2
A B
1
10
1
10
1
16
Access granted by community
to user
Site admission-
control policies
EffectiveAccess
Policy of site to
community
![Page 30: Services for Science](https://reader035.fdocuments.net/reader035/viewer/2022070406/5681420c550346895dae06b1/html5/thumbnails/30.jpg)
31
Authorization: SAML & XACML
VOMS Shibboleth LDAP PERMIS…
GT4 ClientGT4 Server
PDP
AttributesAuthorization
Decision
PIP PIP PIP
SAML
XACML
Globus
![Page 31: Services for Science](https://reader035.fdocuments.net/reader035/viewer/2022070406/5681420c550346895dae06b1/html5/thumbnails/31.jpg)
32
Hosting Services
People create services (data or functions) …
which I discover (& decide whether to use) …
& compose to create a new function ...
& then publish as a new service.
I find “someone else” to host services, so I don’t have to become an expert in operating services & computers!
I hope that this “someone else” can manage security, reliability, scalability, …
!!“Service-Oriented Science”, Science, 2005
![Page 32: Services for Science](https://reader035.fdocuments.net/reader035/viewer/2022070406/5681420c550346895dae06b1/html5/thumbnails/32.jpg)
33
The Importance of “Hosting”and “Management”
Tell me aboutthis star Tell me about
these 20K stars
Support 1000sof users
E.g., Sloan DigitalSky Survey, ~10 TB;others much bigger
![Page 33: Services for Science](https://reader035.fdocuments.net/reader035/viewer/2022070406/5681420c550346895dae06b1/html5/thumbnails/33.jpg)
34
The Two Dimensions of Service-Oriented Science
Decompose across network
Clients integrate dynamically Select & compose services Select “best of breed” providers Publish result as new services
Decouple resource & service providers
Function
Resource
Data Archives
Analysis tools
Discovery toolsUsers
Fig: S. G. Djorgovski
![Page 34: Services for Science](https://reader035.fdocuments.net/reader035/viewer/2022070406/5681420c550346895dae06b1/html5/thumbnails/34.jpg)
35
Dynamic Provisioning:Falkon Applied to Stacking
Purpose On-demand “stacks” of
random locations within ~10TB dataset
Challenge Rapid access to 10-10K
“random” files Time-varying load
Solution Dynamic acquisition of
compute, storage
++++++
=
+
S4 SloanDataWeb page
or Web Service
Joint work with Ioan Raicu & Alex Szalay
Globus
![Page 35: Services for Science](https://reader035.fdocuments.net/reader035/viewer/2022070406/5681420c550346895dae06b1/html5/thumbnails/35.jpg)
Dynamic Provisioning with Falkon:Release after 15 Seconds Idle
![Page 36: Services for Science](https://reader035.fdocuments.net/reader035/viewer/2022070406/5681420c550346895dae06b1/html5/thumbnails/36.jpg)
Dynamic Provisioning with Falkon: Release after 180 Seconds Idle
![Page 37: Services for Science](https://reader035.fdocuments.net/reader035/viewer/2022070406/5681420c550346895dae06b1/html5/thumbnails/37.jpg)
Building Scalable Service Implementations
FunctionalMRI
Ben Clifford, Mihael Hatigan, Mike Wilde,Yong Zhao
Globus
![Page 38: Services for Science](https://reader035.fdocuments.net/reader035/viewer/2022070406/5681420c550346895dae06b1/html5/thumbnails/38.jpg)
AIRSN Program Definition
(Run snr) functional ( Run r, NormAnat a,
Air shrink ) {
Run yroRun = reorientRun( r , "y" );
Run roRun = reorientRun( yroRun , "x" );
Volume std = roRun[0];
Run rndr = random_select( roRun, 0.1 );
AirVector rndAirVec = align_linearRun( rndr, std, 12, 1000, 1000, "81 3 3" );
Run reslicedRndr = resliceRun( rndr, rndAirVec, "o", "k" );
Volume meanRand = softmean( reslicedRndr, "y", "null" );
Air mnQAAir = alignlinear( a.nHires, meanRand, 6, 1000, 4, "81 3 3" );
Warp boldNormWarp = combinewarp( shrink, a.aWarp, mnQAAir );
Run nr = reslice_warp_run( boldNormWarp, roRun );
Volume meanAll = strictmean( nr, "y", "null" )
Volume boldMask = binarize( meanAll, "y" );
snr = gsmoothRun( nr, boldMask, "6 6 6" );
}
(Run or) reorientRun (Run ir, string direction) { foreach Volume iv, i in ir.v { or.v[i] = reorient(iv, direction); } }
![Page 39: Services for Science](https://reader035.fdocuments.net/reader035/viewer/2022070406/5681420c550346895dae06b1/html5/thumbnails/39.jpg)
Virtual Node(s)
SwiftScript
Abstractcomputation
Virtual DataCatalog
SwiftScriptCompiler
Specification Execution
Virtual Node(s)
Provenancedata
ProvenancedataProvenance
collector
launcher
launcher
file1
file2
file3
AppF1
AppF2
Scheduling
Execution Engine(Karajan w/
Swift Runtime)
Swift runtimecallouts
C
C CC
Status reporting
Provisioning
FalkonResource
Provisioner
AmazonEC2
Dynamic Provisioning:Swift Architecture
Yong Zhao, Mihael Hatigan, Ioan Raicu, Mike Wilde, Ben Clifford, et al.
Globus
![Page 40: Services for Science](https://reader035.fdocuments.net/reader035/viewer/2022070406/5681420c550346895dae06b1/html5/thumbnails/40.jpg)
41
Services for Science They’re new
A new approach to communicating A (not-so new) approach to structuring systems
They’re real Excellent infrastructure and tools (Globus, Introduce,
gRAVI, Taverna, Swift, etc.) Substantial numbers of services out there
They’re challenging Sociology: incentives, rewards Infrastructure: hosting Provenance: justifying “results” Scaling: services, requests