Vision of UNICORE – UniGrids and beyond Daniel Mallmann [email protected] 2 nd CoreGRID...
-
Upload
vanessa-bruce -
Category
Documents
-
view
216 -
download
0
Transcript of Vision of UNICORE – UniGrids and beyond Daniel Mallmann [email protected] 2 nd CoreGRID...
![Page 1: Vision of UNICORE – UniGrids and beyond Daniel Mallmann d.mallmann@fz-juelich.de 2 nd CoreGRID Summer School Bonn, Germany 24 th – 28 th July 2006.](https://reader035.fdocuments.net/reader035/viewer/2022062718/56649e7c5503460f94b7ea9d/html5/thumbnails/1.jpg)
Vision of UNICORE – UniGrids and beyond
Daniel Mallmann [email protected]
2nd CoreGRID Summer SchoolBonn, Germany
24th – 28th July 2006
![Page 2: Vision of UNICORE – UniGrids and beyond Daniel Mallmann d.mallmann@fz-juelich.de 2 nd CoreGRID Summer School Bonn, Germany 24 th – 28 th July 2006.](https://reader035.fdocuments.net/reader035/viewer/2022062718/56649e7c5503460f94b7ea9d/html5/thumbnails/2.jpg)
2
Objectives
Develop Unicore/GS
Based on Unicore and the Open Grid Services Architecture
Compliant with the Web Services Resource Framework
Develop generic software components
Visualisation and steering of simulations
Device monitoring and control
Access to distributed data and databases
Validate UniGrids concepts and demonstrate the application of Unicore/GS in Industrial and scientific domains
![Page 3: Vision of UNICORE – UniGrids and beyond Daniel Mallmann d.mallmann@fz-juelich.de 2 nd CoreGRID Summer School Bonn, Germany 24 th – 28 th July 2006.](https://reader035.fdocuments.net/reader035/viewer/2022062718/56649e7c5503460f94b7ea9d/html5/thumbnails/3.jpg)
3
Duration, Funding
6th Framework Programme of the European Commission
2nd Call of the IST 2003-2004 Workprogramme♦ Objective: GRID-based systems and solving complex problems
Instrument: STReP
Duration: 24 months
Start: July 2004
Effort: 297 PM (12 developer + manager)
Costs: 3.0 M€
Grant: 1.9 M€
83
![Page 4: Vision of UNICORE – UniGrids and beyond Daniel Mallmann d.mallmann@fz-juelich.de 2 nd CoreGRID Summer School Bonn, Germany 24 th – 28 th July 2006.](https://reader035.fdocuments.net/reader035/viewer/2022062718/56649e7c5503460f94b7ea9d/html5/thumbnails/4.jpg)
4
European Context
inteliGRIDSemantic Grid based virtual organisations
ProvenanceTrust and provenance
for Grids
DataminingGridDatamining
tools & services
K-WF GridKnowledge based
workflow & collaboration
GRIDCOORDBuilding the ERA in Grid research
OntoGridKnowledge Services for the semantic Grid
HPC4UFault tolerance,dependability
for Grid
SIMDATGrid-based generic enabling application technologies to
facilitate solution of industrial problems
NextGRID EU-driven Grid services architecture for businesS
and industry
AkogrimoMobile Grid architecture
and services for dynamic virtual organisations
CoreGRIDEuropean-wide virtual laboratory for longer term Grid research-creating the
foundation for next generation Grids
Specific support action Integrated project Network of excellence Specific targeted research project
Courtesy: Max Lemke, European Commission
UniGridsExtended OGSA
Implementation based on UNICORE
UniGridsExtended OGSA
Implementation based on UNICORE
![Page 5: Vision of UNICORE – UniGrids and beyond Daniel Mallmann d.mallmann@fz-juelich.de 2 nd CoreGRID Summer School Bonn, Germany 24 th – 28 th July 2006.](https://reader035.fdocuments.net/reader035/viewer/2022062718/56649e7c5503460f94b7ea9d/html5/thumbnails/5.jpg)
5
European Context
SSAsSSAs supporting NESSI and the Grid communitysupporting NESSI and the Grid community
DataminingGrid
data, knowledge, data, knowledge, semanticssemantics
OntoGrid
InteliGridK-WF Grid
KnowArc Chemomentum
A-Ware Sorma
platforms, user platforms, user environmentsenvironments
Specific support action Integrated project Network of excellence Specific targeted research project
CoreGRIDvirtual laboratories
HPC4U
g-Eclipse
Gredia
GridComp
QosCosGrid
Grid4all
Provenance
AssessGridGridTrust
trust, securitytrust, security
Grid services, Grid services, business modelsbusiness models
ArguGrid Edutain @ Grid
GridEconGridCoord Grid@Asia
Nessi -GridChallengers Degree
NextGRIDservice
architecture
Akogrimomobile
services
BREINagents &
semantics
BeinGridbusiness
experiments
SIMDATindustrial
simulations
XtreemOS
ProvenanceProvenance
AssessGridAssessGridGridTrustGridTrust
trust, securitytrust, security
Grid services, Grid services, business modelsbusiness models
ArguGridArguGrid Edutain @ Grid
Edutain@ Grid
GridEconGridEconGridCoordGridCoord Grid@AsiaGrid@Asia
Nessi -GridNessi -GridChallengersChallengers DegreeDegree
NextGRIDservice
architecture
NextGRIDservice
architecture
Akogrimomobile
services
Akogrimomobile
services
BREINagents &
semantics
BREINagents &
semantics
BeinGridbusiness
experiments
SIMDATindustrial
simulations
SIMDATindustrial
simulations
XtreemOSLinux based
Gridoperating
system
BeinGridbusiness
experiments
BeinGridbusiness
experiments
Courtesy: Max Lemke, European Commission
UniGridsUniGrids
![Page 6: Vision of UNICORE – UniGrids and beyond Daniel Mallmann d.mallmann@fz-juelich.de 2 nd CoreGRID Summer School Bonn, Germany 24 th – 28 th July 2006.](https://reader035.fdocuments.net/reader035/viewer/2022062718/56649e7c5503460f94b7ea9d/html5/thumbnails/6.jpg)
6
Partner
Research Centre Jülich(Project manager)
Consorzio Interuniversitarioper il Calcolo Automatico dell’Italia Nord Orientale
Fujitsu Laboratories of Europe
University of Warsaw
Intel GmbH
University of Manchester
T-Systems SfR
![Page 7: Vision of UNICORE – UniGrids and beyond Daniel Mallmann d.mallmann@fz-juelich.de 2 nd CoreGRID Summer School Bonn, Germany 24 th – 28 th July 2006.](https://reader035.fdocuments.net/reader035/viewer/2022062718/56649e7c5503460f94b7ea9d/html5/thumbnails/7.jpg)
7
Technology Foundations
Develop Unicore Services
Job and file management
User authorisation
Dynamic resource discovery and management
Extend security to include explicit trusted delegation
NetworkJob
Supervisor
UnicoreGateway
Unicore Client
Target System A
Target System B
Target System C
ResourceDatabase
ResourceBroker
UserDatabase
![Page 8: Vision of UNICORE – UniGrids and beyond Daniel Mallmann d.mallmann@fz-juelich.de 2 nd CoreGRID Summer School Bonn, Germany 24 th – 28 th July 2006.](https://reader035.fdocuments.net/reader035/viewer/2022062718/56649e7c5503460f94b7ea9d/html5/thumbnails/8.jpg)
8
Hosting Environment
Build on Jetty Server and Axis SOAP engine
WS Resource Framework ♦ ResourceProperties♦ Lifetime♦ BasicProfile♦ ServiceGroup
WS Notification♦ NotificationProducer♦ NotificationConsumer♦ SubscriptionManager
WS Addressing♦ Address URL♦ Resource Disambiguator
Technology Foundations
![Page 9: Vision of UNICORE – UniGrids and beyond Daniel Mallmann d.mallmann@fz-juelich.de 2 nd CoreGRID Summer School Bonn, Germany 24 th – 28 th July 2006.](https://reader035.fdocuments.net/reader035/viewer/2022062718/56649e7c5503460f94b7ea9d/html5/thumbnails/9.jpg)
9
Technology Foundations
Web Service Unicore Gateway
HTTP and HTTPS connector
Support for other protocols
Dispatches incoming SOAP messages based on
♦ WS-Addressing headers or♦ URL information from the HTTP transport layer
Web Service Unicore User Database
Supports different storage formats – default XML file
Interface for secure, remote administration
Single UUDB can be used by multiple services
![Page 10: Vision of UNICORE – UniGrids and beyond Daniel Mallmann d.mallmann@fz-juelich.de 2 nd CoreGRID Summer School Bonn, Germany 24 th – 28 th July 2006.](https://reader035.fdocuments.net/reader035/viewer/2022062718/56649e7c5503460f94b7ea9d/html5/thumbnails/10.jpg)
10
Usite B
Vsite B2Vsite B1
Usite A
Vsite A1
Technology Foundations
Multisite Job
Gateway
InternetInternet
TargetSystem
Interface
NetworkJob
Supervisor
TargetSystem
Interface
TargetSystem
Interface
NetworkJob
Supervisor
NetworkJob
Supervisor
Client
Gateway
![Page 11: Vision of UNICORE – UniGrids and beyond Daniel Mallmann d.mallmann@fz-juelich.de 2 nd CoreGRID Summer School Bonn, Germany 24 th – 28 th July 2006.](https://reader035.fdocuments.net/reader035/viewer/2022062718/56649e7c5503460f94b7ea9d/html5/thumbnails/11.jpg)
11
Secondary Network Job Supervisor
Primary Network Job Supervisor
SSLSSL
Client
= User certificate = NJS certificate
Job
SubJob
Multisite Job
Consigner: entity that consigns a job or sub-job♦ Expressed by use in SSL connection
Endorser: entity that authorises the tasks to be performed♦ Expressed by signing of serialized AJO direct acyclic graph
Technology Foundations
![Page 12: Vision of UNICORE – UniGrids and beyond Daniel Mallmann d.mallmann@fz-juelich.de 2 nd CoreGRID Summer School Bonn, Germany 24 th – 28 th July 2006.](https://reader035.fdocuments.net/reader035/viewer/2022062718/56649e7c5503460f94b7ea9d/html5/thumbnails/12.jpg)
12
Usite A
Vsite A1
Explicit Trust Delegation
Gateway
InternetInternet
TargetSystem
Interface
NetworkJob
Supervisor
Client
Portal
Technology Foundations
![Page 13: Vision of UNICORE – UniGrids and beyond Daniel Mallmann d.mallmann@fz-juelich.de 2 nd CoreGRID Summer School Bonn, Germany 24 th – 28 th July 2006.](https://reader035.fdocuments.net/reader035/viewer/2022062718/56649e7c5503460f94b7ea9d/html5/thumbnails/13.jpg)
13
SSL
Network Job Supervisor
SSL
PortalWS- Client(Browser)
Job
User: name
= User certificate = Portal certificate
Explicit Trust Delegation
User: new role besides consignor and endorser♦ Entity (user) on whose behalf tasks will be performed
Trusted Agents (Portal)♦ Added to the UUDB explicitly♦ Allowed to endorse AJO on behalf of users
Technology Foundations
![Page 14: Vision of UNICORE – UniGrids and beyond Daniel Mallmann d.mallmann@fz-juelich.de 2 nd CoreGRID Summer School Bonn, Germany 24 th – 28 th July 2006.](https://reader035.fdocuments.net/reader035/viewer/2022062718/56649e7c5503460f94b7ea9d/html5/thumbnails/14.jpg)
14
Interoperability
UniGrids Atomic Services
Job Management
Storage Management
File Transfer
Target System
Unicore Globus CGSP
GridClient
GridPortal
TargetSystemService
JobManagement
Service
StorageManagement
Service
FileTransferService
Target System Factory
UniGrids Atomic Services
Standardisation
OGSA-BESBasic Execution Services
OGSA-DMIData Movement Interface
![Page 15: Vision of UNICORE – UniGrids and beyond Daniel Mallmann d.mallmann@fz-juelich.de 2 nd CoreGRID Summer School Bonn, Germany 24 th – 28 th July 2006.](https://reader035.fdocuments.net/reader035/viewer/2022062718/56649e7c5503460f94b7ea9d/html5/thumbnails/15.jpg)
15
Globus Toolkit 4 Hosting Environment
Globus Security
Target System Factory
TSF
Target SystemService
TSS
JobMgmt
ServiceJMS
StorageMgmt
ServiceSMS
FileTransferService
FTS
GridFTP/GASSGRAM/TSI
UniGrids Atomic Service Interfaces
UniGrids Atomic Services for GTK4
Available as Open Source (BSD license) at SourceForge (gpe4gtk.sourceforge.net)
Interoperability
![Page 16: Vision of UNICORE – UniGrids and beyond Daniel Mallmann d.mallmann@fz-juelich.de 2 nd CoreGRID Summer School Bonn, Germany 24 th – 28 th July 2006.](https://reader035.fdocuments.net/reader035/viewer/2022062718/56649e7c5503460f94b7ea9d/html5/thumbnails/16.jpg)
16
Security
Client to server♦ Security provider concept
allows to plug in security model on client side
GTK4 server to Unicore/GS server♦ Extract original user certificate
from proxy certificate chain,pass to UGS (reduced security model)
Unicore/GS server to GTK4 server♦ UGS could use proxy certificate from
ETD-enabled MyProxy server
Generic, standards-based VOMS preferable
GSIETD
GTK4Unicore/GS
Interoperability
![Page 17: Vision of UNICORE – UniGrids and beyond Daniel Mallmann d.mallmann@fz-juelich.de 2 nd CoreGRID Summer School Bonn, Germany 24 th – 28 th July 2006.](https://reader035.fdocuments.net/reader035/viewer/2022062718/56649e7c5503460f94b7ea9d/html5/thumbnails/17.jpg)
17
Expert ClientApplication
Client Portal Client
GB
GB GB
GBGB
GBGB
Clients and GridBeans
GPE Clients work on top of Unicore/GS, GTK4 or ChinaGrid
♦ Only requirement: UniGrids Atomic Services Interfaces
GridBeans are portable without modifications♦ Client API hides underlying infrastructure details
Interoperability
![Page 18: Vision of UNICORE – UniGrids and beyond Daniel Mallmann d.mallmann@fz-juelich.de 2 nd CoreGRID Summer School Bonn, Germany 24 th – 28 th July 2006.](https://reader035.fdocuments.net/reader035/viewer/2022062718/56649e7c5503460f94b7ea9d/html5/thumbnails/18.jpg)
18
Grid Ontology
Cooperation with OntoGrid
Encoded in OWL, now published on the UniGrids web site (http://www.unigrids.org/ontology.html)
Permits reasoning and thus can fill in gaps in manually entered descriptions
Interoperability
![Page 19: Vision of UNICORE – UniGrids and beyond Daniel Mallmann d.mallmann@fz-juelich.de 2 nd CoreGRID Summer School Bonn, Germany 24 th – 28 th July 2006.](https://reader035.fdocuments.net/reader035/viewer/2022062718/56649e7c5503460f94b7ea9d/html5/thumbnails/19.jpg)
19
Grid Economics
Economic Service Infrastructure
Full WSRF-Based Resource Broker♦ Use of extensible Economic terms for resource selection♦ Resource consumption feedback from Job Execution Engines
Resource Accounting System♦ Publication through
WS Resource Properties
Payment System♦ Based on MCS Payment
protocol♦ Uses broker as agent for
user and/or their VO
![Page 20: Vision of UNICORE – UniGrids and beyond Daniel Mallmann d.mallmann@fz-juelich.de 2 nd CoreGRID Summer School Bonn, Germany 24 th – 28 th July 2006.](https://reader035.fdocuments.net/reader035/viewer/2022062718/56649e7c5503460f94b7ea9d/html5/thumbnails/20.jpg)
20
Higher-level Services
Visualization and application steering
VISualization Interface Toolkit (VISIT)♦ Light-weight library that supports
bi-directional data exchange between visualizations and parallel applications
VISIT GridBean♦ Manage participants of visualization
sessions via VISIT / GS family
Framework works with all VISIT - enabled applications
Unique Unicore/GS feature for interactive scenarios
![Page 21: Vision of UNICORE – UniGrids and beyond Daniel Mallmann d.mallmann@fz-juelich.de 2 nd CoreGRID Summer School Bonn, Germany 24 th – 28 th July 2006.](https://reader035.fdocuments.net/reader035/viewer/2022062718/56649e7c5503460f94b7ea9d/html5/thumbnails/21.jpg)
21
Higher-level Services
Unicore/GS Streaming Framework (UGSF)
Components♦ Web service♦ Streaming server♦ Client library
Architecture♦ Streaming Service is a
stand-alone application that performs streaming
♦ Client uses dedicated protocol to connect to the streaming service and obtains real data stream
♦ Web service uses RMI to communicate with streaming service and setup streams
![Page 22: Vision of UNICORE – UniGrids and beyond Daniel Mallmann d.mallmann@fz-juelich.de 2 nd CoreGRID Summer School Bonn, Germany 24 th – 28 th July 2006.](https://reader035.fdocuments.net/reader035/viewer/2022062718/56649e7c5503460f94b7ea9d/html5/thumbnails/22.jpg)
22
Higher-level Services
File transfer service (including data filter)
Exploiting GridFTP
Client – Server transfers
Server – Server transfers♦ Including SRB
Client GridBean available
![Page 23: Vision of UNICORE – UniGrids and beyond Daniel Mallmann d.mallmann@fz-juelich.de 2 nd CoreGRID Summer School Bonn, Germany 24 th – 28 th July 2006.](https://reader035.fdocuments.net/reader035/viewer/2022062718/56649e7c5503460f94b7ea9d/html5/thumbnails/23.jpg)
23
Higher-level Services
Access to distributed data, access to databases
Interface to OGSA-DAI resources♦ WSRF Data services deployed in
Unicore/GS Container♦ OGSADAI GridBean♦ OGSADAI Client
![Page 24: Vision of UNICORE – UniGrids and beyond Daniel Mallmann d.mallmann@fz-juelich.de 2 nd CoreGRID Summer School Bonn, Germany 24 th – 28 th July 2006.](https://reader035.fdocuments.net/reader035/viewer/2022062718/56649e7c5503460f94b7ea9d/html5/thumbnails/24.jpg)
24
Applications
Deploy test beds
Prove UniGrids concepts inindustrial and scientific domains
Oil reservoir modelling
Bio-molecular, genomic and proteomic
Nuclear risk management
Re-engineerindustrial applications using Unicore/GS WSRF approach
![Page 25: Vision of UNICORE – UniGrids and beyond Daniel Mallmann d.mallmann@fz-juelich.de 2 nd CoreGRID Summer School Bonn, Germany 24 th – 28 th July 2006.](https://reader035.fdocuments.net/reader035/viewer/2022062718/56649e7c5503460f94b7ea9d/html5/thumbnails/25.jpg)
25
Standardisation
Influence standardisation process for selected Grid standards
JSDL
WS-RF
OGSA-BES
Promote European research results
Leverage partner participation in standards organisations
SAGA-RG
WS-N
OGSA-RSS
![Page 26: Vision of UNICORE – UniGrids and beyond Daniel Mallmann d.mallmann@fz-juelich.de 2 nd CoreGRID Summer School Bonn, Germany 24 th – 28 th July 2006.](https://reader035.fdocuments.net/reader035/viewer/2022062718/56649e7c5503460f94b7ea9d/html5/thumbnails/26.jpg)
26
Beyond UniGrids
Grid projects exploiting UniGrids results
D-Grid (www.d-grid.de)
DEISA (www.deisa.org)
OMII-Europe (omii-europe.com)
A-WARE (www.a-ware.org)
Chemomentum (www.chemomentum.org)
EGEE-II (www.eu-egee.org)
UNICORE 6 Release Schedule
Alpha release ≡ UniGrids results: July 2006
Beta release: mid 2007
Final release: end 2007
![Page 27: Vision of UNICORE – UniGrids and beyond Daniel Mallmann d.mallmann@fz-juelich.de 2 nd CoreGRID Summer School Bonn, Germany 24 th – 28 th July 2006.](https://reader035.fdocuments.net/reader035/viewer/2022062718/56649e7c5503460f94b7ea9d/html5/thumbnails/27.jpg)
27
A-WARE
Objectives
Simplify usage of Grid♦ Workflow management♦ User focused
workflow orchestration♦ Repository to store workflows
Fill an existing gap ♦ Between middleware and portals
![Page 28: Vision of UNICORE – UniGrids and beyond Daniel Mallmann d.mallmann@fz-juelich.de 2 nd CoreGRID Summer School Bonn, Germany 24 th – 28 th July 2006.](https://reader035.fdocuments.net/reader035/viewer/2022062718/56649e7c5503460f94b7ea9d/html5/thumbnails/28.jpg)
28
Chemomentum
Objectives
Provide an integrated Grid solution for workflow-centric, complex applications;focus on the end users
Provide Grid-enabled applications, data services and knowledge management solutions
♦ Risk assessment♦ Toxicity prediction♦ Drug design
Test-drive developed services in the context of the European regulatory initiative “Registration and Evaluation of Chemicals” (REACH)
♦ Aimed at reducing animal testing, by developing in silico, Grid-based tools
![Page 29: Vision of UNICORE – UniGrids and beyond Daniel Mallmann d.mallmann@fz-juelich.de 2 nd CoreGRID Summer School Bonn, Germany 24 th – 28 th July 2006.](https://reader035.fdocuments.net/reader035/viewer/2022062718/56649e7c5503460f94b7ea9d/html5/thumbnails/29.jpg)
29
OMII Europe
Objectives
Provide key software components for building e-infrastructures
Common interoperable services:♦ Database Access
♦ Virtual Organisation Management
♦ Portal
Infrastructure integration♦ Initial EGEE/UNICORE/Globus interoperability
♦ Interoperable security framework
♦ Accounting
♦ Job Submission
♦ Job Monitoring
![Page 30: Vision of UNICORE – UniGrids and beyond Daniel Mallmann d.mallmann@fz-juelich.de 2 nd CoreGRID Summer School Bonn, Germany 24 th – 28 th July 2006.](https://reader035.fdocuments.net/reader035/viewer/2022062718/56649e7c5503460f94b7ea9d/html5/thumbnails/30.jpg)
30
Summary
UniGrids built Unicore/GS
Based on Unicore and the Open Grid Services Architecture
Compliant with the Web Services Resource Framework
UniGrids achieved interoperability
UniGrids developed generic software components
UniGrids results are the base of UNICORE 6
UniGrids results are exploited by several Grid projects