Grid Programming Environment (GPE)
-
Upload
cameroon45 -
Category
Technology
-
view
656 -
download
0
Transcript of Grid Programming Environment (GPE)
Grid Programming Environment: Implementation & Globus Interface
Ralf RateringThomas KentemichAlexander LukichevIgor Odintsov
Grid Day in CGSLNovember 3, 2005
November 3 2005
- 2 -
*Other brands and names are the property of their respective owners © Copyright 2005 Intel Corporation. All Rights Reserved.
Agenda
Intel GPE Component Overview
Intel GPE Implementation Setting up a GPE Grid Globus Interface Current State and Next Steps
November 3 2005
- 3 -
*Other brands and names are the property of their respective owners © Copyright 2005 Intel Corporation. All Rights Reserved.
Intel GPE Component Overview
November 3 2005
- 4 -
*Other brands and names are the property of their respective owners © Copyright 2005 Intel Corporation. All Rights Reserved.
Application Programming Layer
GPE Component Overview
ApplicationsFSI, Energy, Mfg, Telco, Pharma, Retail, ...
Resource Virtualization Layer
Service OrientedArchitecture/Infrastructure
(SOA & SOI)
Grid Middleware
Compute Storage Network
Operating System
Compute Storage Network
Operating System
...
Man
ag
em
ent
Secu
rity
Application Programming LayerGrid Programming Library
Grid SDK Client FrameworkGridBeans
Globus Toolkit-based Open
Source implementation
(GPE4GTK)
Prototype
Examples and Templates
November 3 2005
- 5 -
*Other brands and names are the property of their respective owners © Copyright 2005 Intel Corporation. All Rights Reserved.
Grid Platform
Grid Application I/F (WSRF, JSDL, ...)
Mg
mt
Co
nso
le
Application Programming Layer
Grid Middleware
Grid Platform I/F(WSRF, WSDM,
JSDL, …)
Workflow Engine
Scheduler / BrokerTarget System
Registry
Database Access
H/W Platform I/F(CIM & WS–based)
Service OS
JobManagement
Service(JMS)
TargetSystemService(TSS)
StorageManagement
Service(SMS)
FileImportService
(FIS)
FileExportService(FES)
TargetSystemFactory(TSF)
MRTE
Virtual Machine
OS
Target System
Interface (TSI)
Virtual Machine
OS
Target System
Interface (TSI)create & manage
Application Deployment Service
Application Client Web portals
Grid Platform
...
...
Ad
min
Clie
nt
GPE Architecture (Fitting in Grid SW Architecture)
1
2
3
= GPE components under development
Hosting Environments and Server Implement. (UNICORE/GS, GTK4, …)
Expert Client
Firmware ( Platform S/W)
Platform H/W & Platform *Ts(Compute, Storage, Network)
Remote File Manager
GridBeans
= GPE components (not started)
Firmware ( Platform S/W)
Platform H/W & Platform *Ts(Compute, Storage, Network)
Applications
November 3 2005
- 6 -
*Other brands and names are the property of their respective owners © Copyright 2005 Intel Corporation. All Rights Reserved.
OGSA (Open Grid Services Architecture)– Share components with other architectures
WS* (WS-Addressing, WSRF, WSN, etc.)– Interoperation with other Grid Middleware
JSDL (Job Submission Description Language)
– High level job description that can be submitted to all target systems offering a JSDL interface
BPEL (Business Process Execution Language)
– Integration of Grid Bean services into larger business process workflows
CIM (Common Information Model)– Used to describe resources
– Usage of CIM management interfaces for Grid administration
Emerging Standards
November 3 2005
- 7 -
*Other brands and names are the property of their respective owners © Copyright 2005 Intel Corporation. All Rights Reserved.
Intel GPE Implementation
November 3 2005
- 8 -
*Other brands and names are the property of their respective owners © Copyright 2005 Intel Corporation. All Rights Reserved.
Expert Client
GPE Client Framework Full access to Grid for expert
users and administrators Workflow Editor Supports multiple applications
and user identities
Application Client
Light-weight Java application running on mobile devices
Simple application specific interface
Portal Client
JSR-168 compliant application portlets that can be integrated into existing portals like UPortal, Jetspeed or GridSphere
November 3 2005
- 9 -
*Other brands and names are the property of their respective owners © Copyright 2005 Intel Corporation. All Rights Reserved.
Expert Client
Application Client
Portal Client
Write GridBeans for your applications
GridBean Service
GridBean
GridBean
GridBean
GridBean
GridBean
GridBean
GridBean
GridBean SDK
GPE Client API, libraries and tools
GridBean
GridBean
GridBean
A GridBean dynamically integrates application-specific user interfaces in different clients
November 3 2005
- 10 -
*Other brands and names are the property of their respective owners © Copyright 2005 Intel Corporation. All Rights Reserved.
Expert Client
Application Client
Portal Client
GPE as interoperability framework
UNICORE/GS
Globus Toolkit 4
Other OGSA-compliant Grid
servers
China Grid Support Package
SOAP, WSDL,WSRF, WS-Addressing, WS-Security,JSDLand other specifications
Atomic Services
Atomic ServiceClient API
November 3 2005
- 11 -
*Other brands and names are the property of their respective owners © Copyright 2005 Intel Corporation. All Rights Reserved.
Atomic Services Overview
Target System Factory(TSF)
Atomic service interfaces define basic set of operations and properties that have to be available on a Grid system
– Operations describe the functionality
– Properties describe the state (WS-Resource Framework)
Different implementations of interfaces for different infrastructures
– Protocols, languages, security models, ...
Target SystemService(TSS)
JobManagement
Service(JMS)
StorageManagement
Service(SMS)
FileTransferService(FTS)
Managefiles on storage
Add a new target systemto the Grid
Manage transfers to/from storage
ManageJobs on target system
Managetarget system
November 3 2005
- 12 -
*Other brands and names are the property of their respective owners © Copyright 2005 Intel Corporation. All Rights Reserved.
JobManagement
Service(JMS)
StorageManagement
Service(SMS)
FileTransferService(FTS)
Operations Service Resource Type Properties
Atomic Service Interfaces based on WSRF
Extends WS-RP/LTstart()abort()hold()resume()
Extends WS-RP/LTJob-EPR submit(JSDL, initialTT)
Job3Job2
Status=RunningOriginalJSDL=<JSDL..>ExecutionJSDL=<JSDL..>etc.
Job1
create
Extends WS-RP/LTlistDirectory(path)listFileProperties(path)createDirectory(path)changePermissions(path, perm)copy(path, path)rename(path, path)delete(path)import(path, protocol, isPipe)export(path, protocol, isPipe)
HomeRoot
AvailableFileSpace=127kbSupportedProtocols= {GridFTP, UPL, scp}etc.
TS-Property
Temp
Work
create
Target System Factory(TSF)
Target SystemService(TSS)
Target System
Memory=1GbRunning Jobs=23CPU Type=ia64Storage=Temp, Rootetc.
create
TSR-EPR create(config)
Extends WS-RP/LT
FileTransfer 1
FileTransfer 2
Sourcefile=c:/tmp/test.txtDestFile=/tmp/test.txtTransferred=300kbetc.
create
November 3 2005
- 13 -
*Other brands and names are the property of their respective owners © Copyright 2005 Intel Corporation. All Rights Reserved.
Hiding platform-specific information with Application Resources
Abstract job concept borrowed from UNICORE– No concrete platform specific information in
job decription (paths, libraries, etc.)
– Job will be incarnated on target system
– For security and portability reasons!
Available applications can be queried from target system resource properties
Use JSDL Posix extensions to specify required application resources in submitted job
November 3 2005
- 14 -
*Other brands and names are the property of their respective owners © Copyright 2005 Intel Corporation. All Rights Reserved.
Transparency
Virtualization of the computational resources requires the job definitions to be abstract.
Non-abstract definition:
/usr/bin/povray /var/uspaces/ec82a0f0-4b7d-bf2585/povray.pov
Abstract definition:
<Job><Application name=“POVRay”/><Field name=“Input”>povray.pov</Field>
</Job>
November 3 2005
- 15 -
*Other brands and names are the property of their respective owners © Copyright 2005 Intel Corporation. All Rights Reserved.
Abstract Job Definitions
Advantages of abstract job definitions:
Easy to use with brokers Don’t need to disclose private
information about target system configuration
One step job submission
November 3 2005
- 16 -
*Other brands and names are the property of their respective owners © Copyright 2005 Intel Corporation. All Rights Reserved.
File Transfers Atomic Services support different
protocols–FTP, GridFTP, plain HTTP(S), SOAP with
Attachements, any other Storage management announces
available protocols via its resource properties–Client queries available protocols and
selects appropriate one–GridFTP for large high-performance
transfers–HTTPS or SOAP w/a to work with
firewall limitations
November 3 2005
- 17 -
*Other brands and names are the property of their respective owners © Copyright 2005 Intel Corporation. All Rights Reserved.
Client
Advanced Services: Workflow Execution
Workflow Execution
Service(TSS)
BPEL-enabled JSDL
Target System
Target System
CustomWeb-service
orchestrate
First implementation combining BPEL, JSDL and WSRF
Use Grid-specific BPEL subset– Allows integration into larger
business processes
Implemented as WSRF service itself– Information about workflow (state,
input/output files, etc.) is available through resource properties
November 3 2005
- 18 -
*Other brands and names are the property of their respective owners © Copyright 2005 Intel Corporation. All Rights Reserved.
Advanced Services: Target System Registry
Registry keeps track of static and dynamic information– Hardware properties, available
software, ...
– Workload, available disk space, ...
TSS contacts registry on startup and when properties change (WS-Notification)
Clients and services query informations about target systems from registry
Client Target System
Target System
Target System
Register/update
Query
Resource Broker
Target System Registry
November 3 2005
- 19 -
*Other brands and names are the property of their respective owners © Copyright 2005 Intel Corporation. All Rights Reserved.
GridBeans InternalsG
rid
Bea
n
Job DefinitionJob Definition
GPE Workflow Engine:
Atomic Target System:
• Contact arbitrary web-services
• Perform XML evaluations
• Incarnate user’s task
• Perform computations
November 3 2005
- 20 -
*Other brands and names are the property of their respective owners © Copyright 2005 Intel Corporation. All Rights Reserved.
GridBeans
POVRay
Visit
Compiler Gaussian
GridBean API
FinancialSimulationInterface
GPE Client API
GPE Client Framework
Web Portals
GPE Application Client
GPE Expert Client
Higher-level application services
EngineeringApplications
FinancialSimulations
etc.
Grid Programming API•Easy to implement•Portable•Easy to maintain
Advanced servicesAtomic Services
China Grid Support Package
Target SystemRegistry
UNICORE/GS
Globus ToolkitResource
Broker
Workflow
•Higher-level custom services take advantage of basic functionality provided by atomic and advanced services•Optional deployment of user interfaces through GridBeans
November 3 2005
- 21 -
*Other brands and names are the property of their respective owners © Copyright 2005 Intel Corporation. All Rights Reserved.
Setting up a GPE Grid
November 3 2005
- 22 -
*Other brands and names are the property of their respective owners © Copyright 2005 Intel Corporation. All Rights Reserved.
Adding a target system to a Grid
Target SystemStatic
Properties
Incarnation Database(IDB)
Target System
Interface (TSI)
Lightweight Java component to perform the actual work
Service container
Configuration file describing hardware and software properties
Configuration file defining systems specific paths to commands, libraries, etc.
1. Start Target System Interface
2. Define Static Properties and Incarnation Rules3. Start Service Container
Atomic services
November 3 2005
- 23 -
*Other brands and names are the property of their respective owners © Copyright 2005 Intel Corporation. All Rights Reserved.
Target System
TSI
Web Server
Service container
Target System
TSI
Target System
TSI
Atomic services
Site A
Target System
TSI
Service container
Atomic services
Target System
TSI
Service container
Atomic services
Site B
Web Server
Service container
Advanced services
Application services
Site C
A GPE Inter-Grid
Clients
Globus Interface
November 3 2005
- 25 -
*Other brands and names are the property of their respective owners © Copyright 2005 Intel Corporation. All Rights Reserved.
Why Globus?
De-facto standard, widely used Powerful, fast evolving Has enough features to host GPE services
November 3 2005
- 26 -
*Other brands and names are the property of their respective owners © Copyright 2005 Intel Corporation. All Rights Reserved.
Differences between Globus and UNICORE/GS hosting environments
Globus Toolkit 4 UNICORE/GS
Transport-level security
TLS v1 (SSL v3) SSL v2
Delegation Proxy certificates Extended Trust Delegation
WSRF specs Draft v1.0 Latest specs
de-facto standard
November 3 2005
- 27 -
*Other brands and names are the property of their respective owners © Copyright 2005 Intel Corporation. All Rights Reserved.
Portability
GPE client core
Globus client API implementation
Unicore/GS client API implementation
Globus atomic services interface
GPE service core
UNICORE/GS
November 3 2005
- 28 -
*Other brands and names are the property of their respective owners © Copyright 2005 Intel Corporation. All Rights Reserved.
Incarnations Database
povray +I<SOURCE> +O<TARGET> +F<OUTPUT_FORMAT> +W<WIDTH> …
fields
Fields are taken from the abstract job definition and are merged with the default values for user or application. The system is very flexibly configurable.
GPE uses incarnations database to store the templates of user’s jobs to incarnate.
November 3 2005
- 29 -
*Other brands and names are the property of their respective owners © Copyright 2005 Intel Corporation. All Rights Reserved.
Current State and Next Steps
November 3 2005
- 30 -
*Other brands and names are the property of their respective owners © Copyright 2005 Intel Corporation. All Rights Reserved.
GPE & GPE4GTK
GPE Alpha version has been released at UNICORE SourceForge project on June 1, 2005– Full working version for demos and developers
GPE-enhanced GTK4 implementation– SourceForge project „GPE4GTK“
– First release on October 28, 2005
– Beta release on December 31, 2005
UNICORE/GS support in European UniGridS project
Cooperation with China Grid to integrate GPE until end of 2006
Possible cooperation with SimDAT
November 3 2005
- 31 -
*Other brands and names are the property of their respective owners © Copyright 2005 Intel Corporation. All Rights Reserved.
Summary
Intel GPE......provides easy portable solution for
different Grid infrastructures including UNICORE/GS and GTK4
...provides a client framework to give users access to the infrastructure
...provides the GridBean concept and a programming API for Grid developers
...is available as Open Source today
Thank you !