GridLab: A Grid Application Toolkit and Testbed Jarosław Nabrzyski GridLab Project Manager Poznań...
-
Upload
aubrie-wilkerson -
Category
Documents
-
view
222 -
download
0
Transcript of GridLab: A Grid Application Toolkit and Testbed Jarosław Nabrzyski GridLab Project Manager Poznań...
GridLab:GridLab:A A Grid Application Grid Application ToolkitToolkit and Testbed and Testbed
Jarosław NabrzyskiGridLab Project ManagerPoznań Supercomputing and Networking Center, Poland
[email protected]@gridlab.org
www.GridLab.org
MANs Meeting, Warsaw, NASK, September 2002
GridLab ProjectGridLab Project
EU Funded (5+M€) 2002-2004 Application and Testbed
orientated Applications and grid
application developers are in the center of the project
MANs Meeting, Warsaw, NASK, September 2002
GridLab MembersGridLab Members PSNC (Poznań) -
coordination AEI (Potsdam) ZIB (Berlin) Univ. of Lecce Cardiff University Vrije Univ.
(Amsterdam) SZTAKI (Budapest) Masaryk Univ. (Brno) NTUA (Athens) Sun Compaq (HP) ANL (Chicago, I. Foster) ISI (LA, C.Kesselman) UoWisconsin (M. Livny)
Also collaborating with: Users!!!
(EU Astrophysics Network,High Performance Computing Group, Ottawa)
DFN TiKSL/GriKSL NSF ASC Project GrADS Project GriPhyn/iVDGL
MANs Meeting, Warsaw, NASK, September 2002
GridLab on a mapGridLab on a map
MANs Meeting, Warsaw, NASK, September 2002
GridLab AimsGridLab Aims
Get Computational Scientists using the “Grid” and Grid services for real, everyday, production work (AEI Relativists, EU Network, Grav Wave Data Analysis, Cactus User Community).
Make it easier for applications to make flexible, efficient, robust, use of the resources available to their virtual organizations.
Dream up, prototype, and test new application scenarios which make adaptive, dynamic, wild, and futuristic uses of resources.
MANs Meeting, Warsaw, NASK, September 2002
What GridLab Isn’t …What GridLab Isn’t …
Don’t want to develop low level Grid Infrastructure
Don’t want to repeat work which has already been done (want to incorporate and assimilate it … Globus APIs, OGSA, ASC Portal (GridSphere/Orbiter), GPDK, GridPort, DataGrid, GriPhyn)
MANs Meeting, Warsaw, NASK, September 2002
GridLab end user GridLab end user requirementsrequirements
Give us an application oriented environment, Applications running on resources of one or
more virtual organisations, Flexible, easy-to-use, simple interfaces to
resources, jobs, and data (including compiling, tracking jobs, cataloguing data)
Give us means to make efficient and effective use of resources,
Fault tolerance is a key requirement, The ability to work in a disconnected
environment, Mobile working,
MANs Meeting, Warsaw, NASK, September 2002
GridLab end user requirements GridLab end user requirements (cont)(cont)
The ability to run in environments as minimalistic as laptops with no grid infrastructure to fully deployed Virtual Organisations
Complexity should be hidden as much as possible,
Provide a collaborative infrastructure, The infrastructure must cater for all classes of
applications, from lightweight to largescale, The infrastructure must provide capabilities to
customise choice of service implementation (e.g. using efficiency, reliability, first succeeding, all).
MANs Meeting, Warsaw, NASK, September 2002
SolutionSolution
“Grid Application Toolkit” or GATProvides a layer between applications and emerging grid technologies. Provides an application developer orientated API, allowing the flexible use of different tools and services, as well as providing protection from developing software.
“GridLab Testbed/VO”Diverse controllable environment for developing and testing applications and tools, software maintained by people who know it.
End Users
GAT Tool Developers
Grid Infrastructure
Developers
GAT-APIDevelopers
MANs Meeting, Warsaw, NASK, September 2002
General ArchitectureGeneral Architecture
MANs Meeting, Warsaw, NASK, September 2002
What is the GAT ?What is the GAT ?
Set of application developer APIs for Grid tools, services and software libraries, (and example implementations) that support the development of grid-enabled aplications (open source!)
Usable from any high level “application” (any generic code, Cactus, Triana, Portals, Scripts, …)
More or less … Set of calls GAT_ToolOrService(arguments) Your chosen tools/services: Resource broker,
information server, application manager, grid monitoring, data manager, notification, etc, etc
Set of APIs for dealing with the GAT (registration, information, errors, fault tolerance)
MANs Meeting, Warsaw, NASK, September 2002
GAT ArchitectureGAT Architecture
Applications
GAT-API
GridLab Services
Grid Core Services
Cactus, Triana, Generic Codes, Portals, Scripts
High level calls based on functionality
GAT_FileCopy, GAT_ResourceFind
Resource Broker, Monitoring System,
Data ManagerGRAM, GridFTP, GridSSH, MDS
but not limited to Globus!
(Triana and JXTA)
MANs Meeting, Warsaw, NASK, September 2002
Proposed GAT Proposed GAT ImplementationImplementation
Applications
GAT Engine
GAT Adaptors
Grid Infrastructure(GAT/GridLab+base/core grid services)
Cactus, Triana, Generic Codes, Portals, Scripts
Layer providing access to services
Resource brokers, basic grid services (GRAM,
GSIFTP, MDS)
Library containing registration routines and empty implementations
GAT-API
MANs Meeting, Warsaw, NASK, September 2002
GAT ExampleGAT Example
Machine A
GridftpGridscp
Machine B
Gridscp
“Application” (Cactus Migration Tool, User Portal, Grid Script)
wants to move a file between two other
machines
How to move the file?
Available software
User authentification
Disk properties (disksize,
user quota, inodes)
Network bandwidth
MANs Meeting, Warsaw, NASK, September 2002
GAT ExampleGAT Example
GAT_FileCopy(GAT, “ThisMachine:ThisFile”,”ThatMachine:ThatFile”)
GAT_GFTPAdaptorIGAT_FileCopyIGAT_FileMove
IGAT_MkDir
GAT API call from Cactus Application, or Portal, or
Python script
GridFTP Adaptor says it knows how to both Copy and Move files between machines
GridSSH Adaptor only knows how to Copy files
GAT_GSCPAdaptorIGAT_FileCopy
GAT EngineEngine decides (by different means) how to respect this
request
MANs Meeting, Warsaw, NASK, September 2002
ImplementationImplementation
GridLab is an early OGSA adopter, however We start with the web services
implementation (all the GridLab services will be web services based and in the next step OGSA compliant)
Several implementations of GAT will be developed
MANs Meeting, Warsaw, NASK, September 2002
Workpackages Workpackages
WP1: Grid Application Toolkit (AEI) This is a key component of GridLab - link between Grid
middleware and applications, usable by any conforming application or middleware component. Requiring input from, and connecting to, most other workpackages and components.
WP2: Cactus Grid Application Toolkit (AEI) Provides an extended GAT interface for Cactus, a very
general toolkit framework supporting different Grid applications, from astrophysics to chemical engineering. Cactus will be one of the primary application drivers for the GAT, and the project generally.
WP3: Work-flow Application Toolkit (CARDIFF) Will develop Grid capabilities for a widely used dataflow
programming environment, Triana, used in gravitational wave and other data analysis areas.
MANs Meeting, Warsaw, NASK, September 2002
Workpackages (cont.)Workpackages (cont.)
WP4: Grid Portals (AEI) Will be highly application driven, aimed at providing uniform,
flexible and intuitive user access to Grid resources from anywhere, as well as administration tools for maintaining a Grid environment.
WP5: Testbed management (MU) Will administrate and maintain an active development testbed
across roughly a dozen EU sites (leveraging the work of the EGrid), deploying technologies as they are developed by the project. This workpackage will also coordinate with sites in the USA-based NCSA Alliance and others to test and develop interoperability.
WP6: Security (PSNC) Will develop the required security mechanisms and will ensure
the integration of all the technologies developed under other WPs, taking into account the various local security requirements and state of the art solutions.
MANs Meeting, Warsaw, NASK, September 2002
Workpackages (cont.)Workpackages (cont.)
WP7: Adaptive Application Components (VU) develops a set of components and APIs to be plugged into the
toolkit, for example to take monitoring information and implement basic techniques for short-term forecasting and behavior adaptation/optimization.
WP8: Data Handling and Visualization (ZIB) will provide Grid aware techniques for data management, analysis,
and visualization, needed especially for applications that make use of multiple sites in a dynamic, time dependent manner, leaving data unpredictably scattered across the Grid.
WP9: Resource Management (PSNC) will develop resource need estimators, resource brokers, and other
tools, for both Grid users and the applications themselves to make intelligent decisions about which Grid resources should be used at any instant in the lifetime of a simulation.
MANs Meeting, Warsaw, NASK, September 2002
Workpackages (cont.)Workpackages (cont.)
WP10: Information Services (ISUFI) will extend existing Grid middleware toolkits with
dynamic features needed by applications to select appropriate Grid resources and to provide simulation information to collaborative user groups.
WP11: Monitoring (SZTAKI) will develop new components that will fit in the
general Grid monitoring architecture to support application steering, adaptive monitoring, and automatic analysis and prediction of performance data.
WP12: Access for mobile users (PSNC) will develop and test Grid access and monitoring
technologies through a variety of mobile devices,
MANs Meeting, Warsaw, NASK, September 2002
Workpackages (cont.)Workpackages (cont.)
WP13: Information Dissemination and Exploitation (PSNC) will ensure the active dissemination of the project results
through a variety of channels, including active participation in international organizations (e.g. GGF), co-development with other Grid projects in the USA and EU, participation in international conferences, training programs, instruction of GridLab technologies into various communities, and introduction into the commercial vendor world.
WP14: Project Management (PSNC) day-to-day scientific, financial and administrative
management of the project, including careful orchestration and monitoring of work across groups, major project decisions, liaisons with external projects and with the international advisory board, reporting
MANs Meeting, Warsaw, NASK, September 2002
More info / summaryMore info / summary
www.GridLab.org [email protected],
[email protected] [email protected]’re welcome to join our testbed We are open for collaborations!