Grid Computing Yoab Gorfu Abe Guerra Kay Odeyemi Renel Smith.

Grid Computing

Yoab GorfuAbe GuerraKay OdeyemiRenel Smith

Presentation Outline

Introduction Architecture Large Deployment Example - National

Fusion Grid Grid Toolkits

– Globus Toolkit – Stateful Web Services

Introduction

“A computational grid is a hardware and software infrastructure that provides dependable, consistent, pervasive, and inexpensive access to high-end computational capabilities.”

Criteria for a Grid• Coordinates resources that are not subject to

centralized control.• Uses standard, open, general-purpose protocols

and interfaces.• Delivers nontrivial qualities of service.

Introduction

‘Grid Problem’ - ‘coordinated resource sharing and problem solving in dynamic, multi-institutional virtual organizations’ [1]

Virtual Organizations (VOs) • Vary dramatically• Core set of requirements

Introduction

VO requirements• Flexibility• Control• Varied resources• Usage modes

Introduction

Introduction

Grid Computing Benefits:• Exploit underutilized resources• Resource balancing• Virtualize resources across an enterprise• Enable collaboration for virtual organizations

Companies involved in Grid Computing

Avaki Axceleon CapCal Centrata DataSynapse Distributed Science Elepar Entropia.com Grid Frastructure GridSystems Groove Networks IBM Intel

Powerllel ProcessTree Sharman Networks Kazza Sun Gridware Sysnet Solutions Tsunami Research Ubero United Devices Veritas Xcomp

Jivalti Mithral Mind Electric Mojo Nation NewsToYou.com NICE, Italy Noemix, Inc. Oracle Parabon Platform Computing Popular Power

Source: http://www.gridcomputing.com/

Computation Grid Projects

Particle Physics –global sharing of data and computation

Astronomy–‘Virtual Observatory' for multi-wavelength astrophysics

Chemistry–remote control of equipment and electronic logbooks

Engineering–industrial healthcare and virtual organizations

Bioinformatics–data integration, knowledge discovery and workflow

Healthcare –sharing normalized mammograms

Environment–Ocean, weather, climate modeling, sensor networks

Grid Architecture

Protocol architecture Standards-based open architecture offers:

• Interoperability• Services• API flexibility

Grid Architecture

Grid Architecture

Fabric Layer – ‘provides the resources to which shared access is mediated by Grid protocols’• Resource-specific operations• Functionality vs. simplicity

Grid Architecture

Fabric layer should provide:• Enquiry mechanisms• Resource management mechanisms

Grid Architecture

Connectivity Layer – ‘defines core communication and authentication protocols required for Grid-specific network transactions’• Data exchange• Verification

Grid Architecture

Connectivity layer should provide:• Single sign on• Delegation• Integration with various local security solutions• User-based trust relationships

Grid Architecture

Resource Layer – ‘defines protocols for the secure negotiation, initiation, monitoring, control, accounting, and payment of sharing operations on individual resources’• Use Fabric Layer functions• Information vs. Management protocols

Grid Architecture

Resource layer should provide:• Fabric layer functionality• ‘exactly once’ semantics• Error reporting

Grid Architecture

Collective Layer – ‘contains protocols and services which capture interactions across collections of resources’• General vs. specific purpose

Grid Architecture

Collective layer could provide:• Software discovery services• Community accounting and payment services• Collaboratory services

Grid Architecture

Applications Layer – ‘comprises the user applications that operate within a VO environment.’

National Fusion Grid

A Collaboratory Pilot project that is creating and deploying collaborative software tools throughout the magnetic fusion research community


Simple Goals To advance scientific understanding and innovation

in magnetic fusion research by enabling more efficient use of existing experimental facilities and more effective integration of experiment, theory, and modelling.

To advance scientific understanding and innovation in fusion research

Making widespread use of Grid technologies http://www.fusiongrid.org/


VISION FOR THE FUSION GRID Data, Codes, Analysis Routines, Visualization Tools should be

thought of as network accessible services

Shared security infrastructure

Collaborative nature of research requires shared visualization applications and widely deployed collaboration technologies

— Integrate geographically diverse groups

Not focused on CPU cycle scavenging or “distributed” supercomputing (typical Grid justifications)

— Optimize the most expensive resource - people’s time


The problems of data sharing and rapid data analysis the National Fusion Collaboratory community adopted: – a common data acquisition and management

system – common relational database run-management

schema


Geographically Diverse Community

3 Large Experimental Facilities — Alcator, C-Mod, DIII-D — NSTX ~$1B replacement cost

40 U.S. fusion research sites— Over 1000 scientists in 37 state

Efficient collaboration is a requirement! — Integrate geographically diverse groups

One future worldwide machine — Not based in US — US needs collaboration tools to benefit


National Magnetic Fusion Research Community FUSION COMMUNITY HAS 40 US SITES IN 37 STATES


Design and Implementation of Access Grid Produced of both design and architecture

documents for review by public (beginning introduction into GGF document process)

Demonstrated full-featured prototypes in Nov 2002 at SC2002 of new venue architecture, venue client, workspace docking complete with application sharing


Building the Fusion Grid (Progressive testbeds) Deployment Phrase Use Policies and Issues of Trust Moving to Real-Time Wrapping it up

Globus & the Globus Toolkit

Globus– Open source community focused on Grid computing

Globus Toolkit– Started in the late 1990’s to address common Grid

application problems– … found at www.globus.org– Includes

A set of services focused on infrastructure management Tools for building new Web services, in Java, C, and Python Standards-based security infrastructure Client APIs and command line programs

Globus Toolkit & Web Services

Grid

Web

WSRF

Started far apart in apps & tech

OGSI

GT2

GT1

HTTPWSDL,

WS-*

WSDL 2,

WSDM

Have beenconverging

Grid

Web

WSRFWSRFWSRF



OGSI

GT2

GT1

HTTPWSDL,

WS-*

WSDL 2,

WSDM

Have beenconverging

OGSI

GT2

GT1

HTTPWSDL,

WS-*

WSDL 2,

WSDM

OGSI

GT2

GT1

HTTPWSDL,

WS-*

WSDL 2,

WSDM

Have beenconverging

Modeling Stateful Resources with Web Services

Web Services Background What is a Web Service? Web Service Environments A Brief Taxonomy of State and Services Stateless Implementations, Stateful

Interfaces


What is a Web Service?

Machine to Machine over a network via exchange of SOAP messages

Conveyance via HTTP

Key facility in distributed environment known as SOA


Why Web Service in Grid Discussion?

Convergence in Grid and SOA

Many grid implementations use Web Services


Web Services are usually Stateless

All information needed by the service is contained in the input message

All results are return via the output message The service does not ‘remember’ what it just

did on completion Not that useful for Grid


State and Web Services

Most applications are not stateless

Grid application need their components to keep state

Web services can be components of Grid applications


State and Web Services

Two general ways for representing state

The service keep track of it’s state

The service has other systems keep track of state for it

Ideally, Option 2 preferred


WS-Resource

Protocol for modeling stateful resources

Standards for read, update and querying of state values.


WS-Resource Lifecycle

Assignment & Use

Destruction

Creation


WS-Resource Example


WS Resource – ACID properties

Atomicity

Consistency

Isolation

Durability

References

Foster, Ian; “Globus Toolkit Version 4: Software for Service-Oriented Systems:, IFIP International Conference on Network and Parallel Computing, Springer-Verlag LNCS 3779, pp 2-13, 2005

Foster, Ian; “WS-Resource Framework: Globus Alliance Perspectives”, GlobusWORLD, January 20, 2004

Foster, I., C. Kesselman, and S. Tuecke, The Anatomy of the Grid: Enabling Scalable Virtual Organizations. International Journal of High Performance Computing Applications, 2001. 15(3): p. 200-222.

Foster, I., Frey, J., Graham, S., Tuecke, S., Czajkowski, K., Ferguson, D., Leymann, F., Nally, M., Storey, T. and Weerawaranna, S. Modeling Stateful Resources with Web Services. Globus Alliance, 2004.

Keahey, K, Fredian, T., Peng, D.P. Schissel, M. Thompson, I. Foster, M. Greenwald, D. McCune, Computational Grids in Action: The National Fusion Collaboratory, submitted to Future Generation Computer System, October 2002. 18(8): p. 1005-1015.

Grid Computing Yoab Gorfu Abe Guerra Kay Odeyemi Renel Smith.

Documents

Transcript of Grid Computing Yoab Gorfu Abe Guerra Kay Odeyemi Renel Smith.