Ian FosterArgonne National Laboratory

University of Chicago

Univa Corporation

Globus®

Bridging the Gap Steve Tuecke

CEO, Univa Corporation

2

Overview

Why Globus New partners What’s next The larger context The Globus community

3

Overview

Why Globus Motivation Uses

New partners What’s next The larger context The Globus community

4

The Application-Infrastructure Gap

Dynamicand/or

DistributedApplications

A

1

B

1

99

Shared Distributed Infrastructure

5

Bridging the Gap:Service-Oriented Infrastructure

Service-oriented applications Wrap applications as

services Compose applications

into workflows Service-oriented

infrastructure Provision physical

resources to support application workloads

ApplnService

ApplnService

Users

Workflows

Composition

Invocation

Provisioning

6

Globus is Service-Oriented Infrastructure Technology

Software for service-oriented infrastructure Service enable new & existing resources E.g., GRAM on computer, GridFTP on storage system,

custom application service Uniform abstractions & mechanisms

Tools to build applications that exploit service-oriented infrastructure Registries, security, data management, …

Open source & open standards Each empowers the other

Enabler of a rich tool & service ecosystem

7

Globus asService-Oriented Infrastructure

IBM

IBM

Uniform interfaces,security mechanisms,Web service transport,

monitoring

Computers StorageSpecialized resource

UserApplication

UserApplication

UserApplication

IBM

IBM

GRAM GridFTPHost EnvUser Svc

DAIS

Database

ToolTool Reliable

FileTransfer

MyProxy

Host EnvUser Svc

MDS-Index

A Typical eScience Use of Globus:Network for Earthquake Eng. Simulation

Links instruments, data, computers, people

An eBusiness Use of Globus:SAP Demonstration @ GlobusWorld

3 Globus-enabled applns: CRM: Internet Pricing Configurator (IPC) CRM: Workforce

Management (WFM) SCM: Advanced Planner

& Optimizer (APO) Applications modified to:

Adjust to varying demand & resources

Use Globus to discover & provision resources

IPCDispatcher

IPCServerRequest:

Price Query

Delegation ofRequest

Response: PricelistDepending on: - Time - Discount - Number of Items - …

Web Browsers / Batch Processes(typically several thousand requests)

IPCServer

1

2

2

3

SAP AG R/3 Internet Pricing & Configurator (IPC)

10

A

1

B

1

9

A

1 2

B

1 2

1

7

Strategy A:Reduce TCO

18 Servers 11 Servers

A

1 2

B

1 2

1

14

Strategy B:Improve

Performance

9 Servers Each 16 Servers Each

9

Cheaper and/or Faster ...

11

Overview

Why Globus New partners

Globus Alliance Globus Consortium Univa

What’s next The larger context The Globus community

12

How Globus Works

Globus is a distributed open source community with many contributors & users CVS, documentation, bugzilla, email lists Modular structure allows many to contribute

Globus Alliance Board provides governance when needed Meritocracy: individuals who demonstrate ongoing

contributions & commitment Primarily: what to include, when to release

Globus Alliance is an informal partnership of organizations led by Board members

13

Evolution of the Globus Alliance

Argonne/U.Chicago (Childers, Foster): 1995 USC/ISI (Kesselman): 1995 Edinburgh (Atkinson, Parsons): 2003 Swedish PDC (Johnsson, Mulmo): 2003 NCSA (Welch): 2004 Univa (Czajkowski, Tuecke): 2004

Other contributors will surely be added

14

From eScience to eBusiness

Since ~2001, growing interest in Globus for commercial use Enterprises, IT vendors, ISVs asking Globus

leaders to address commercial needs But hard to do in a research laboratory

In response, we have created two new organizations Globus Consortium Univa

15

Globus Consortium(www.globusconsortium.com)

Nonprofit organization funded by companies to advance Globus Toolkit for enterprise use Initial sponsor members: HP, IBM, Intel, Sun Initial contributors: Nortel, Univa

First two projects already identified Member-driven software quality improvements Contributions to job submission standards

Other projects to be defined, e.g. Develop new features key to enterprise use Education & outreach

16

Provider of commercial support, services, & products around open source Globus Commercial distribution of GT4 & beyond Integration with enterprise systems Committed to open source & open standards

Founded by Tuecke, Foster, Kesselman Tuecke left Argonne to be CEO Foster, Kesselman remain at Argonne, ISI

Experienced management team Rich Miller, Vas Vasiliadis, Paul Davé,

Bob Mandel

17

Overview

Why Globus New partners What’s next

Globus Toolkit v4 The larger context The Globus community

18

Globus asService-Oriented Infrastructure

IBM

IBM

GRAM GridFTPHost EnvUser Svc

ReliableFile

Transfer

MyProxy

Uniform interfaces,security mechanisms,Web service transport,

monitoring

Host EnvUser Svc

Computers StorageSpecialized resource

UserApplication

UserApplication

UserApplication

IBM

IBM

DAIS

Database

MDS-Index

ToolTool

Data Management

SecurityCommonRuntime

Execution Management

Information Services

Web Services

Components

Non-WS

Components

Pre-WSAuthenticationAuthorization

GridFTP

GridResource

Allocation Mgmt(Pre-WS GRAM)

Monitoring& Discovery

System(MDS2)

C CommonLibraries

GT2

WSAuthenticationAuthorization

ReliableFile

Transfer

OGSA-DAI[Tech Preview]

GridResource

Allocation Mgmt(WS GRAM)

Monitoring& Discovery

System(MDS4)

Java WS Core

CommunityAuthorization

ServiceGT3

ReplicaLocationService

XIO

GT3

CredentialManagement

GT4

Python WS Core[contribution]

C WS Core

CommunitySchedulerFramework

[contribution]

DelegationService

GT4

Globus Open Source Grid Software

21

Java Services in Apache AxisPlus GT Libraries and Handlers

YourJava

Service

YourPythonService

YourJava

Service RF

T

GR

AM

Del

egat

ion

Inde

x

Trig

ger

Arc

hive

r

pyGlobusWS Core

YourC

Service

C WS Core

RLS

Pre

-WS

MD

S

CA

S

Pre

-WS

GR

AM

Sim

pleC

A

MyP

roxy

OG

SA

-DA

I

GT

CP

Grid

FT

P

C Services using GT Libraries and Handlers

SERVER

CLIENT

InteroperableWS-I-compliant

SOAP messaging

YourJavaClient

YourC

Client

YourPythonClient

YourJavaClient

YourC

Client

YourPythonClient

YourJavaClient

YourC

Client

YourPythonClient

YourJavaClient

YourC

Client

YourPythonClient

X.509 credentials =common authentication

Python hosting, GT Libraries

GT4 Components

22

Our Goals for GT4

Usability, reliability, scalability, … Web service components have quality equal

or superior to pre-WS components Documentation at acceptable quality level

Consistency with latest standards (WS-*, WSRF, WS-N, etc.) and Apache platform WS-I Basic (Security) Profile compliant

New components, platforms, languages And links to larger Globus ecosystem

23

GT4 Web Services Core

Supports both Globus services (GRAM, RFT, Delegation, etc.) & user-developed services

Redesign to enhance scalability, modularity, performance, usability

Leverages existing WS standards WS-I Basic Profile: WSDL, SOAP, etc. WS-Security, WS-Addressing

Adds support for emerging WS standards WS-Resource Framework, WS-Notification

Java, Python, & C hosting environments

24

CustomWeb

ServicesWS-Addressing, WSRF,

WS-Notification

CustomWSRF Web

Services

GT4WSRF Web

Services

WSDL, SOAP, WS-Security

User Applications

Reg

istr

yA

dmin

istr

atio

n

GT

4 C

onta

iner

GT4 Web Services Core

26

GT4 Security

VO

RightsUsers

Rights’

ComputeCenter

Access

Services (runningon user’s behalf)

Rights

Local Policyon VO identityor attributeauthority

CAS or VOMSissuing SAMLor X.509 ACs

SSL/WS-Securitywith ProxyCertificates

Authz Callout

KCA

MyProxy

27

GT4 Security Extensible authorization framework based on

Web services standards SAML-based authorization callout

As specified in GGF OGSA-Authz WG

Integrated policy decision engine XACML policy language, per-operation policies, pluggable

Credential management services MyProxy (One time password support) Community Authorization Service

Standalone delegation service Implements IETF RFC

28

GT4’s Use of Security Standards

29

GT4 Data Management

Stage large data to/from nodes Replicate data for performance & reliability Locate data of interest Provide access to diverse data sources

File systems, parallel file systems, hierarchical storage (GridFTP)

Databases (OGSA DAI)

30

GT4 Data Functions

Find your data: Replica Location Service Managing ~40M files in production settings

Move/access your data: GridFTP, RFT High-performance striped data movement

27 Gbit/s memory-to-memory on a 30 Gbit/s link (90% utilization) with 32 IBM TeraGrid nodes.

17.5 Gbit/s disk-to-disk limited by the storage system Reliable movement of 120,000 files (so far)

Couple data & execution management GRAM uses GridFTP and RFT for staging

31

GridFTP in GT4

100% Globus code No licensing issues Stable, extensible

IPv6 Support XIO for different transports Striping multi-Gb/sec wide area transport Pluggable

Front-end: e.g., future WS control channel Back-end: e.g., HPSS, cluster file systems Transfer: e.g., UDP, NetBLT transport

Bandwidth Vs Striping

0

2000

4000

6000

8000

10000

12000

14000

16000

18000

20000

0 10 20 30 40 50 60 70

Degree of Striping

Ba

nd

wid

th (

Mb

ps

)

# Stream = 1 # Stream = 2 # Stream = 4

# Stream = 8 # Stream = 16 # Stream = 32

Disk-to-disk onTeraGrid

32Reliable File Transfer:Third Party Transfer

RFT Service

RFT Client

SOAP Messages

Notifications(Optional)

DataChannel

Protocol Interpreter

MasterDSI

DataChannel

SlaveDSI

IPCReceiver

IPC Link

MasterDSI

Protocol Interpreter

Data Channel

IPCReceiver

SlaveDSI

Data Channel

IPC Link

GridFTP Server GridFTP Server

Fire-and-forget transfer Web services interface Many files & directories Integrated failure recovery

33

Replica Location Service

Identify location of files via logical to physical name map

Distributed indexing of names, fault tolerant update protocols

GT4 version scalable & stable

Managing ~40 million files across ~10 sites

IndexIndex

Local DB

Update send (secs)

Bloom filter

(secs)

Bloom filter (bits)

10K <1 2 1 M

1 M 2 24 10 M

5 M 7 175 50 M

34

Execution Management (GRAM)

Common WS interface to schedulers Unix, Condor, LSF, PBS, SGE, …

More generally: interface for process execution management Lay down execution environment Stage data Monitor & manage lifecycle Kill it, clean up

A basis for application-driven provisioning

35

GT4 GRAM

2nd-generation WS implementation optimized for performance, flexibility, stability, scalability

Streamlined critical path Use only what you need

Flexible credential management Credential cache & delegation service

GridFTP & RFT used for data operations Data staging & streaming output Eliminates redundant GASS code

36

GRAMservices

GT4 Java Container

GRAMservices

Delegation

RFT FileTransfer

Transferrequest

GridFTPRemote storage element(s)

Localscheduler

Userjob

Compute element

GridFTP

sudo

GRAMadapter

FTPcontrol

Local jobcontrol

Delegate

FTP data

Cli

ent

Job

functions

Delegate

Service host(s) and compute element(s)

GT4 GRAM Structure:WSRF/WSN Poster Child

37

Monitoring and Discovery “Every service should be monitorable and

discoverable using common mechanisms” WSRF/WSN provides those mechanisms

A common aggregator framework for collecting information from services, thus: MDS-Index: Xpath queries, with caching MDS-Trigger: perform action on condition MDS-Archiver: Xpath on historical data

Deep integration with Globus containers & services: every GT4 service is discoverable GRAM, RFT, GridFTP, CAS, …

38

GT4 Container

GT4 Monitoring & Discovery

GRAM User

MDS-Index

GT4 Cont.

RFT

MDS-Index

GT4 Container

MDS-Index

GridFTP

adapter

Registration &WSRF/WSN Access

Custom protocolsfor non-WSRF entities

Clients(e.g., WebMDS)

Automatedregistrationin container

WS-ServiceGroup

GT4 Documentation

is Much Improved!

40

Working with GT4

Download and use the software, and provide feedback Join gt4friends@globus.org mail list

Review, critique, add to documentation Globus Doc Project: http://gdp.globus.org

Tell us about your GT4-related tool, service, or application Email foster@mcs.anl.gov

41

Overview

Why Globus New partners What’s next The larger context

Web services, utility computing, virtualization, …

The Globus community

42

Infrastructure Complexity:What’s the Answer?

Web services Utility computing Virtualization Data center automation Adaptive enterprise …

Recall:

Service-oriented applications

demand

service-oriented infrastructure

43

Overview

Why Globus New partners What’s next The larger context The Globus community

Who’s using Globus? What are they using it for? The Globus ecosystem Growing the Globus community

26776 U.S.

2753 China

1318 Japan

1017 India

750 U.K.

495 Italy

488 Germany

391 Brazil

328 S. Korea

306 Taiwan

268 France

241 Canada

211 Viet Nam

211 Spain

202 Russia

187 Pakistan

159 Australia

142 Singapore

131 Greece

119 Colombia

111 Poland

109 Netherlands

107 Thailand

94 Switzerland

77 Chile

74 Sweden

68 Belgium

66 Venezuela

66 Romania

64 Indonesia

62 Mexico

61 Turkey

60 Malaysia

58 Portugal

57 Austria

54 Ireland

44 Hong Kong

40 Hungary

38 Egypt

38 Argentina

34 Uruguay

31 Ukraine

29 Slovakia

25 Israel

23 Yugoslavia

23 Iran

22 Bulgaria

22 Uzbekistan

22 Czech Rep.

22 N. Korea

21 Lithuania

21 Croatia

20 Finland

20 New Zealand

17 Nigeria

17 South Africa

16 Jordan

16 Slovenia

16 Afghanistan

15 Denmark

15 Philippines

14 Vanuatu

14 Luxembourg

14 Tunisia

12 Virgin Is. (U.K.)

12 Peru

12 Yemen

11 Norway

11 Algeria

11 Armenia

10 Iceland

9 Zambia

9 Virgin Is. (U.S.)

9 Uganda

9 Bosnia & Herz.

8 Kenya

7 Zimbabwe

7 Saudi Arabia

7 Ecuador

7 Macedonia

6 Bolivia

6 Comoros

6 Zaire

6 Lebanon

5 Puerto Rico

5 Namibia

5 Togo

5 Tajikistan

5 Paraguay

5 Albania

5 Sudan

4 Estonia

4 Camaroon

4 Ghana

4 Tuvalu

4 Costa Rica

4 Cuba

4 UAE

4 Tonga

4 W. Samoa

4 Tanzania

3 Syria

3 Bahamas

3 Ethiopia

3 Mongolia

3 Sri Lanka

3 Wallis & Futuna Is.

3 Belarus

3 Bangladesh

2 Falkland Islands

2 Kuwait

2 Sierra Leone

2 Trinidad & Tobago

2 Guyana

2 American Samoa

2 Andorra

2 Georgia

2 Cook Islands

2 Turkmenistan

2 Gabon

2 The Gambia

2 Kazakhstan

2 Macau

2 Malta

2 Jamaica

2 Latvia

2 Turks & Caicos

1 Bhutan

1 Ascension Island

1 Cyprus

1 Mozambique

1 Tokelau

1 Greenland

1 Nepal

1 Swaziland

1 Iraq

1 Serbia

1 Barbados

1 Cambodia

1 Qatar

1 Saint Vincent

1 Laos

1 San Marino

1 Libya

1 Benin

1 Angola

1 Chad

1 Gibraltar

1 Haiti

1 Guatemala

1 Malawi

1 Equatorial Guinea

1 Palau

1 Bermuda

1 Botswana

1 Suriname

386

69 d

ow

nlo

ad

s in

20

04

fro

m g

lobu

s.org

26776 U.S.

2753 China

1318 Japan

1017 India

750 U.K.

495 Italy

488 Germany

391 Brazil

328 S. Korea

306 Taiwan

268 France

241 Canada

211 Viet Nam

211 Spain

202 Russia

187 Pakistan

159 Australia

142 Singapore

131 Greece

119 Colombia

111 Poland

109 Netherlands

107 Thailand

94 Switzerland

77 Chile

74 Sweden

68 Belgium

66 Venezuela

66 Romania

64 Indonesia

62 Mexico

61 Turkey

60 Malaysia

58 Portugal

57 Austria

54 Ireland

44 Hong Kong

40 Hungary

38 Egypt

38 Argentina

34 Uruguay

31 Ukraine

29 Slovakia

25 Israel

23 Yugoslavia

23 Iran

22 Bulgaria

22 Uzbekistan

22 Czech Rep.

22 N. Korea

21 Lithuania

21 Croatia

20 Finland

20 New Zealand

17 Nigeria

17 South Africa

16 Jordan

16 Slovenia

16 Afghanistan

15 Denmark

15 Philippines

14 Vanuatu

14 Luxembourg

14 Tunisia

12 Virgin Is. (U.K.)

12 Peru

12 Yemen

11 Norway

11 Algeria

11 Armenia

10 Iceland

9 Zambia

9 Virgin Is. (U.S.)

9 Uganda

9 Bosnia & Herz.

8 Kenya

7 Zimbabwe

7 Saudi Arabia

7 Ecuador

7 Macedonia

6 Bolivia

6 Comoros

6 Zaire

6 Lebanon

5 Puerto Rico

5 Namibia

5 Togo

5 Tajikistan

5 Paraguay

5 Albania

5 Sudan

4 Estonia

4 Camaroon

4 Ghana

4 Tuvalu

4 Costa Rica

4 Cuba

4 UAE

4 Tonga

4 W. Samoa

4 Tanzania

3 Syria

3 Bahamas

3 Ethiopia

3 Mongolia

3 Sri Lanka

3 Wallis & Futuna Is.

3 Belarus

3 Bangladesh

2 Falkland Islands

2 Kuwait

2 Sierra Leone

2 Trinidad & Tobago

2 Guyana

2 American Samoa

2 Andorra

2 Georgia

2 Cook Islands

2 Turkmenistan

2 Gabon

2 The Gambia

2 Kazakhstan

2 Macau

2 Malta

2 Jamaica

2 Latvia

2 Turks & Caicos

1 Bhutan

1 Ascension Island

1 Cyprus

1 Mozambique

1 Tokelau

1 Greenland

1 Nepal

1 Swaziland

1 Iraq

1 Serbia

1 Barbados

1 Cambodia

1 Qatar

1 Saint Vincent

1 Laos

1 San Marino

1 Libya

1 Benin

1 Angola

1 Chad

1 Gibraltar

1 Haiti

1 Guatemala

1 Malawi

1 Equatorial Guinea

1 Palau

1 Bermuda

1 Botswana

1 Suriname

386

69 d

ow

nlo

ad

s in

20

04

fro

m g

lobu

s.org

26776 U.S. 2753 China 1318 Japan 1017 India 750 U.K. 495 Italy 488 Germany 391 Brazil 328 S. Korea 306 Taiwan

Top 10

GlobalCommunity

47

Grids for Physics: LHC Computing Grid

48

“Paradigm Shift: Transition to Grid”

Jun: 80%:20%

25% of DC’04

Aug: 27%:73%

42% of DC’04

May: 89%:11%

11% of DC’04

Jul: 77%:23%

22% of DC’04

424 CPU · Years

Slide: Tony Doyle, University of Glasgow

49Grid2003 Open Science Grid 30 sites (2100-2800 CPUs) & growing 400-1300 concurrent jobs 8 substantial applications + CS experiments Running since October 2003

Korea

http://www.ivdgl.org/grid2003

50

ExampleOSG

Workflows

Genome sequence analysis

Physicsdata

analysis

Sloan digital sky

survey

51

Further Application Examples

Earth System Grid:O(100TB) online data

STAR: 5 TB transfer(SRM, GridFTP)

NASA/NVO: Mosaicsfrom multiple sources

Fusion Grid:1000s of jobs

系统架构 ： DartGrid 的分层模型

“We’ve used GT3 to build the largest database grid system for Traditional Chinese Medicine, integrating

about 50 TCM-relevant databases.” (Zhejiang U.)

VO的地址栏

语义浏览面板

本体论树

DartGri d 虚拟组织资源浏览面板

Q3语义查询显示面板

语义注册面板

53

Nimrod-G, Australia

Astrophysics Air PollutionAntenna Design

Airfoil Design

Circuit Design Monte Carlo

Computational Chemistry

Public Health Policy

CFD

Cardiac Modelling

Climate

Earth Sciences

54

The Globus Ecosystem

Globus components address core issues relating to resource access, monitoring, discovery, security, data movement, etc. GT4 being the latest version

A larger Globus ecosystem of open source and proprietary components provide complementary components A growing list of components

These components can be combined to produce solutions to Grid problems We’re building a list of such solutions

55

Many Tools Build on, or Can Contribute to, GT4-Based Grids

Condor-G, DAGman MPICH-G2 GRMS Nimrod-G Ninf-G Open Grid Computing Env. Commodity Grid Toolkit GriPhyN Virtual Data System Virtual Data Toolkit GridXpert Synergy Platform Globus Toolkit

VOMS PERMIS GT4IDE Sun Grid Engine PBS scheduler LSF scheduler GridBus TeraGrid CTSS NEES IBM Grid Toolbox …

56Documenting

The Grid Ecosystem

The Grid Ecosystem: Software Components for Grid SystemsAnd Applications

www.grids-center.org

57

Example Solutions

Portal-based User Reg. System (PURSE) VO Management Registration Service Service Monitoring Service TeraGrid TGCP Tool Lightweight Data Replicator GriPhyN Virtual Data System

58

The Globus Developer Community

We want to facilitate community contributions to the Globus software Increasingly open governance & mechanisms Documentation of larger ecosystem What else can we do?

Tell us how you think we should evolve E.g., adopt Apache processes? Contribute more of Globus to Apache?

We’ve already made major contributions to Apache Axis, WS-Addressing, and WS-Security

Also WSRF/WSN Apache (Apollo, Hermes)

59

Globus and its User Community

How can “we” best support “you”? We try to provide the best software we can We use bugzilla & other community tools We work to grow the set of contributors

How can “you” best support “us”? Become a contributor: of software, bug fixes,

answers to questions, documentation Provide us with success stories that can justify

continued Globus development Promote Globus within your communities

Contact me at foster@mcs.anl.gov

60

2005 and Beyond

We have a solid Web services base We now want to build, on that base, a open

source service-oriented infrastructure Virtualization New services for provisioning, data

management, security, VO management End-user tools for application development Etc., etc.

See Carl Kesselman’s talk for more details!

61

Summary:Globus is Bridging the Gap

Application to infrastructure Science to commercial Web services to Grid Data to computation Silo to silo Organization to organization Country to country Developer to developer Developer to user

Acknowledgments Carl Kesselman, our long-time Globus co-conspirators Gregor von Laszewski, Kate Keahey, Jennifer Schopf, Mike

Wilde, other Argonne colleagues Globus Alliance members at Argonne, U.Chicago, USC/ISI,

Edinburgh, PDC Miron Livny, U.Wisconsin Condor project, Rick Stevens,

Argonne & U.Chicago Other partners in Grid technology, application, &

infrastructure projects DOE SciDAC program, NSF Middleware Initiative, NASA,

IBM for generous support

63

For More Information

Globus Alliance www.globus.org

Globus Consortium www.globusconsortium.com

Global Grid Forum www.ggf.org

Background information www.mcs.anl.gov/~foster

2nd Editionwww.mkp.com/grid2