Transcript of The Grid Ian Foster Argonne National Laboratory University of Chicago Globus Alliance foster.
- Slide 1
- The Grid Ian Foster Argonne National Laboratory University of
Chicago Globus Alliance www.mcs.anl.gov/~foster
- Slide 2
- 2 foster@mcs.anl.gov ARGONNE CHICAGO Grid Computing is in the
News
- Slide 3
- 3 foster@mcs.anl.gov ARGONNE CHICAGO Why Should You Care? 1)
Grid is a disruptive technology [Vision] u It ushers in a
virtualized, collaborative, distributed world 2) Grid addresses
pain points now [Reality] u Grids are built not bought, but are
delivering real benefits in commercial settings 3) An open Grid is
to your advantage [Future] u Standards are being defined now that
will determine the future of this technology
- Slide 4
- 4 foster@mcs.anl.gov ARGONNE CHICAGO Why Should You Care? 1)
Grid is a disruptive technology [Vision] u It ushers in a
virtualized, collaborative, distributed world 2) Grid addresses
pain points now [Reality] u Grids are built not bought, but are
delivering real benefits in commercial settings 3) An open Grid is
to your advantage [Future] u Standards are being defined now that
will determine the future of this technology
- Slide 5
- 5 foster@mcs.anl.gov ARGONNE CHICAGO The Power Grid: On-Demand
Access to Electricity Time Quality, economies of scale Decouple
production & consumption, enabling l On-demand access l
Economies of scale l Consumer flexibility l New devices
- Slide 6
- 6 foster@mcs.anl.gov ARGONNE CHICAGO But Computing Isnt Really
Like Electricity! l How about access computing resources like we
access Web content? u We have no idea where a website is, or on
what computer or operating system it runs Two interrelated
opportunities 1) Enhance economy, flexibility, access by
virtualizing computing resources 2) Deliver entirely new
capabilities by integrating distributed resources
- Slide 7
- 7 foster@mcs.anl.gov ARGONNE CHICAGO Virtualization Servers:
Execution Application Services: Distribution Applications: Delivery
Application Virtualization Automatically connect applications to
services Dynamic & intelligent provisioning Infrastructure
Virtualization Dynamic & intelligent provisioning Automatic
failover Source: The Grid: Blueprint for a New Computing
Infrastructure (2 nd Edition), 2004
- Slide 8
- 8 foster@mcs.anl.gov ARGONNE CHICAGO Distributed System
Integration NY Financial Institution Insurance Group UK Financial
Institution NY Financial Institution Capital Markets Group
- Slide 9
- 9 foster@mcs.anl.gov ARGONNE CHICAGO And Ultimately
- Slide 10
- 10 foster@mcs.anl.gov ARGONNE CHICAGO The Grid Resource sharing
& coordinated problem solving in dynamic virtual organizations
1. Enable integration of distributed service & resources 2.
Using general-purpose protocols & infrastructure 3. To achieve
useful qualities of service The Anatomy of the Grid, Foster,
Kesselman, Tuecke, 2001
- Slide 11
- 11 foster@mcs.anl.gov ARGONNE CHICAGO Terminology (1) Cluster
Grid Enterprise Grid Global Grid
- Slide 12
- 12 foster@mcs.anl.gov ARGONNE CHICAGO Terminology (2) Autonomic
Computing Service- Oriented Architecture Utility Computing Utility
Computing Autonomic Computing Service- Oriented Architecture
Grid
- Slide 13
- 13 foster@mcs.anl.gov ARGONNE CHICAGO Why Should You Care? 1)
Grid is a disruptive technology [Vision] u It ushers in a
virtualized, collaborative, distributed world 2) Grid addresses
pain points now [Reality] u Grids are built not bought, but are
delivering real benefits in commercial settings 3) An open Grid is
to your advantage [Future] u Standards are being defined now that
will determine the future of this technology
- Slide 14
- 14 foster@mcs.anl.gov ARGONNE CHICAGO Grid Addresses Pain
Points Now l Low utilization of enterprise resources l High cost of
provisioning for peak demand l Inadequate resources prevent use of
advanced applications l Lack of information integration
- Slide 15
- 15 foster@mcs.anl.gov ARGONNE CHICAGO Grid Deployment Trends
Mission Criticality Department Enterprise Collaboration Internet
Corporate Scientific Corporate
- Slide 16
- 16 foster@mcs.anl.gov ARGONNE CHICAGO Gridified Infrastructure
Early Commercial Applications Financial Services Derivatives
Analysis Statistical Analysis Portfolio Risk Analysis Derivatives
Analysis Statistical Analysis Portfolio Risk Analysis Manufacturing
Mechanical/ Electronic Design Process Simulation Finite Element
Analysis Failure Analysis Mechanical/ Electronic Design Process
Simulation Finite Element Analysis Failure Analysis LS /
Bioinformatics Cancer Research Drug Discovery Protein Folding
Protein Sequencing Cancer Research Drug Discovery Protein Folding
Protein Sequencing Other Web Applications Weather Analysis Code
Breaking/ Simulation Academic Web Applications Weather Analysis
Code Breaking/ Simulation Academic Sources: IDC, 2000 and Bear
Stearns- Internet 3.0 - 5/01 Analysis by SAI Grid Services Market
Opportunity 2005 Energy Seismic Analysis Reservoir Analysis Seismic
Analysis Reservoir Analysis Entertainment Digital Rendering Massive
Multi-Player Games Massive Multi-Player Games Streaming Media
Leading adopters (Oct 2003) * Financial services: 31% Life
sciences: 26% Manufacturing: 18% *Grids 2004: From Rocket Science
To Business Service, The 451 Group
- Slide 17
- 17 foster@mcs.anl.gov ARGONNE CHICAGO Grid Deployment
Strategies l A range of excellent commercial & open source
products for resource federation u Federate enterprise computing
resources u Federate enterprise information resources u Globus
Toolkit : inter-enterprise sharing l But, Grids are built, not
bought u Integration with other enterprise systems is needed to
deliver complete solution l Start small & with well-defined ROI
case u Grow based on experience
- Slide 18
- 18 foster@mcs.anl.gov ARGONNE CHICAGO Novartis PC Grid links
3,700 desktop systems u Research & development applications u
Potentially mainstream business computing u > 5 teraflop/s
computing power u Estimate savings of $200M over 3 years We have
projects we calculate would take 6 years on a single supercomputer.
Today, the run time is 12 hours. Peter Sany, Novartis CIO
- Slide 19
- 19 foster@mcs.anl.gov ARGONNE CHICAGO University of Texas
- Slide 20
- 20 foster@mcs.anl.gov ARGONNE CHICAGO Royal Dutch/Shell l
Improve accuracy and speed of summarization & scientific
modeling applications u More robust, scalable IT infrastructure
that adjusts as volumes fluctuate u Open standards ease integration
of software u Cut processing time of seismic data, while improving
the quality of the output u Focus employees on key scientific, not
IT problems Grid computing is important to Shell because it offers
the potential to create a truly unlimited resource, with a uniform
interface to a variety of services. This is a significant
opportunity for Shell to engage its independent companies in closer
cooperation. J.N. Buur, Principal Research Physicist, Shell
International Exploration and Production B.V.
- Slide 21
- 21 foster@mcs.anl.gov ARGONNE CHICAGO Kansai Electric Power Co.
l Japans second largest electric utility company has various
information in a heterogeneous, distributed database environment l
Integrate information across departments and affiliated companies
to enable information sharing Federated Database DB2 Server ADABAS
Server NOTES Server Oracle Server Application DB Client CRM
Accounting Workflow
- Slide 22
- NEES (Network for Earthquake Engineering Simulation)
Collaboratory U.Nevada Reno www.neesgrid.org
- Slide 23
- 23 foster@mcs.anl.gov ARGONNE CHICAGO NCSA Computational Model
m1m1 f1f1 UIUC Experimental Model f1f1 m1m1 f2f2 f2f2 U. Colorado
Experimental Model NEESgrid at Work
- Slide 24
- 24 foster@mcs.anl.gov ARGONNE CHICAGO CMS High Energy Physics l
Generate data for a physics experiment u Simulate 1.5 million
events for physics studies @ ~500 sec per event u 2 months
continuous running across 5 sites u Managed by a single person
- Slide 25
- 25 foster@mcs.anl.gov ARGONNE CHICAGO Why Should You Care? 1)
Grid is a disruptive technology [Vision] u It ushers in a
virtualized, collaborative, distributed world 2) Grid addresses
pain points now [Reality] u Grids are built not bought, but are
delivering real benefits in commercial settings 3) An open Grid is
to your advantage [Future] u Standards are being defined now that
will determine the future of this technology
- Slide 26
- 26 foster@mcs.anl.gov ARGONNE CHICAGO Grid Vision, Marketing,
and Reality l Vision u Computing resources can be shared like
content on the Wb l Marketing u Have we got a [Data, compute,
knowledge, information, desktop, PC, enterprise, cluster, ] Grid
for you! l Reality u Commercial products mostly noninteroperable u
Open source tools offer de facto standards, but are also far from a
complete solution
- Slide 27
- 27 foster@mcs.anl.gov ARGONNE CHICAGO Standards Matter! l Open,
standard protocols u Enable interoperability u Avoid product/vendor
lock-in u Enable innovation/competition on end points u Enable
ubiquity l In Grid space, must address how we u Describe, discover,
& access resources u Monitor, manage, & coordinate,
resources u Account & charge for resources For many different
types of resource
- Slide 28
- 28 foster@mcs.anl.gov ARGONNE CHICAGO Increased functionality,
standardization Custom solutions 1990199520002005 Open Grid
Services Arch Real standards Multiple implementations Web services,
etc. Managed shared virtual systems Research Globus Toolkit Defacto
standard Single implementation Internet standards Developing Grid
Standards 2010
- Slide 29
- 29 foster@mcs.anl.gov ARGONNE CHICAGO Open Grid Services
Architecture Adopt service-oriented architecture u Key to
virtualization, discovery, composition, local-remote transparency +
Standard service description & access u Leverage industry
standard Web services + Distributed service management protocols u
A component model for Web services = A framework for creating,
managing, & delivering interoperable services The Physiology of
the Grid: An Open Grid Services Architecture for Distributed
Systems Integration, Foster, Kesselman, Nick, Tuecke, 2002
- Slide 30
- 30 foster@mcs.anl.gov ARGONNE CHICAGO Web Services: Basic
Functionality OGSA Open Grid Services Architecture (GGF, OASIS,
W3C) OGSI: Interface to Grid Infrastructure Applications in Problem
Domain X Compute, Data & Storage Resources Distributed
Application & Integration Technology for Problem Domain X Users
in Problem Domain X Virtual Integration Architecture Generic
Virtual Service Access and Integration Layer - Structured Data
Integration Structured Data Access Structured Data
RelationalXMLSemi-structured Transformation Registry Job Submission
Data TransportResource Usage Banking BrokeringWorkflow
Authorisation
- Slide 31
- 31 foster@mcs.anl.gov ARGONNE CHICAGO Open Grid Services
Infrastructure Implementation Service data element Other standard
interfaces: factory, notification, collections Hosting
environment/runtime (C, J2EE,.NET, ) Service data element Service
data element GridService (required) Data access Lifetime management
Explicit destruction Soft-state lifetime Introspection: What port
types? What policy? What state? Client Grid Service Handle Grid
Service Reference handle resolution
- Slide 32
- 32 foster@mcs.anl.gov ARGONNE CHICAGO OGSA Standards: Status l
WSDL 1.2 in progress in W3C l OGSI 1.0 completed July 2003 in GGF l
WS-Agreement draft in GGF l WS-Management drafts in OASIS l OGSA
Data Access & Integration in GGF l WS-Security specifications
in OASIS l SAML & XACML in OASIS l WS-Addressing status unclear
l Other work in progress
- Slide 33
- 33 foster@mcs.anl.gov ARGONNE CHICAGO Standards: Summary l Grid
and Web Services are merging u Grid is an aggressive use case of
Web Services l Web Services standards landscape is in flux u OGSI/A
will need to evolve with it u Uncertain status of security &
policy standards continues to be a big source of concern l Grid
services standards landscape heating up l W3C, OASIS, GGF are key
standards orgs l Open source software important for adoption
- Slide 34
- 34 foster@mcs.anl.gov ARGONNE CHICAGO OGSA Status:
Implementations l Globus Toolkit v3: Linux for the Grid u Open
source middleware, commercial support u A range of computation
& data management, registry, and security functions l Some nice
announced OGSI-based products u IBM, Avaki, Platform, Sun, NEC, HP,
UD, Entropia, DataSynapse, Insors, Oracle, etc. u Read the fine
print: Intent is to use OGSI- based products, OGSA-compliant
software, Embraces fundamental OGSA concepts
- Slide 35
- 35 foster@mcs.anl.gov ARGONNE CHICAGO Globus Toolkit History:
An Unreliable Memoir DARPA, NSF begin funding Grid work NASA
initiates Information Power Grid Globus Project wins Global
Information Infrastructure Award MPICH-G released The Grid:
Blueprint for a New Computing Infrastructure published GT 1.0.0
Released Early Application Successes Reported GT 1.1.1 Released GT
1.1.2 Released GT 1.1.3 Released NSF & European Commission
Initiate Many New Grid Projects GT 1.1.4 and MPICH-G2 Released
Anatomy of the Grid Paper Released First EuroGlobus Conference Held
in Lecce Significant Commercial Interest in Grids NSF GRIDS Center
Initiated GT 2.0 beta Released Physiology of the Grid Paper
Released GT 2.0 Released GT 2.2 Released Only Globus.Org; not
downloads from: NMI UK eScience EU DataGrid IBM Platform etc.
- Slide 36
- 36 foster@mcs.anl.gov ARGONNE CHICAGO HP Grid Product Offerings
l Globus Toolkit for HP-UX, Tru64 Unix, Linux u Globus Toolkit 2.4
u Tested and optimized for HP platforms u Commercial support! l HP
Utility Data Center (UDC) u Data Center operation &
construction services u Service-centric design u Intent is to
construct UDC products using OGSI-compliant services l Enterprise
Grid consulting
- Slide 37
- 37 foster@mcs.anl.gov ARGONNE CHICAGO HP and Planetary
Computing shared, traded resources value clusters grid-enabled
systems programmable data center virtual data center Open VMS
clusters, TruCluster, MC ServiceGuard Tru64, HP-UX, Linux switch
fabric computestorage UDC computing utility or GRID today
- Slide 38
- 38 foster@mcs.anl.gov ARGONNE CHICAGO Pop Quiz: The Grid Is a)A
collaboration & resource sharing infrastructure for scientific
applications b)A distributed service integration and management
technology c)A disruptive technology that enables a virtualized,
collaborative, distributed world d)An open source technology &
community e)A marketing slogan f)All of the above
- Slide 39
- 39 foster@mcs.anl.gov ARGONNE CHICAGO Summary: Why You Should
Care 1) Grid is a disruptive technology [Vision] u It ushers in a
virtualized, collaborative, distributed world 2) Grid addresses
pain points now [Reality] u Grids are built not bought, but are
delivering real benefits in commercial settings 3) An open Grid is
to your advantage [Future] u Standards are being defined now that
will determine the future of this technology
- Slide 40
- 40 foster@mcs.anl.gov ARGONNE CHICAGO To Learn More Deep tech
& strategic info Jan 20-23, 2004, San Fran www.globusworld.org
Working, research groups Three meetings a year www.ggf.org Weekly
email & web newsletter www.gridtoday.com Commercial conf &
expo May 24-26, Philadelphia www.gridtoday.com 2nd Edition
www.mkp.com/grid2 Background information www.mcs.anl.gov/ ~foster
Become a Globus corporate affiliate