GridComputing

Ahmad Ibrahim IS086924Mohemmad Omar IS087987

What is Grid Computing?

A A Grid Computing systemGrid Computing system is a is a collection of distributed collection of distributed computing resource available computing resource available over a local or wide area over a local or wide area network, that appears to an network, that appears to an end user or application as end user or application as one one large virtuallarge virtual computing computing systemsystem

Is an approach to distributed Is an approach to distributed computing that spans not computing that spans not only locations but also only locations but also organizationsorganizations, , machine machine architecturesarchitectures, and , and software software boundariesboundaries

Characteristics Allows the integrated, collaborative use of computers, Allows the integrated, collaborative use of computers,

networks, databases, and scientific instruments owned networks, databases, and scientific instruments owned and managed by multiple organizationsand managed by multiple organizations

Facilitates the solution of Facilitates the solution of computational problemscomputational problems– large-scalelarge-scale– complexcomplex– multi-institutionalmulti-institutional– multidisciplinary multidisciplinary – Large data storage and/or computational Large data storage and/or computational

requirementsrequirements

Grid computing started out as the simultaneous Grid computing started out as the simultaneous application of the resources of many networked application of the resources of many networked computers to a computers to a singlesingle problemproblem..

Characteristics

For many years, computational grids have been used to For many years, computational grids have been used to solve large-scale problems in solve large-scale problems in sciencescience and and engineeringengineering

CurrentlyCurrently used in the following fields : used in the following fields :– Medical research : protein folding, cancer drug Medical research : protein folding, cancer drug

developmentdevelopment– Astronomy : SETI data analysisAstronomy : SETI data analysis– Mathematical / Statistical problemsMathematical / Statistical problems– Climate modelsClimate models

Grid computing is beginning to enter the Grid computing is beginning to enter the commercialcommercial worldworld– Financial analysisFinancial analysis– ForecastingForecasting– Enterprise GridsEnterprise Grids

Characteristics Involves sharing of heterogeneous resources:Involves sharing of heterogeneous resources:

– Hardware platformsHardware platforms– Hardware / software architecturesHardware / software architectures– Computer languagesComputer languages– Different placesDifferent places– Different administrative domainsDifferent administrative domains

Main Classifications

Grid Types:Grid Types:– Computational GridsComputational Grids : computers set aside resources : computers set aside resources

allocated to number-crunch data or provide coverage allocated to number-crunch data or provide coverage for CPU-intensive workloadsfor CPU-intensive workloads

– Data GridsData Grids : share data resources and storage : share data resources and storage capacity, unified interface for all data repositories in capacity, unified interface for all data repositories in an organization, through which data can be queried, an organization, through which data can be queried, managed, and securedmanaged, and secured

– Scavenging GridsScavenging Grids : used to locate and exploit : used to locate and exploit machine cycles on idle servers and desktops for use machine cycles on idle servers and desktops for use in resource-intensive tasksin resource-intensive tasks

Internal vs External GridsInternal vs External Grids– External gridsExternal grids : usually geographically-distributed, : usually geographically-distributed,

non-profit research effortsnon-profit research efforts– Internal gridsInternal grids : large commercial enterprise with : large commercial enterprise with

complex problems who aim to fully exploit their complex problems who aim to fully exploit their unused internal computing powerunused internal computing power

System usersScientists and engineersUsing computation toAccomplish Lab missions

Intelligent InterfaceA knowledge-based environment that offers users guidance on complex computing tasks

MiddlewareSoftware tools that enable interaction among users, applications, and system resources

Grid Operating SystemThe software which coordinates the interplay of computers, networks, and storage

HardwareHeterogeneous collection of high-performance computer hardware and software resources

NetworkingThe hardware and software that permits communication among distributed users and computer resources

Mass StorageA collection of devices and software that allows temporary and long-term archival storage of information

SoftwareSoftware applications and components for computational problems

How does it work?

Grid Computing vs the Internet

The Internet is about getting computers to The Internet is about getting computers to talktalk togethertogether

Grid computing is about getting computers to Grid computing is about getting computers to workwork together together

• The Internet is a network ofThe Internet is a network of communicationcommunication

• Grid computing is a network ofGrid computing is a network of computationcomputation: provides tools : provides tools andand protocols for resource sharing protocols for resource sharing of aof a variety of IT resources variety of IT resources

Grid Computing vs.Clusters / Distributed Computing

Clusters and Distributed computing systems Clusters and Distributed computing systems requirerequire– physical proximityphysical proximity– Operational homogeneityOperational homogeneity

Grids are geographically distributed, Grids are geographically distributed, heterogeneousheterogeneous

Clusters and Distributed computing systems Clusters and Distributed computing systems are based on the Data Center / Computer are based on the Data Center / Computer Center computersCenter computers

Grids include end-user computers as wellGrids include end-user computers as well

Electric Power Grid

Electric Power:Electric Power:– Household electrical devices simply plug to an electric Household electrical devices simply plug to an electric

outletoutlet– Use Use onlyonly the electric power you need the electric power you need– Pay Pay onlyonly for the electric power you used for the electric power you used

Computers – typical :Computers – typical :– You pay for a computer with certain computing power You pay for a computer with certain computing power

(CPU)(CPU)– If you use less computing power than what your computer If you use less computing power than what your computer

provides, provides, you still pay for allyou still pay for all of it of it– If you use more computing power than what your If you use more computing power than what your

computer provides, computer provides, you have to buy a better computeryou have to buy a better computer

Computers – Grid :Computers – Grid :– Multiple computers (Multiple computers (including yoursincluding yours) share computing ) share computing

resources and make up a virtual computerresources and make up a virtual computer– You only use the computing power you needYou only use the computing power you need

Applications

Distributed Distributed supercomputing / supercomputing / computational sciencecomputational science

High – capacity / High – capacity / throughput computing: throughput computing: large-scale simulation, large-scale simulation, chip design, and chip design, and parameter studiesparameter studies

Content sharing: for Content sharing: for example, sharing digital example, sharing digital content among peerscontent among peers

Remote software access / Remote software access / rentingrenting

Applications

Data-intensive Data-intensive computing: drug design, computing: drug design, particle physics, stock particle physics, stock predictionprediction

On-demand, real-time On-demand, real-time computing: medical computing: medical instrumentation, mission instrumentation, mission critical initiativescritical initiatives

Collaborative computing Collaborative computing (e-science, e-(e-science, e-engineering), engineering), collaborative design, data collaborative design, data explorationexploration

Three Uses of Grid Computing

Compute grids Data grids Collaborative grids

Types of Grids

• Cluster GridBeowulf clusters• Enterprise GridDepartmental clusters, servers and PC network• Utility GridAccess resources over internet on demand

14

Government

• Small number of users• Couple small numbers of high-end

resources• Goals:

– Provide “strategic computing reserve” for crisis management

– Support collaborative investigations of scientific and engineering problems

• Need to integrate diverse resources and balance diversity of competing interests

15

Materials Science Collaborators• Scientists operating a variety of

instruments (electron microscopes, particle accelerators, X-ray sources) for characterization of materials

• Highly distributed and fluid community

• Sharing of instruments, archives, software, computers

• Virtual Grid – strong focus and narrow goals– Dynamic membership, decentralized, sharing

resources16

Benefits

Permits sharing of resources throughout an Permits sharing of resources throughout an organization, or among organizationsorganization, or among organizations

Make effective use of Make effective use of underused underused computing computing resourcesresources

Provide access to remote databases and Provide access to remote databases and softwaresoftware

Reduce significantly the number of servers Reduce significantly the number of servers needed (25-75%)needed (25-75%)

Allow Allow on-demand aggregation of resourceson-demand aggregation of resources at at multiple sitesmultiple sites

Reduce execution time for large-scale data Reduce execution time for large-scale data processing applicationsprocessing applications

Benefits

Provide load sharing across a Provide load sharing across a set of platformsset of platforms

Provide Provide fault tolerancefault tolerance

Take advantage of time-zone and random Take advantage of time-zone and random diversitydiversity

Provide flexibility to meet unexpected Provide flexibility to meet unexpected emergency demands: can rent external emergency demands: can rent external resources for a required period instead of resources for a required period instead of buying additional capacitybuying additional capacity

Virtual data centersVirtual data centers

Disadvantages Proprietary approachesProprietary approaches

– Leading vendors (HP, Sun, IBM, MS, Oracle, etc.) Leading vendors (HP, Sun, IBM, MS, Oracle, etc.) have proprietary, have proprietary, incompatibleincompatible approaches, approaches,

– which which defeats the purpose defeats the purpose of Grid computingof Grid computing

Business case not always easy to sell to upper managementBusiness case not always easy to sell to upper management– Need to present a business case based on economics, efficiency, Need to present a business case based on economics, efficiency,

not technical detailsnot technical details

Vendors need to show how their software effectively Vendors need to show how their software effectively manages a grid environmentmanages a grid environment

SecuritySecurity– Confidentiality, Integrity, Access to resources, dataConfidentiality, Integrity, Access to resources, data

Performance monitoringPerformance monitoring– Zoning mechanisms to ensure applications competing for Zoning mechanisms to ensure applications competing for

resources do not affect each otherresources do not affect each other

Evolution of Grids

Early StageEarly Stage1990s1990s

Second StageSecond StageEarly MidEarly Mid2000s 2000s2000s 2000s

Third StageThird StageLate 2000sLate 2000s

PerformancePerformanceand QoSand QoS

PersonalPersonalDevicesDevices

SMPs orSMPs orSuperSupercomputerscomputers

Local ClusterLocal ClusterComputingComputing

LocalLocalData Data GridsGrids

EnterpriseEnterpriseCluster / GridCluster / Grid

Partner GridsPartner Grids

Global GridGlobal Grid

[source: Minoli, Daniel,[source: Minoli, Daniel, A Networking Approach to Grid Computing, A Networking Approach to Grid Computing, 2005]2005]

Enterprise Grids Example : AstraZeneca PLC Grids Data GridData Grid

– Connects R&D databases from sites in UK, Sweden, and Connects R&D databases from sites in UK, Sweden, and USAUSA

– Significant savings in finding informationSignificant savings in finding information– Efficiency gains due to shortening the time R&D or design Efficiency gains due to shortening the time R&D or design

staff needs to find informationstaff needs to find information– Large investment in broadband links to connect data Large investment in broadband links to connect data

centers in different countriescenters in different countries

Enterprise GridEnterprise Grid– Processing power shared between R&D sites in UK, Processing power shared between R&D sites in UK,

Sweden, and USASweden, and USA– Efficiency due to processing power sharing, plus access to Efficiency due to processing power sharing, plus access to

datadata– Savings on R&D time, time to marketSavings on R&D time, time to market– Permits more efficient collaboration between sitesPermits more efficient collaboration between sites– Significant investment in security, and in high-performance Significant investment in security, and in high-performance

broadband linksbroadband links

Some Examples of GridsGrid NameGrid Name SponsorSponsor PurposePurpose

BlueGridBlueGrid IBMIBM IBM computation R&DIBM computation R&D

DISCOMDISCOM Sandia National LabsSandia National Labs Defense researchDefense research

DOE Science GridDOE Science Grid DOE Office of ScienceDOE Office of Science Scientific researchScientific research

European Union DataGridEuropean Union DataGrid European UnionEuropean Union Scientific researchScientific research

EuroGrid GRIPEuroGrid GRIP European UnionEuropean Union Computation R&DComputation R&D

Globus ProjectGlobus Project DARPA, NASA, Msoft, DARPA, NASA, Msoft, othersothers

Grid tech researchGrid tech research

GridLabGridLab European UnionEuropean Union Grid tech researchGrid tech research

Grid Research IntegrationGrid Research Integration National Science National Science FoundationFoundation

Grid middleware Grid middleware developmtdevelopmt

Intern. Data Grid LabIntern. Data Grid Lab National Science National Science FoundationFoundation

International large scale International large scale grid tech researchgrid tech research

Information Power GridInformation Power Grid NASANASA Aerospace researchAerospace research

Earthquake Eng. Earthquake Eng. SimulationsSimulations

National Science National Science FoundationFoundation

Earthquake engineeringEarthquake engineering

Particle Physics Data GridParticle Physics Data Grid DOE ScienceDOE Science High-energy physics High-energy physics researchresearch

TeraGridTeraGrid National Science National Science FoundationFoundation

Link major US Link major US universitiesuniversities

UK Grid Support CenterUK Grid Support Center UK eScienceUK eScience Grid projects in UKGrid projects in UK

TeraGrid

(www.teragrid.com)(www.teragrid.com) Completed in September 2004Completed in September 2004 Massively parallel supercomputer clustersMassively parallel supercomputer clusters 40 teraflops40 teraflops of computing power of computing power 2 petabytes2 petabytes of rotating storage of rotating storage Connected network of US supercomputing Connected network of US supercomputing

centers (currently 8, and growing)centers (currently 8, and growing) Each of the four original sites operates a Linux Each of the four original sites operates a Linux

cluster, interconnected by means of a 10-30 cluster, interconnected by means of a 10-30 Gigabit/sec dedicated optical networkGigabit/sec dedicated optical network

Other Grid ProjectsOther Grid Projects

Other Grid Projects