ETM 555 Supplementary Lecture Notes Version 5. / 2011 Contents:

ETM 555 1

ETM 555Supplementary Lecture NotesVersion 5. / 2011

Contents:Part 1: Hardware/Software Systems, Grid / Cloud Computing

ETM 555 2

Part 1Hardware/Software

Systems, Grid Computing

ETM 555 3

Hardware Hardware Parallel/Distributed Processing Parallel/Distributed Processing High Performance ComputingHigh Performance ComputingTop 500 listTop 500 listGrid computing Grid computing

ETM 555

picture of ASCI WHITE,

the most powerfulcomputer in the world (2001)

ETM 555 4

Von Neumann Architecture

CPU RAM Device Device

• sequential computer

BUS

ETM 555 5

History of Computer Architecture

• 4 Generations (identified by logic technology)

1. Tubes

2. Transistors

3. Integrated Circuits

4. VLSI (very large scale integration)

ETM 555 6

PERFORMANCE TRENDS

ETM 555 7

PERFORMANCE TRENDS

• Traditional mainframe/supercomputer performance 25% increase per year

• But … microprocessor performance 50% increase per year since mid 80’s.

ETM 555 8

Moore’s Law

• “Transistor density doubles every 18 months”

• Moore is co-founder of Intel.

• 60 % increase per year

• Exponential growth

• PC costs decline.

• PCs are building bricks of all future systems.

ETM 555 9

VLSI Generation

ETM 555 10

Bit Level Parallelism(upto mid 80’s)

• 4 bit microprocessors replaced by 8 bit, 16 bit, 32 bit etc.

• doubling the width of the datapath reduces the number of cycles required to perform a full 32-bit operation

• mid 80’s reap benefits of this kind of parallelism (full 32-bit word operations combined with the use of caches)

ETM 555 11

Instruction Level Parallelism(mid 80’s to mid 90’s)

• Basic steps in instruction processing (instruction decode, integer arithmetic, address calculations, could be performed in a single cycle)

• Pipelined instruction processing

• Reduced instruction set (RISC)

• Superscalar execution

• Branch prediction

ETM 555 12

Thread/Process Level Parallelism(mid 90’s to present)

• On average control transfers occur roughly once in five instructions, so exploiting instruction level parallelism at a larger scale is not possible

• Use multiple independent “threads” or processes

• Concurrently running threads, processes

ETM 555 13

Evolution of the Infrastructure

• Electronic Accounting Machine Era: 1930-1950

• General Purpose Mainframe and Minicomputer Era: 1959-Present

• Personal Computer Era: 1981 – Present

• Client/Server Era: 1983 – Present

• Enterprise Internet Computing Era: 1992- Present

ETM 555 14

Memory Hierarchy

Registers

Cache

Real Memory

Disk

CD

Fast

Slow

ETM 555 15

Sequential vs Parallel Processing

• physical limits reached

• easy to program

• expensive supercomputers

• “raw” power unlimited

• more memory, multiple cache

• made up of COTS, so cheap

• difficult to program

ETM 555 16

Amdahl’s Law• The serial percentage of a program is fixed. So speed-up obtained by

employing parallel processing is bounded.

• Lead to pessimism in in the parallel processing community and prevented development of parallel machines for a long time.

Speedup = 1

s + 1-s

P

• In the limit:

Spedup = 1/s s

ETM 555 17

Gustafson’s Law• Serial percentage is dependent on the number of

processors/input.

• Demonstrated achieving more than 1000 fold speedup using 1024 processors.

• Justified parallel processing

ETM 555 18

Grand Challenge Applications

• Important scientific & engineering problems identified by U.S. High Performance Computing & Communications Program (’92)

ETM 555 19

Flynn’s Taxonomy

• classifies computer architectures according to:

1. Number of instruction streams it can process at a time

2. Number of data elements on which it can operate simultaneously

Data Streams

Single Multiple

Single

Multiple

Instruction Streams

SISD SIMD

MIMDMISD

ETM 555 20

SPMD Model (Single Program Multiple Data)

• Each processor executes the same program asynchronously

• Synchronization takes place only when processors need to exchange data

• SPMD is extension of SIMD (relax synchronized instruction execution)

• SPMD is restriction of MIMD (use only one source/object)

ETM 555 21

Parallel Processing Terminology• Embarassingly Parallel:

-applications which are trivial to parallelize

-large amounts of independent computation

-Little communication

•Data Parallelism:

-model of parallel computing in which a single operation can be applied to all data elements simultaneously

-amenable to SIMD or SPMD style of computation

•Control Parallelism:

-many different operations may be executed concurrently

-require MIMD/SPMD style of computation

ETM 555 22

Parallel Processing Terminology• Scalability:

- If the size of problem is increased, number of processors that can be effectively used can be increased (i.e. there is no limit on parallelism).

- Cost of scalable algorithm grows slowly as input size and the number of processors are increased.

- Data parallel algorithms are more scalable than control parallel alorithms

• Granularity:

- fine grain machines: employ massive number of weak processors each with small memory

- coarse grain machines: smaller number of powerful processors each with large amounts of memory

ETM 555 23

Shared Memory Machines

Shared Address Space

process(thread)

process(thread)

process(thread)

process(thread)

process(thread)

•Memory is globally shared, therefore processes (threads) see single address space

•Coordination of accesses to locations done by use of locks provided by thread libraries

•Example Machines: Sequent, Alliant, SUN Ultra, Dual/Quad Board Pentium PC

•Example Thread Libraries: POSIX threads, Linux threads.

ETM 555 24

Shared Memory Machines• can be classified as:

-UMA: uniform memory access

-NUMA: nonuniform memory access

based on the amount of time a processor takes to access local and global memory.

Inter-connectionnetwork/or BUS

Inter-connection

network

Inter-connection

network

P

P

..

P

M

M

..

M

P

M

P

M

..

P

M

P

M

P

M

..

P

M

M

M

M

..

M

(a)(b) (c)

ETM 555 25

Distributed Memory Machines

Network

process

process

process

process

processM

M

M

M

M

•Each processor has its own local memory (not directly accessible by others)

•Processors communicate by passing messages to each other

•Example Machines: IBM SP2, Intel Paragon, COWs (cluster of workstations)

•Example Message Passing Libraries: PVM, MPI

ETM 555 26

Beowulf Clusters

•Use COTS, ordinary PCs and networking equipment

•Has the best price/performance ratio

PC cluster

ETM 555 27

Multi-Core Computing

• A multi-core microprocessor is one which combines two or more

independent processors into a single package, often a single integrated circuit.

• A dual-core device contains only two independent microprocessors.

ETM 555 28

Comparison of Different Architectures

CPU State

CacheExecution

unit

Single Core Architecture

ETM 555 29


CPU State

CacheExecution

unit

Multiprocessor

CPU State

CacheExecution

unit

ETM 555 30


CPU State

CacheExecution

unit

Hyper-Threading Technology

CPU State

ETM 555 31


CPU State

CacheExecution

unit

Multi-Core Architecture

CPU State

CacheExecution

unit

ETM 555 32


CPU State

Executionunit

Multi-Core Architecture with Shared Cache

CPU State

Cache

Executionunit

ETM 555 33


Multi-Core with Hyper-Threading Technology

CPU State

CacheExecution

unit

CPU State CPU State

CacheExecution

unit

CPU State

ETM 555

Graphics Processing Units (GPUs)

• GPU devotes more transistors to data processing

ETM 555 35

Hillis’ Thesis ’85 (back to the future !)

Piece of

silicon

Sequential computer Parallel computer

• proposed “The Connection Machine” with massive number of processors each with small memory operating in SIMD mode.

• CM-1, CM-2 machines from Thinking Machines Corporation (TMC)were examples of this architecture with 32K-128K processors.

ETM 555

Floating Point Operations for the CPU and the GPU

ETM 555

Memory Bandwidth for the CPU and the GPU

ETM 555

NVIDIA GPU Supports Various Languages or Application Programming Interfaces

ETM 555

Automatic Scalability

A multithreaded program is partitioned into blocks of threads that execute independently from each other, so that a GPU with more cores will automatically execute the program in less time than a GPU with fewer cores.

ETM 555

Grid of Thread Blocks

ETM 555

Memory Hierarchy

ETM 555

GPU Programming Model

• Heterogeneous Programming • Serial code executes on the host while parallel code executes on the device.

ETM 555 43

ETM 555 44

Top 500 Most Powerful Computers List

• http://www.top500.org/list/2011/06

ETM 555 45

Grid Computing• provide access to computing power and various resources

just like accessing electrical power from electrical grid

• Allows coupling of geographically distributed resources

• Provide inexpensive access to resources irrespective of their physical location or access point

• Internet & dedicated networks can be used to interconnect distributed computational resources and present them as a single unified resource

• Resources: supercomputers, clusters, storage systems, data resources, special devices

ETM 555 46

Grid Computing

• the GRID is, in effect, a set of software tools, which when combined with hardware, would let users tap processing power off the Internet as easily as the electrical power can be drawn from the electricty grid.

• Examples of Grids:

-TeraGrid (USA)

-EGEE Grid (Europe)

- TR-Grid (Turkey)

ETM 555

GRID COMPUTING

Power Grid Compute Grid

ETM 555 48

ArcheologyAstronomyAstrophysicsCivil ProtectionComp. ChemistryEarth SciencesFinanceFusionGeophysicsHigh Energy PhysicsLife SciencesMultimediaMaterial Sciences…

>250 sites48 countries>50,000 CPUs>20 PetaBytes>10,000 users>150 VOs>150,000 jobs/day

ETM 555 49

Virtualization

• Virtualization is abstraction of computer resources. • Make a single physical resource such as a server, an operating system, an application, or storage device appear to function as

multiple logical resources• It may also mean making multiple physical resources such as storage devices or servers appear as a single logical resource • Server virtualization enables companies to run more than one operating system at the same time on a single machine

ETM 555 50

Advantages of Virtualization

• Most servers run at just 10-15 %capacity – virtualization can increase server utilization to 70% or higher. • Higher utilization means fewer computers are required to process the same amount of work. Fewer machines means less

power consumption.• Legacy applications can also be run on older versions of an operating system• Other advantages: easier administration, fault tolerancy, security

ETM 555 51

VMware Virtual Platform

Virtual machine 1

Apps 1

OS 1

X86, motherboarddisks, display, net ..

Virtual machine 2

Apps 2

OS 2

X86, motherboarddisks, display, net ..

VMware Virtual Platform

X86, motherboard, disks, display, net ..

Virtual machines

Real machines

•VMware is now 40 billion dollar company !!

ETM 555 52

Cloud Computing

•Style of computing in which IT-related capabilities are provided “as a service”,allowing users to access technology-enabled services from the Internet ("in the cloud") without knowledge of, expertise with, or control over the technology infrastructure that supports them.

•General concept that incorporates software as a service (SaaS), Web 2.0 and other recent, well-known technology trends, in which the common theme is reliance on the Internet for satisfying the computing needs of the users.

ETM 555

Cloud Computing

• Virtualisation provides separation between infrastructure and user runtime environment

• Users specify virtual images as their deployment building blocks

• Pay-as-you-go allows users to use the service when they want and only pay for what they use

• Elasticity of the cloud allows users to start simple and explore more complex deployment over time

• Simple interface allows easy integration with existing systems

53

ETM 555

Cloud: Unique Features

• Ease of use

– REST and HTTP(S)

• Runtime environment

– Hardware virtualisation

– Gives users full control

• Elasticity

– Pay-as-you-go

– Cloud providers can buy hardware faster than you!

54

ETM 555

Cloud computing is about much more than technological capabilities.

Technology is the mechanism, but, as in any shift in business, the driver is economics.

Nicholas Carr,The author of “The Big Switch”

ETM 555

We want to pay only for what we useAnd we want to control it accurately.

Better Economics

ETM 555

Facing New Challenges

• Complexity of modern IT infrastructures: physical servers, virtual machines, clusters, Grids, geographical distribution

• Cost of electricity• Credit crunch• Further pressures to reduce costs• Openness to the acceptable security concept

ETM 555

http://www....

ETM 555

ETM 555

The 70/30 SwitchThe 70/30 Switch

ETM 555

Finding Solutions

• Improving utilisation rates through market based algorithms for resource allocation

• Accessing external infrastructures on-demand• Using a single management platform for all computing

resources

ETM 555

Cloud vs Grid

From the customers/end users point of view

They are the same

ETM 555

Grid/cloud market structure

Hardware(owned)

Hardware(service)

Middleware

Applications

Customer

Net

wor

k

ETM 555

The Grid/Cloud

Advantages

• Lower cost

• Access to larger infrastructure

– Faster calculations

– More storage

• Speed

– Faster calculations

– Easier provisioning

Disadvantages

• Very complicated

• Security

• Lack of confidence

– Trust

– Compatibility

ETM 555

Improving Utilization

Hardware Servers: (-) low utilisation rates, scalability problems

Uti

lisati

on

Virtual Servers: (+) improved utilisation rates, better scalability, easy disaster recovery(-) increased number of servers to manage, incompatible virtualization platforms

Enterprise/Departmental Grid:(+) improves utilisation rates of physical servers, enables collaboration(-) limited scalability, lack of interoperability between vendors, limited efficiency of policy based mechanisms

100%

0%

Grid

Cloud Computing:(+) no need to own hardware, shared access, improved utilisation through pay-as-you-use(-) incompatible platforms, ‘fair price’ is dubious to users

ETM 555

Issue Classic Grid Computing Cloud computing

Why we need it?(The Problem)

To enable the R&D community to achieve its research goals in reasonable time.Computation over large data sets, or of paralleizable compute-intensive applications.

Reduce IT costs.On-demand scalability for all applications, including research, development and business applications.

Main Target Market

First - AcademiaSecond – certain industries

Mainly Industry

Business Model – Where the money comes from?

AcademiaSponsor-based (Mainly government money).

Industry paysInternal Implementations.

Hosted by commercial companies, paid-for by users. Based on the economies of scale and expertise. Only pay for what you need, when you need it: (On- Demand + Pay per Use).

Grid and Clouds

ETM 555

Competition

HardwareOperatingSystem

Virtualisation

EnterpriseGrid

EnterpriseCloud

Cloud

ConstellationTechnologies

Incompatible Standards

Creates pools

of resources

Higher

utilisation

rates

Incompatible Standards

Interfaces

Interfaces and Market Mechanisms

Key differentiators:• Open source – no vendor lock-in• Scalability

ETM 555

ETM 555

Challenges

Security and Trust

Customer SLA – compare Cost/Performance

Dynamic VM migration – Unique Universal IP

Clouds Interoperability

Data Protection & Recovery

Standards: Security

Management Tools

Integration with Internal Infrastructure

Small compact economical applications

Cost/Performance prediction and measurement

Keep it Transparent and Simple

ETM 555

"The future is about having a platform in the cloud," Microsoft Chief Steve Ballmer said of the trend in a July, 2008 e-mail to employees.

Cloud Market

ETM 555

“By 2012 ,80 percent of Fortune 1000 companies

will pay for some cloud computing service ,

And

30 percent of them will pay for

cloud computing infrastructure.”

Gartner, 2008

Cloud Market

ETM 555

• - CIOs -> Do more with Less (Energy costs / Recession will boost it)

- Lower cost for Scalability

- Enterprise IT budget - Spending 80% on MAINTENANCE

- In average, we utilize only 15% of our computing resources capacity

- Peak Times economy

- The Enterprise IT is not its core business

- Psychology of Internet/Cloud trust (SalesForce, Gmail, Internet banking, etc.)

- Ideal for Developers

Why Now? (Economy)

ETM 555

Cost savings, leveraging economies of scale

Pay only for what you use

Resource flexibility

Rapid prototyping and market testing

Increased speed to market

Improved service levels and availability

Self-service deployment

Reduce lock-in and switching costs

Why Now? (Benefits)

ETM 555

Clouds Types

VM Based (EC2, GoGrid)

Storage Based (EMC, S3)

Customers Applications based (Google)

Cloud Applications based (SalesForce)

Grid Computing/HPC Applications

Mobile Clouds (iPhone UI, WEB APPS)

Private Clouds

Cloud of Clouds

ETM 555

Summary

Cloud Computing - The New IT Economy

Pay-per-Use for On-Demand Scalability

All major vendors are investing in Clouds

Cloud Trading Market will evolve

VM will be mobile across clouds

Mobile phones (iPhone) cloud users

International implications (Access to Data)

ETM 555

Example Cloud: Amazon Web Services

• EC2 (Elastic Computing Cloud) is the computing service of Amazon

– Based on hardware virtualisation

– Users request virtual machine instances, pointing to an image (public or private) stored in S3

– Users have full control over each instance (e.g. access as root, if required)

– Requests can be issued via SOAP and REST

76

ETM 555


•S3 (Simple Storage Service) is a service for storing and accessing data on the Amazon cloud– From a user’s point-of-view, S3 is independent

from the other Amazon services– Data is built in a hierarchical fashion, grouped in

buckets (i.e. containers) and objects– Data is accessible via various protocols

•Elastic Block Store– Locally mounted storage– Highly available

77

ETM 555


• Other AWS services:

– SQS (Simple Queue Service)

– SimpleDB

– Billing services: DevPay

– Elastic IP (Static IPs for Dynamic Cloud Computing)

– Multiple Locations

78

ETM 555


• Pricing information

http://aws.amazon.com/ec2/

79

ETM 555

EC2 – “Google of the Clouds”

According to Vogels (Amazon CTO), 370,000 developers have registered for Amazon Web Services since their start in 2002, and the company now spends more bandwidth on the developers than it does on e-commerce. http://www.theregister.co.uk/2008/06/26/amazon_trumpets_web_services/

In the last two months of 2007 usage of Amazon Web Services grew by 40%

$131 million revenues in Q1 from AWS 60,000 customersThe majority of usage comes from banks, pharmaceuticals and other large corporations

ETM 555 81

ETM 555 82

Hadoop

ETM 555 83

• IDC estimate put the size of the “digital universe” at - 0.18 zettabytes in 2006-forecasting a tenfold growth by 2011 to 1.8 zettabytes

• The New York Stock Exchange generates about one terabyte of new trade data per day

• Facebook hosts approximately 10 billion photos, taking up one petabyte of storage.

• The Internet Archive stores around 2 petabytes of data, and is growing at a rate of 20 terabytes per month.

• The Large Hadron Collider near Geneva, Switzerland, produce about 15 petabytes of data per year.

Data Explosion

ETM 555 84

•Common A set of components and interfaces for distributed filesystems and general I/O (serialization, Java RPC, persistent data structures).

•Avro A serialization system for efficient, cross-language RPC, and persistent data storage.

•MapReduce A distributed data processing model and execution environment that runs on large clusters of commodity machines.

•HDFS A Distributed filesystem that runs on large clusters of commodity machines.

•Pig A data flow language and execution environment for exploring very large datasets. Pig runs on HDFS and MapReduce clusters.

Hadoop Projects

ETM 555 85

•Hive A distributed data warehouse. Hive manages data stored in HDFS and provides a query language based on SQL (and which is translated by the runtime engine to MapReduce jobs) for querying the data.

•Hbase A distributed, column-oriented database. HBase uses HDFS for its underlying storage, and supports both batch-style computations using MapReduce and point queries (random reads).

•ZooKeeper A distributed, highly available coordination service. ZooKeeper provides primitives such as distributed locks that can be used for building distributed applications.

•Sqoop A tool for efficiently moving data between relational databases and HDFS.

Hadoop Projects

ETM 555 86

RDBMS Compared to MapReduce

• MapReduce can be seen as a complement to an RDBMS

• MapReduce is a good fit for problems that need to analyze the whole dataset, in a batch fashion, particularly for ad hoc analysis.

• An RDBMS is good for point queries or updates, where the dataset has been indexed to deliver low-latency retrieval and update times of a relatively small amount of data.

• MapReduce suits applications where the data is written once, and read many times, whereas a relational database is good for datasets that are continually updated.

ETM 555 87

RDBMS Compared to MapReduce

ETM 555 88

Amazon’s Cloud Load Balancing Service

• Elastic Load Balancing automatically distributes incoming application traffic across multiple Amazon EC2 instances.• http://docs.amazonwebservices.com/ElasticLoadBalancing/latest/DeveloperGuide/

ETM 555 89

Dynamic Load Balancing Web-Server Systems

Server1

Server2

Server3

Server4

ServerM

Client1

Client2

Client3

Client4

ClientN

ETM 555 90


• Client-Based Approach

– Web clients, if they are aware of the Web-server system’s replicated servers, can actively route requests

– After receiving a request, the Web client selects a node of the web server cluster and, after resolving the address mapping, submits the request to the selected node, which is then responsible for responding to the client.

– Approaches:

• Netscape: wwwi.netscape.com

• Via smart client, e.g java applet

ETM 555 91


• DNS-Based Approach

– The cluster DNS—the authoritative DNS server for the distributed Web system’s nodes—translates the symbolic site name (URL) to the IP address of one server

ETM 555 92


• Dispatcher-Based Approach

ETM 555 93


• Server-Based Approach

ETM 555 Supplementary Lecture Notes Version 5. / 2011 Contents:

Documents

Transcript of ETM 555 Supplementary Lecture Notes Version 5. / 2011 Contents: