Cloud Talks EB Sep 20

8/8/2019 Cloud Talks EB Sep 20

1/42

Cloud Computing

Development


2/42

Cloud Computing

Development

Shallow Introduction


3/42

Introduction


4/42

What is the cloud computing

Is it computing while in flight?

Image Courtesy SevensHeaven.nl


5/42


Is it computing while in flight?

NO

Image Courtesy SevensHeaven.nl


6/42


What is it about then?


7/42



Cloud computing isconsumption of computing

resources without worryingabout specifics.


8/42



As well as ability to add orremove resources according

to the demand.


9/42



Similar to the power grid andtelephone network.


10/42



Similar to the power grid andtelephone network.


11/42

How does it work?

Consumer signs up for the service.(Same as if you get a mobile phone plan)

Consumer uses services according to their needs

Provider sends the bill at the end of the cycle

Consumer pays


12/42

Provider Models

Software As A Service

SAAS

EmailCRM

Office Apps


13/42

Provider Models


SAAS

EmailCRM

Office Apps

Platform As A Service

PAAS

Application ServersDatabases

Middleware


14/42

Provider Models


SAAS

EmailCRM

Office Apps


PAAS

Application ServersDatabases

Middleware

Infrastructure As AService

IAAS

Bare Hardware(Sort of )


15/42

Providers


SAAS

Google (GMail)Salesforce

Microsoft (Office Live)


16/42

Providers


SAAS




PAAS

Google App Engine

Heroku / Engine Yard (Rails)

Windows Azure (.NET)


17/42

Providers


SAAS




PAAS

Google App EngineHeroku / Engine Yard (Rails)

Windows Azure (.NET)

Infrastructure As AService

IAAS

Amazon AWSRackspace

GoGrid


18/42

Provider: Windows Azure

Platform as a service

Windows based

Storage provided through blob storage, drives,SQL Azure

State is stored and propagated with Queues and

Tables

Integrated with Visual Studio

Eclipse plug-in for PHP


19/42

Slide courtesy Vlad Vinogradsky from Microsoft


20/42


21/42

Provider: Rackspace

Infrastructure as a service

Very Basic just a few Linux or Windows images

Provides storage with CloudFilesVery Cheap

Open source API

Relatively New


22/42

Provider: Amazon AWS

Oldest on the market

Many services / Images / Third party providers

Provides computation through EC2 / EMR Provides state / storage through S3, SQS, RDS,SimpleDB

Multiple APIs


23/42

Sample Prices

AmazonCompute $0.10+ VM/HrStorage $0.15+ GB/Month

$0.15+ GB/XFer

RackspaceCompute $0.02+ VM/HrStorage $0.15+ GB/Month

$0.22+ GB/XFer

MicrosoftCompute $0.12 VM/HrStorage $0.15 GB/moBandwidh $0.15 GB/XFer


24/42

Development


25/42

Practical Considerations

Cloud Development is slightly differentfrom traditional in house model.


26/42

Practical Considerations

Cloud Development is slightly differentfrom traditional in house model.

Everything is virtualized (most of the time) Everything is distributed

Per instance reliability is much lower Overall reliability is much higher


27/42

Cloud Programming Model


28/42

Cloud Programming Model

Compute and Interfacenodes are not reliable,they can crash anddisappear at any time.

Storage and State arereliable and heavilydistributed.

At any time we can start

more compute orinterface nodes and shutthem down when demandsubsides.


29/42

Cloud Programming Model on Azure

Compute : Worker Nodes

State: Tables / Queues / SQL

Storage: SQL / Tables /Blobs / Drives

Client Inteface: Web Nodes


30/42

Cloud Programming Model on AWS

Compute : EC2 Instances

State: S3 / Queues /SimpleDB / RDS

Storage: S3 / SimpleDB / RDS

Client Inteface: S3 / EC2 /CloudFront


31/42

AWS Details: S3

S3 = Simple Storage Service

Guaranteed to be reliable

Simple {Key, Value} storage

Keys are stored within buckets

Values could be as large as 5GB

Default Storage Mechanism for AWS


32/42

AWS Details: Simple DB

Schema less database

Main storage unit is domain ( similar to table )

Each record can have many attributes, new attributes could

be added at any time

Similar to LISP / Scheme attributes

Can query domain for records containing particular

attributeNo Joins / Unions with other domains

Eventual Consistency


33/42

AWS Details: RDS

RDS = Relational Data Storage

MySQL in a cluster mode

Preferred to simply running DB server within instance(ask me why for details)


34/42

AWS Details: SQS

SQS = Simple Queue System

Massively scalable

Allows to put message in the queue and retrieve later on

Retrieving the message hides it from the other users

When message is processed it is deleted from the queue

If message is not deleted before the timeout it is returnedback


35/42

AWS Details: EC2

EC2 = Elastic Compute Cloud

Allows to run arbitrary virtual machinesProvided they are compatible with Amazons modified Xen

Kernels and Startup Disks are stored in S3

Also have large local storage

Machines are not exactly like physical machines

Local storage is not persistentWhen machine is shut down all local data disappears.

Hardware TCP [No packet layer / No Broadcast ]

Can launch many copies of the machine at the same time

Lots of preconfigured machines


36/42

AWS Details: Other Services

EMR = Elastic Map ReduceLets run Hadoop jobs on EC2

CloudFrontContent Delivery Network

ELB = Elastic Load Balancer

EBS = Elastic Block StorageS3 backed persistent storage

Public Data Sets - Lots of publicly available dataCensus ( 1980 , 1990, 2000 ), Wikipedia logs, Freebasedumps, Genetic and Chemistry data


37/42

Starting Up

Amazon Account

Credentials KeyID : SecretKey X509 Ceriticate


38/42

Helpful Tools

S3 Fox - Firefox extension for browsing S3

Elastic Fox - Firefox extension for operating EC2 Transmit - Mac utility for S3 ($) Right Scale - Web based platform for managing

everything ( Free / $ )


39/42


40/42

Demo


41/42

DemoRunning Hadoop on EC2


42/42

Questions

????

Cloud Talks EB Sep 20

Documents

Transcript of Cloud Talks EB Sep 20