Amazon Web Services

Overview and introduction

About myself

• Head of Development Estonia @ Insparx GmbH• 24 years programming, 10 years leading teams• Startup experience – running on AWS / Azure clouds

• Linked - http://ee.linkedin.com/in/konstantinroot/ • Twitter - @konstantin_root

Overview

• What is AWS?

• What people usually mean when talking about AWS and why they are wrong

• Difference between AWS and VPS?• Mix AWS with other providers

When to use (or not)

• When to use?– Startups– For quick tests / experiments

• Requires deployment automation!

– Your business requires compliance with secure regulations from specific industry

• When not to use?– You have predictable load and want to have best

$$$/resources

Competition

• Azure• Rackspace• VPS – Linode, DigitalOcean, Softlayer, Hertzner, etc.

Understanding AWS locations

• 8 regions, 44 edge locations

Availability zones

• What are they?• Why they are

required?• How they are

connected?

Services availability

• Availability zone could go down– Usually related to EBS, ELB, EC2

problems

• Whole region could go down• If you require 24/7 with tight SLAs

– use multiple regions

Pricing structure

• Free tier and Startup plan• Each region have slightly different prices• US East is biggest, oldest and cheapest

region• Expect to pay for every

transaction/resource usage– Good predictability of expenses– Better service vs “all you can eat” providers

AWS overview

• Compute & Networking – EC2, EMR (Hadoop), Elastic Load Balancer & Auto Scaling, VPC & Direct Connect, Route 53

• Storage & CDN – EBS, S3, Glacier, CloudFront• Databases – RDS (MySQL, Oracle, SQL Server), DynamoDB,

SimpleDB, Redshift, ElastiCache (Memcached, Redis)• Application services – SQS (queue), SES (email), SNS (push

notifications), FPS (payment), CloudSearch, SWF (workflow), Elastic Transcoder

• Management & Deployment – IAM, CloudWatch, Elastic Beanstalk, OpsWorks (Chef), CloudFormation, CloudHSM (Hardware Security Module)

Services control

• Web Console• CLI– Azure minute of shame

• API• 3rd party services and

products

Payment

• Estonia credit cards are OK

• Expenses report• Understand and use free

tier

Support

• Forums - free– Azure minute of shame ;)

• Emails• Commercial support

Where to start?

Understand IAM

• Access to AWS resources• Root account vs user vs services• Groups, Users, Roles• What is role?– Permissions to access resource API calls

• 2 way authorization– Free application for mobile phones

@Live demo

Access from application

• Default option – use Access Key ID / Secret Access Key combo

• EC2 IAM Role– Avoid hardcoding credentials into

application configuration– Retrieve temporary token

programmatically

Amazon Elastic Compute Cloud

What is EC2

• Xen virtualization• Limited resources per server– Elastic Compute Unit (ECU) ~=

1-1.2Ghz 2007 Opteron/Xeon

• You share resources with others on server– Noisy neighborhood problem– Local disk usage problems

Instance types

• Details @ http://aws.amazon.com/ec2/instance-types/

• Micro instance – tricky one• EBS traffic counts in network limit

– that’s what EBS optimized instances for

Purchase options

• Understanding On-Demand vs Reserved vs Spot

• Different Reserved tiers• Reserved instances marketplace and ability to

change your purchases• What is Spot instance and when to use it?– Protect against price fluctuation– Don’t forget to bid on higher end instances with

same price

Security groups

• Understanding them• Best practices– Azure minute of

shame ;)

@Live demo

EC2 image options

• AWS Marketplace• All major OS are presented by vendors– Ubuntu selector -

http://cloud-images.ubuntu.com/locator/ec2/

• Create your own• EBS vs S3 for AMI

@Live demo

Storage options

• Local storage vs EBS• Local storage– SATA or SSD– No persistence on reboot

• EBS ~100 IOPS– Provisioned IOPS up to 4000 (16KB

block)– You could RAID EBS volumes– Choose EBS optimized instances when

required

Permanent IP address

• Elastic IP• Allocate and (re)assign

to servers• Low cost ~$3.6• Default limit is 5

Elastic scaling• When to use it?• You could use your own solution• Azure year of shame

When not to use EC2

• Have CPU intensive tasks not tied closely with AWS? EC2 is bad $$$ investment vs other providers

• High memory instances are expensive• Email server hosting

File storage options:EBS vs S3 vs Glacier

EBS

• Speed is limited, but could increase it with $$$• Pay for everything

– Pay for provisioned storage: 1GB = $0.10 ($0.125 for provisioned IOPS)– 1 million I/O requests = $0.10

• Could create on the fly snapshots to S3– First one is full then incremental ones

• Part of recent outages was related to it

S3

• Unlimited amount of objects• High durability/availability• High speed within region – 5-15MB/sec• Pay per operation and size only

– Modifications: 1K requests = $0.005– Read: 1K requests = $0.004– Used storage: 1GB = $0.095– 1GB traffic OUT = $0.12 (region traffic free)

• No folders (they are emulated)• 3rd party tool for accessing data - CloudBerry

Glacier

• Use case – archive lots of data, but read them rarely

• Integration with S3 – move files to Glacier automatically

• Pricing– Storage: 1GB = $0.01– Upload / retrieve: 1k operations = $0.050– Traffic: 1GB OUT = $0.120 (region traffic free)– Read 5% of data free then pay for it

Other services

Database options

• Relational databases – MySQL, Oracle, SQL Server– No PostgreSQL

• NoSQL– DynamoDB – fast and great, but $$$– SimpleDB – don’t use unless you

understand perfectly your reasons

• Caching– Memcached– Redis

• Redshift – warehousing solution for TB/PB scale

DynamoDB in details• Great product with some limitations• High durability and quick operations (<10ms)

– http://www.dynaspeed.net/ • Unique pricing:

– Pay per provisioned capacity (1 unit = 1KB)• 10 write units = $0.0065/hour• 50 read units = $0.0065/hour

– Indexed storage after 100MB: $0.25/GB– Traffic OUT: $0.12/GB (region traffic free)– Great saving if purchased upfront in 5k read/write units– Free tier – 5 writes, 10 reads

Elastic Load Balancer

• Integrates with EC2 health checks• DNS failover integration with Route 53• Auto Scaling integration• SSL termination, session stickiness• $0.025/hour, $0.008/GB for traffic• Not as flexible as HAProxy, but OK in

most cases

CloudFront

• AWS version of CDN• Many edge locations in all regions• For static files uses S3 folder as the source

– Could invalidate static objects with some delay

• Could cache dynamic content (think of Varnish)

• Low pricing – you pay only for actual usage

Message queue

• Limitations:– Max 256KB size of message– Long poll, but with 20sec limit – Limited routing options

• Expensive for large amounts:– 1 million requests = $0.50– 1 request unit = 64KB– Traffic $0.12/GB

• Alternatives:– Hosted RabbitMQ, ActiveMQ, Redis, ZeroMQ– 3rd party solution – IronMQ (http://www.iron.io/mq)

Route 53

• DNS service• Scalable and with good API• Advanced capabilities– DNS failover using health checks– Zone apex mapping to ELB– Weighted Round Robin– Latency based routing

Blogs to follow

AWS team http://aws.typepad.com/aws/Azure team http://weblogs.asp.net/scottgu/default.aspx