SCALING ON AWS – FROM MVP TO 100 MILLION USERS

by: Muhammad Umair

Cloudifie: Cloud for Start-ups@cloudifie

Introduction

Startup Challenges & Limitations

Challenges: High Competition Rapid Growth Fast Paced Marketing

Limitations: Tight Budget Time Constraints Limited Resources

Solution

Change traditional model, use agile solutions

Fail forward, fail fast and fail cheap

Focus on Product/App, not Infrastructure

Cloud Computing

Model of computing which allows to use shared resources on demand.

IaaS Offers Computing Infrastructure, like VM,

Storage, Network etc.

PaaS Offers Computing Platform e.g. Execution

runtimes, database, web-servers

SaaS Offer Application Services like CRM, Email,

Monitoring etc

Why AWS

1. Develop, Test, Deploy and go Global using same automated infrastructure.

2. Elastic infrastructure, easily scale up and down in no-time.

3. Low infrastructure cost – instead of high fixed cost.

Infrastructure: Global Footprint

AWS Building Blocks

AWS Services (Grouped)

MVP: Single Tier Architecture

Full application stack on single EC2 instance Application Database Web server Management etc

Elastic IP address Amazon Route53 for

DNS

1-100 request served concurrently

www.cloudifie.com

More Users, What to Do?

Scale Vertically Scale up for higher

capacity Scale down for lower

capacity

Scale from AWS Micro to Large computing instances

Easy to change instance types but will hit an endpoint eventually.

t2.micro

c4.8xlarge

Instance Types1. General Purpose

T2: instances with ability to burst above baseline performance M4: largest generation of GP, balanced compute, memory &

network

2. Compute Optimized C3: compute optimized for web-servers, batch processing &

analytics C4: largest in CO, best for high performance science &

engineering apps

3. Memory Optimized R3: recommended for high performance databases, distributed

memory cache, in-memory analytics & SAP, SharePoint etc.

4. Storage Optimized I2 & D2: Good for high IO NoSQL DBs & MPP data warehousing,

file systems, MapReduce/Hadoop, log processing

Vertical Scale Limitation

Can serve user up-to certain extend, like 100-1000 users.

Everything in same basket

No redundancy No failover

www.cloudifie.com

Two-Tiers Architecture

Solution?

Two-Tiers Architecture

Separately host application and

database.

Application on EC2 Self Managed

Database on EC2

or Use Amazon

Database Services

www.cloudifie.com

Benefits of Two-Tiers Architecture

Tailor instances for each tier Instance Type Storage

Easily manageable Scale each tier separately Enhanced Security

Separate security policy for each tier

DB behind the VPC subnet

www.cloudifie.com

Amazon Database Services

Rational Database Service (RDS) Managed relational database service Supports MySQL, Postgres, Oracle, SQL Server, and

Amazon Aurora ( up to 30 million select per minute)

Amazon DynamoDB: Fast, predictable, highly-scalable Amazon NoSQL data

store

Amazon Redshift Simple & cost effective, fully managed petabyte scale data warehouse

Which Database Service?

Choose Relational Database Service until there is clear use case for NoSQL, Why? Feature-rich well worn technology Mature & big community offering loads of help, code

and huge knowledgebase Can serve up to million users easily Patterns for read heavy scalability

Use NoSQL When

Super low latency is key

Dataset is very large > 5 TB

Dataset is highly non-relational or schema-less constructs

Application is data intensive ( ingestion of thousands of recodes/Sec)

EC2-RDS Architecture

Everything in NOT same basket More Secure Highly Scalable Easily Manageable (Managed

DB)

But still No redundancy No failover

www.cloudifie.com

Amazon Elastic Load Balancer

Allows seamless Horizontal Scaling by distributing load evenly

Cross-zone load balancing Automatic health checks for

failover Session stickiness: Routes

request to same instance Publish usage matrices to

CloudWatch

Add Redundancy

Use Amazon Elastic Load Balancer

Address Failover by adding infrastructure Redundancy

Ensure Multi-AZ redundancy for High Availability

User > 1000

cloudifie.com

Need More Capacity?

Scale Horizontally or Scaling Out Increase capacity by adding more resources

Add more Servers in each AZ

Increase count of database instances

100k+ Users

cloudifie.com

Amazon Auto Scaling

Allows to scale EC2 fleet automatically

Save time & effort through Automation Define scaling up & down policies

Optimize cost by on demand scaling

Where we are now

Cross zones failover load balancing High availability Capacity to serve 100k+ users Layered structure Auto-scaling

Revisit AWS Building Blocks

AWS offer three very important tools for web applications

1. Simple Storage Service (S3) Web services interface for storage & retrieval Best for static contents like images, css, js etc.

2. ElasticCache Very fast, highly scalable and fully managed Memchached & Redis

3. CloudFront AWS CDN for static & dynamic content caching and delivery through edge locations

Let’s Refine Architecture

Refine application architecture Reduce load on web servers by shifting it

to other managed services Move 3-Teirs: Web, App & DB

Store static contents on S3 Reduce load from Database by caching

frequent queries to AWS ElasticCache Use CloudFront for the distribution of

Static & Dynamic content delivery from edges.

You are good to go for

500K+ Users

But with this COMPLEXITY

You would start to run into issues with speed, performance and infrastructure management.

Solution:

Ensure Monitoring, Analysis & Optimization

Use AWS Application Management Services

AWS Deployment & Management Services

Convenience Control

Abstract high level services More automation granularity

Divide & Rule

Service Oriented Architecture (SOA)

Move services into their own tiers/modules.

Treat each of these as separate pieces of infrastructure & SCALE them

independently.

Loose coupling sets you free

Looser the components are coupled, the bigger they SCALE

Independent components & black box design Decouple interactions User tested services with built-in redundancy & scaling

Controller A

Controller B

Tightly Coupled

Controller A Controller BQQLoosely Coupled

AWS Services

Amazon CloudSearch Amazon

SESAmazon SQS

Elastic Transcoder

Amazon SWFAmazon SNS

Next Big Move

SOA + Loose Coupling = Win WinShift functionality to NoSQL

Amazon SES

Amazon SQS

Amazon SNS

DynamoDB

Amazon CloudSearch

We are good to go for

Million+ Users

5-10 Million Users

Use Federation & Sharding for database Scaling

Federation: Splitting into multiple DBs based on functionality like Users DB, Products DB etc.

Sharding: Splitting one dataset across multiple hosts

User Shard ID

1234 A

1235 B

1236 C

1237 D

10-100 Million Users

Application Fine Tuning

More decoupling & SoA

Scale out from Multi-AZ to Multi-Regions

Build custom solutions

Thank You

so very much