Scaling HBase at Pinterest (Big Data Guru meetup 2014-01-22)
AWS Las Vegas meetup 5-31-16 building &scaling web apps - slide deck & labs
-
Upload
hatech-llc -
Category
Software
-
view
227 -
download
0
Transcript of AWS Las Vegas meetup 5-31-16 building &scaling web apps - slide deck & labs
1©2015-2016 hatech.io
Jon Hathaway, Founder & [email protected]
Welcome to DevOpsIt’s a culture that transcends business boundaries
2©2015-2016 hatech.io
2
Some HATech data points
We build securely for web-scale
We create self-learning environments
We live for automation
We love Microservices, Docker & Containers
● Headquartered in Las Vegas
● Presence in Reno, Malta (EU) & Serbia (SEE), 24/7 coverage
● Established 2015
● Grew from 2 - 8 staff in first 3 months (expected 20, yr end)
● Financed from day 1 by project income
● Core Services:
○ DevOps business transformation consultancy
○ Full-stack automation engineering
○ 24/7 global Managed Services support
○ DevOps Pipeline Manager
3©2015-2016 hatech.io
3
Accessing EC2 Instances with SSH
4©2015-2016 hatech.io
Overview of Topics
Theory:Setting up SSH in AWSAWS VPC, AZ and Region Best PracticesLoadbalancing, Autoscaling and Launch Configurations
PracticalCreate a self installing single node LAMP StackCreate a Multi Node, Autoscaling Web FarmCreate a Multi Node, Autoscaling LAMP Cluster
DemoUsing the Cloudformation Designer
5©2015-2016 hatech.io
Accessing EC2 Instances using SSH
Both Linux and Windows Instances use SSH- Linux = secure login- Windows = SSH Key used to encrypt the Administrator Password
Two ‘components’- Private Key (Keep Safe)- Public Key (Share with others)
ALL Keys in AWS and on Linux must be OpenSSH Format- Windows -> Putty -> Create -> Copy and Paste Public Key
Naming format- <username>_id_rsa- <username>_id_rsa.pub
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html
6©2015-2016 hatech.io
SSH - Generating KeyPair on Windows
- Install Putty MSI - http://www.putty.org/
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html
7©2015-2016 hatech.io
SSH - Generating KeyPair on Linux / OSX
8©2015-2016 hatech.io
SSH - Copying Private key to remote server
- Add Public Key in OpenSSH Format to authorized_keys- Set permissions of authorized_keys so only they user can read the file
9©2015-2016 hatech.io
SSH - Adding Keys to AWS
- AWS Console -> EC2 -> Choose Region - Key Pairs -> ‘Import Key Pair’- Upload Public Key ONLY
10©2015-2016 hatech.io
SSH - Which User Name?
CentOS / Redhat / Arch -> ec2-user@<SERVER>
Debian / Ubuntu -> ubuntu@<SERVER>
11©2015-2016 hatech.io
SSH - Logging in to EC2 instance with Putty
12©2015-2016 hatech.io
SSH - Logging in to EC2 instance with OpenSSH
$ ssh ec2-user@<SERVER>
Or
$ ssh -i ~/.ssh/<Key Name> ec2-user@<SERVER>
13©2015-2016 hatech.io
13
VPC’s, Availability Zones and Regions
14©2015-2016 hatech.io
AWS Basics - Regions
15©2015-2016 hatech.io
AWS Basics - Availability Zones
16©2015-2016 hatech.io
AWS Basics - Virtual Private Cloud (Data Center)
● Virtual Data Center
● Keeps all Application Traffic Localized
● Logically Isolated
● Resources in VPC inherit the VPC benefits
17©2015-2016 hatech.io
AWS Basics - Good HA
● Web traffic is distributed ● Database traffic is distributed● AWS Region SLA is 99.95%● Any two AZ’s can fail
18©2015-2016 hatech.io
AWS Basics - Bad HA
● All web Traffic in single AZ● All DB Traffic in single AZ● Two Single Points of Failure● No SLA for a single AZ
19©2015-2016 hatech.io
19
Load balancing, AutoScale and Launch Configurations
20©2015-2016 hatech.io
Load Balancing - The Basics
● HTTP / HTTPS / TCP (no UDP)● ‘Listener’ - receives traffic● Supports Multiple Listeners per ELB● Port Mapping 1:1 ( TCP8080 -> TCP80 )● Health Check - Adds/Removes EC2● Works with other AWS components
○ Auto Scale Groups
21©2015-2016 hatech.io
Load Balancing - Internet Facing
● Create ELB● Create Healthcheck● Register Instances in ELB
Creates● Publicly IP DNS Record
name-1234567890.region.elb.amazonaws.com
Best Practice● Point ALIAS DNS Record Type to ELB● If not using Route53 use CNAME Type
22©2015-2016 hatech.io
Load Balancing - Internal Facing
● Create ELB○ Check the ‘Create an Internal Load Balancer’
● Create Healthcheck● Register Instances in ELB
Creates● Private IP DNS Record
internal-name-123456789.region.elb.amazonaws.com
Best Practice● Point ALIAS DNS Record Type to ELB● If not using Route53 use CNAME Type
23©2015-2016 hatech.io
Load Balancing - HTTPS
● SSL Certificate installed on ELB○ Manually○ AWS KMS
● Offload SSL to the ELB● Encrypted between Client and VPC● Web Servers can be clear text
24©2015-2016 hatech.io
Auto Scale Groups
● Creates a Group of EC2 Instances○ Max○ Min○ Desired
● Define the EC2 launch Configuration○ Type, AMI ID, Security Group, Subnet○ User Data ‘recipe’
● Scaling Plan○ How to scale○ When to scale
25©2015-2016 hatech.io
Launch Configurations
● Define what your EC2 instances are○ AMI ID○ Size/Type○ User Data○ Security Groups○ Subnets○ Availability Zones
● Ensures consistency
26©2015-2016 hatech.io
Instance Meta and User Data
Meta Data● Useful AWS information about ‘you’● What, Where, Who● Location● AMI , Type, Network Information
User Data● Inject Scripts into the startup of your instance
○ Executed on first boot○ Powershell, Bash etc
27©2015-2016 hatech.io
27
LAB 1
Create a Load Balanced EC2 Cluster
28©2015-2016 hatech.io
28
Cloudformation - Orchestrating AWS
29©2015-2016 hatech.io
Cloudformation
● Template to describe AWS Resources, their relationships and configuration● JSON Document
- Parameters- Defines the input variables to be used to configure the Resources
- Resources- The description of the AWS Resources and their configurations
- Outputs- Information learned during deployment that the user finds useful
30©2015-2016 hatech.io
Cloudformation - Create ‘Stack’
31©2015-2016 hatech.io
Cloudformation - Update ‘Stack’
32©2015-2016 hatech.io
Cloudformation - Simple Anatomy
{ "AWSTemplateFormatVersion" : "version date",
"Description" : "JSON string",
"Parameters" : { set of parameters },
"Resources" : { set of resources },
"Outputs" : { set of outputs }}
Parameters (optional)Specifies values that you can pass in to your template at runtime (when you create
or update a stack). You can refer to parameters in the Resources and Outputs
sections of the template.
Resources (required)Specifies the stack resources and their properties, such as an Amazon Elastic
Compute Cloud instance or an Amazon Simple Storage Service bucket. You can
refer to resources in the Resources and Outputs sections of the template.
Outputs (optional)Describes the values that are returned whenever you view your stack's
properties. For example, you can declare an output for an Amazon S3 bucket
name and then call the aws cloudformation describe-stacks AWS CLI
command to view the name.
33©2015-2016 hatech.io
33
LAB 2
How to automate a simple, single node LAMP Instance with custom packages
34©2015-2016 hatech.io
34
LAB 3
How to deploy a complete RDS based LAMP stack with AutoScaling
35©2015-2016 hatech.io
35
Demo
Using Cloudformation Designer
36©2015-2016 hatech.io
36
How do you get some HATech?
web: hatech.io
twitter: @hatechllc
linkedin: hatech-llc
HATech, LLC
+1 (702) 389 8160
10161 Park Run Drive
Suite 150
Las Vegas
NV, 89145
● Qualified leads receive free 2- hour senior consultant review session
● 10 day+ engagements receive 25% daily rate discount
● 25% off DevOps Discovery for Nevada-based businesses
CONNECT WITH US CONTACT