Zürich - Amazon Web Services€¦ · Security Groups • Restrict access to instances by: –Port...

Zürich

Welcome and Introductions

Belegu RinonImsand Matthias

Agenda

Introduction: Rinon Belegu

Rinon Belegu

Digicomp (Technical-Lead AWS and Veeam)Legendary IT (Owner)

Certification:

AWS Mentor, AWS Trainer, Microsoft Certified Trainer, Veeam Trainer

Cloud experience:

Buildup diffrent Cloud-Solutions 2007 – 2012 (Switzerland)

Implementation of Private-,Public- and Hybrid-Cloud [email protected]

Matthias Imsand

Founder Amanox Solutions (CTO)

Dipl. Ing. FH Informatik

AWS Instructor

Cloud & Linux Module Teacher @ FFHS

Introduction: Matthias ImsandDatacenter and Cloud Solutions

Cloud JourneyHow to successfully move to the cloud

ENABLING

- Trainings

- Workshops

- Seminars

- Labs

ASSESSMENT

- Analysis

- Potential, Readiness

- Strategy

- KPI Definition

- Architecture

PROOF OF CONCEPT

- Pilot projects

- Awareness programs

- Test installation

- Lesson learned

DATA MIGRATION

- Storage concept

- Security

- Migration planning

- Identity / Accounting

APPLICATION MIGRATION

- Forklift / Re-architecting

- Containerization

- Automation

- CI / CD

OPERATION

- Supported by Amanox

- Managed by Amanox

- Operated by Amanox

AWS IoT connected Drone

AWS IoT connected Drone

https://youtu.be/o8kkp46IgvQ

https://youtu.be/o8kkp46IgvQ

AWS Basics

Amazon History

1994: Jeff Bezos incorporated the

company.

1995: Amazon.com launched its

online bookstore.

2005: Amazon

Publishing was

launched.

2006: Amazon

Web Services (AWS)

was launched.

2007: Kindle was

launched.

2011: Amazon

Fresh was launched.

2012: Amazon Game Studios was launched.

2013: Amazon Art was

launched.

2014: Amazon Prime

Now was launched.

2015: Amazon Home

Services and Amazon

Echo were launched.

Amazon Web Services (AWS)

ComputeMessaging

Mobile

App Services

Database

Networking

Development and

Management Tools

Payments

VPCOn-Demand Workforce

Analytics Content Delivery

Storage

Enable businesses and developers to

use web services to build scalable,

sophisticated applications.

2010

61

516

1,017

159

2012 2014 2016

AWS has been continually expanding its services to support virtually any cloud workload,

and it now has more than 90 services that range from compute, storage, networking,

database, analytics, application services, deployment, management, developer, mobile,

Internet of Things (IoT), Artificial Intelligence (AI), security, hybrid and enterprise

applications. AWS has launched a total of 1,017 new features and/or services year to

date* - for a total of 2,913 new features and/or services since inception in 2006.

AWS Pace of Innovation

1,950

AWS Direct

Connect

AWS Elastic Beanstalk

AWS GovCloud (US)

AWS CloudTrail

AWS CloudHSM

Amazon WorkSpaces

Amazon Kinesis

Amazon

AppStream

Amazon SNS

AWS Identity and Access

Management

Amazon Route 53

AWS Import/Export

Amazon SWF

Amazon Redshift

Amazon DynamoDB

Amazon CloudSearch

AWS Data

Pipeline

AWS Certificate Manager

AWS KMS

AWS Config

Amazon RDS

for Aurora

Amazon WorkDocs

AWS

Directory

Service

AWS CodeCommit

AWS CodePipeline

AWS Service

Catalog

Amazon CloudWatch Logs

Amazon EFS

Amazon API

Gateway

Amazon Machine

Learning

AWS Device Farm

AWS WAF

Elasticsearch Service

Amazon QuickSight

AWS Import/Export

Amazon RDS for MariaDB

Amazon Inspector

AWS IoT

Amazon EC2 Container

Registry

Amazon

ElastiCache

AWS

CloudFormation

Amazon Mobile

Analytics

AWS Mobile Hub

AWS Storage GatewayAWS OpsWorks

Amazon Elastic Transcoder

Amazon SES

Amazon EC2

Container Service

Amazon Cognito

AWS CodeDeploy

Amazon Glacier

Amazon WorkMail

AWS Lambda

Services and Features(February 1, 2016)

2,420

AWS Direct

Connect


AWS GovCloud (US)

AWS CloudTrail

AWS CloudHSM

Amazon WorkSpaces

Amazon Kinesis

Amazon

AppStream

Amazon SNS


Management

Amazon Route 53

AWS Import/Export

Amazon SWF

Amazon Redshift

Amazon DynamoDB

Amazon CloudSearch

AWS Data

Pipeline


AWS KMS

AWS Config

Amazon RDS

for Aurora

Amazon WorkDocs

AWS

Directory

Service

AWS CodeCommit

AWS CodePipeline

AWS Service

Catalog


Amazon EFS

Amazon API

Gateway

Amazon Machine

Learning

AWS Device Farm

AWS WAF


Amazon QuickSight

AWS Import/Export


Amazon Inspector

AWS IoT


Registry

Amazon

ElastiCache

AWS

CloudFormation

Amazon Mobile

Analytics

AWS Mobile Hub



Amazon SES

Amazon EC2

Container Service

Amazon Cognito

AWS CodeDeploy

Amazon Glacier

Amazon WorkMail

AWS Lambda

Services and Features(August 1, 2016)

2,913

AWS Direct

Connect


AWS GovCloud (US)

AWS CloudTrail

AWS CloudHSM

Amazon WorkSpaces

Amazon Kinesis

Amazon

AppStream

Amazon SNS


Management

Amazon Route 53

AWS Import/Export

Amazon SWF

Amazon Redshift

Amazon DynamoDB

Amazon CloudSearch

AWS Data

Pipeline


AWS KMS

AWS Config

Amazon RDS

for Aurora

Amazon WorkDocs

AWS

Directory

Service

AWS CodeCommit

AWS CodePipeline

AWS Service

Catalog


Amazon EFS

Amazon API

Gateway

Amazon Machine

Learning

AWS Device Farm

AWS WAF


Amazon QuickSight

AWS Import/Export


Amazon Inspector

AWS IoT


Registry

Amazon

ElastiCache

AWS

CloudFormation

Amazon Mobile

Analytics

AWS Mobile Hub



Amazon SES

Amazon EC2

Container Service

Amazon Cognito

AWS CodeDeploy

Amazon Glacier

Amazon WorkMail

AWS Lambda

Services and Features(January 1, 2017)

AWS Direct

Connect


Schema Conversion Tool

AWS Shield EFS

WorkSpaces

Amazon Lumberyard

Amazon

Pinpoint

AWS IoT

AWS Managed ServicesAmazon Route 53

AWS Import/Export

AWS OpsWorks for

Chef Automate

Redshift

Dynamo DB

Amazon Polly

AWS

Snowball

AWS Organizations

Device Farm

Amazon Config

Amazon RDS

for Aurora

WorkDocs

AWS

Snowball Edge

CodeCommit

AWS CodePipeline

AWS Service Catalog

CloudWatch Logs

Amazon Lex

AWS Greengrass

Amazon EC2

Systems Manager

AWS WAF

Amazon Appstream 2.0

Amazon

Athena

AWS Glue

Amazon Lightsail

Amazon Rekognition

AWS Step Functions

AWS Discovery

Services

AWS Certificate

Manager

Amazon

ElastiCache

Mobile

Analytics

AWS Mobile Hub


AWS Batch

Amazon Inspector

EC2

Container Service

Amazon Cognito

AWS CodeDeploy

AWS Personal Health Dashboard

AWS Snowmobile

Lambda

* As of 1 August 2017

AWS Codebuild

AWS X-Ray

Amazon QuickSight

Amazon Kinesis Firehose

Amazon

Workmail

Amazon Inspector

Machine Learning

3,567Services and Features

(August 1, 2017)

AWS Positioned as a Leader in the Gartner Magic Quadrant

for Cloud Infrastructure as a Service, Worldwide*

AWS is positioned

highest in execution

and furthest in

vision within the

Leaders Quadrant

*Gartner, Magic Quadrant for Cloud Infrastructure as a Service, Worldwide, Leong, Lydia, Petri, Gregor, Gill, Bob, Dorosh, Mike, August 32016

This graphic was published by Gartner, Inc. as part of a larger research document and should be evaluated in the context of the entire document. The Gartner document is available upon request from AWS :

http://www.gartner.com/doc/reprints?id=1-2G2O5FC&ct=150519&st=sb

Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications

consist of the opinions of Gartner's research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability

or fitness for a particular purpose.

http://www.gartner.com/doc/reprints?id=1-2G2O5FC&ct=150519&st=sb

AWS Core Infrastructure and Services

Security

Network

Security

Network

Security Groups NACLs Access Mgmt

VPCVPCEC2 “Classic”

“Public

”

ELBOn-Demand

Provision

Traditional Infrastructure Amazon Web Services

Servers

AMI Amazon EC2 InstancesOn-Premises Servers

SecuritySecurity Groups Network ACLs AWS IAMFirewalls ACLs Administrators

Storage and

DatabaseRDBMSDAS SAN NAS Amazon

EBSAmazon

EFSAmazon

S3

Amazon

RDS

NetworkingVPCELBRouter Network Pipeline Switch

Infrastructure Regions Edge LocationsAvailability Zones

Foundation

Services

Compute(Virtual, Auto-scaling and

Load Balancing)

Networking

ApplicationsVirtual

DesktopsCollaboration and Sharing

Platform

Services

AWS Cloud Computing

Databases

Relational

NoSQL

Caching

Analytics

Cluster

Computing

Real-time

Data

Warehouse

Data

Workflows

App Services

Queuing

Orchestration

App Streaming

Transcoding

Email

Search

Deployment and

Management

Containers

Dev/ops Tools

Resource Templates

Usage Tracking

Monitoring and Logs

Mobile Services

Identity

Sync

Mobile Analytics

Notifications

Storage(Object, Block and Archive)

AWS Regions

OREGON

N. CALIFORNIA

AWS GOVCLOUD

OHIO

N. VIRGINIA

MONTREAL

SÃO PAULO

LONDON

IRELAND

FRANKFURT

BEIJING

SEOUL

TOKYO

NINGXIA (Coming soon)

INDIA

SINGAPORE

SYDNEY

PARIS (Coming soon)

AZ - A AZ - B

Region

AZ - C

AWS Availability Zones

• Each Availability Zone is:– Made up of one or more data

centers.

– Designed for fault isolation.

– Interconnected with other Availability Zones using high-speed private links.

• You choose your Availability Zones.

• AWS recommends replicating across AZs for resiliency.

Availability Zone

AWS Edge Locations

16 AWS Regions

60+ AWS Edge Locations

Ways to access AWS

AWS Management ConsoleEasy-to-use graphical interface that supports majority of Amazon Web Services.

Command Line Interface (CLI)Access to services via discrete commands that can be issued from a Linux

command line, Linux shell script, Windows cmd prompt, Windows batch file,

or Windows PowerShell.

Software Development Kits (SDKs)Launch EC2 instances, configure networks, etc. from most major programming

languages (Python, Ruby, .NET, Java, etc.).

AWS Management Console

and Services

Demonstration

Cloud Deployment Models

Data Center Cloud

• Upfront capital expense

• Provision hardware and staff for

normal operations and disaster

recovery (DR)

• Limited experimentation and

reusability

• Available when needed

• Build up, tear down and reuse with

ease

• Reduced cost and planning for DR,

storage redundancy

• More independence, innovation

within the company

Hybrid Model

• Connect data center and cloud resources

Amazon Virtual Private Cloud (VPC)

• Provision a private, isolated virtual network on

the AWS cloud.

• Have complete control over your virtual

networking environment.Amazon

VPC

Amazon VPC Example

Virtual Private Cloud

AWS Cloud

Public Subnet Private Subnet VPN Only Subnet

DB ServerWeb Server

Customer

Network

R

Internet

App ServerVPC NAT

Gateway

Internet

Gateway

Web Server App Server DB Server

Virtual

Private

Gateway

Amazon Elastic Compute Cloud (EC2)

• Resizable compute capacity

• Complete control of your computing resources

• Reduced time required to obtain and boot new

server instancesAmazon

EC2

Launching an Amazon EC2 Instance

1. Determine the AWS Region in which you want to launch the

Amazon EC2 instance.

2. Launch an Amazon EC2 instance from a pre-configured Amazon

Machine Image (AMI).

3. Choose an instance type based on CPU, memory, storage, and

network requirements.

4. Configure network, IP address, security groups, storage volume,

tags, and key pair.

1) Determine the AWS Region

Determine the right region for

your services, applications, and

data based on these factors.

Proximity to

customers (latency)

Data governance,

legal requirements

Services available

within the region

Costs (vary by region)

2) Launch

Select an AMI based on:

• Region

• Operating system

• Architecture (32-bit or 64-bit)

• Launch permissions

• Storage for the root device

AMI

Instances

Instance

Launch

instances of any

type

Host computer

Host computer

3) Choose an instance type

M4

General

purpose

Compute

optimized

C4

Storage and I/O

optimized

I3

GPU- or FPGA-

enabled

Memory

optimized

D2

M3

X1

I2 HS1

C3

T2

R4

R3

F1

P2

G2

4) Configure

Network placement

and addressing

Block Storage

Ephemeral or EBS

AMI

Tenancy

Instance

Server Role Security groups

Key pairs

User data

Security Groups

• Restrict access to instances by:

– Port range

– IP range

– Security group or resource ID

• Instances can be associated with

multiple security groups.

• Allow data ingress and egress.

• Can be added/modified after launch.

Remote Access 22

Web

Servers

DB

NAT

Web Traffic 80

port 3306

Remote Access 22

User Data

Are supplied to initialize instances automatically and can be a• Linux script

• Windows batch or PowerShell scripts

Can install any software package, such as • Web servers

• Database servers

• Configuration management tools

Are executed by• Cloud-init on Linux

• EC2Config service on Windows

Runs once per instance-id by default

AWS EC2

Demonstration

Amazon Simple Storage Service (S3)

• Storage for the Internet

• Natively online, HTTP access

• Storage that allows you to store and retrieve any

amount of data, any time, from anywhere on the

web

• Highly scalable, reliable, fast and durable

Amazon S3

Amazon S3 Facts

• Can store an unlimited number of objects in a bucket

• Objects can be up to 5 TB; no bucket size limit

• Designed for 99.999999999% durability and 99.99% availability of objects over a given year

• Can use HTTP/S endpoints to store and retrieve any amount of data, at any time, from anywhere on the web

• Is highly scalable, reliable, fast, and inexpensive

• Can use optional server-side encryption using AWS or customer-managed provided client-side encryption

• Auditing is provided by access logs

• Provides standards-based REST and SOAP interfaces

AWS Storage Options: Block vs. Object Storage

Block Storage

Change one block (piece of the file)

that contains the character

Object Storage

Entire file must be updated

What if you want to change one character in a 1-GB file?

Common Use Scenarios

• Storage and backup

• Application file hosting

• Media hosting

• Software delivery

• Store AMIs and snapshots

Amazon S3 Concepts

To upload your data (photos, videos, documents, etc.):

1. Create a bucket in one of the AWS Regions.

2. Upload any number of objects to the bucket.

Amazon S3

[bucket name]

Preview2.mp4

Tokyo Region

(ap-northeast-1)

Bucket

Object

https://s3-ap-northeast-1.amazonaws.com/[bucket name]/Preview2.mp4

https://s3-ap-northeast-1.amazonaws.com/[bucket name]/

Region code Bucket name

Key

Amazon S3 Security

• You can control access to buckets and objects

with:– Access Control Lists (ACLs)

– Bucket policies

– Identity and Access Management (IAM) policies

• You can upload or download data to Amazon S3

via SSL encrypted endpoints.

• You can encrypt data using AWS SDKs.

Amazon S3 Versioning

• Protects from accidental overwrites and deletes with no performance penalty.

• Generates a new version with every upload.

• Allows easily retrieval of deleted objects or roll back to previous versions.

• Three states of an Amazon S3 bucket– Un-versioned (default)

– Versioning-enabled

– Versioning-suspendedVersioning Enabled

Key: photo.gif

ID: 121212

Key: photo.gif

ID: 111111

Amazon Glacier

• Long term low-cost archiving service

• Optimal for infrequently accessed data

• Designed for 99.999999999% durability

• Three to five hours’ retrieval time

• Less than $0.01 per GB/month (depending on

region)

Amazon

Glacier

S3 Lifecycle Policies

Amazon S3 lifecycle policies allow you to delete

or move objects based on age.

Amazon S3

Standard

Amazon S3

Standard -

Infrequent

Access

Delete

30 Days 60 Days 365 Days

Preview2.mp4 Preview2.mp4 Preview2.mp4

S3 Storage Class

Standard Standard - Infrequent

Access

Reduced Redundancy

StorageGlacier

Durability 99.999999999% 99.999999999% 99.99%99.999999999%

Availability 99.99% 99.9% 99.99% N/A

First Byte Latency ms ms ms 3-5h

Lifecycle Management

PoliciesYes Yes Yes Yes

AWS S3

Demonstration

Knowledge Check

Q: What AWS service would help support your web application by hosting static assets and storing user uploaded images and video off-instance?

Q: How would an EC2 instance find its private and public IP addresses?

Q: You want to deploy a new version of your web application. How do you trigger the user data to run again and update your app?

Q: True or False: S3 limits the total amount you can store.

False (There is a 5TB limit per object)

Retrieve the instance metadata. http://169.254.169.254/latest/meta-data/

Amazon S3

You don't. By default, user data is run once, when the instance first

boots.

Agenda

Security, Identities, and Access

Management

AWS Shared Responsibility Model

AWS Foundation Services

Compute Storage Database Networking

AWS Global Infrastructure Regions

Availability ZonesEdge Locations

Client-side Data Encryption

Server-side Data Encryption

Network Traffic Protection

Platform, Applications, Identity, and Access Management

Operating System, Network, and Firewall Configuration

Customer Applications & ContentC

ust

om

ers

Customers are

responsible for

security IN the cloud

AWS is responsible

for the security OF

the cloud

Physical Security

• 24/7 trained security staff

• AWS data centers in nondescript

and undisclosed facilities

• Two-factor authentication for

authorized staff

• Authorization for data center

access

Hardware, Software, and Network

• Automated change-control

process

• Bastion servers that record

all access attempts

• Firewall and other boundary

devices

• AWS monitoring tools

Certifications and Accreditations

ISO 9001, ISO 27001, ISO 27017, ISO 27018, IRAP (Australia), MLPS Level 3 (China),

MTCS Tier 3 Certification (Singapore) and more …

SSL Endpoints

VPC

Secure Transmission

Use secure endpoints

to establish secure

communication

sessions (HTTPS).

Instance Firewalls

Use security groups

to configure firewall

rules for instances.

SSL Endpoints Security Groups

Network Control

Use public and

private subnets,

NAT, and VPN

support in your

virtual private cloud

to create low-level

networking

constraints for

resource access.

SSL Endpoints

Security Groups

SSL Endpoints Security Groups

Instance Firewalls

Use security groups



VPC

Secure Transmission


to establish secure

communication

sessions (HTTPS).

Network Control

Use public and

private subnets,

NAT, and VPN

support in your


to create low-level

networking

constraints for

resource access.

HTTPPorts 80 and 443 only

open to the Internet

SSH/RDPEngineering staff have SSH/RDP

access to Bastion Host

AWS Multi-Tier Security Groups

Bastion

All other internet ports blocked by default

Amazon Virtual Private Cloud (VPC)

VPCSSL Endpoints Security Groups

Network Control

Use public and

private subnets,

NAT, and VPN

support in your


to create low-level

networking

constraints for

resource access.

Instance Firewalls

Use security groups



Secure Transmission


to establish secure

communication

sessions (HTTPS).

AWS Identity and Access Management (IAM)

AWS IAM

3

Manage federated users

and their permissions

2

Manage AWS IAM roles

and their permissions

1

Manage AWS IAM users

and their access

AWS IAM Authentication

• Authentication

• AWS Management Console– User Name and Password

IAM User

AWS IAM Authentication

• Authentication

• AWS CLI or SDK API– Access Key and Secret Key

Access Key ID: AKIAIOSFODNN7EXAMPLE

Secret Access Key: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

Java Python .NET

AWS SDK & APIAWS CLI

IAM User

AWS IAM User Management - Groups

User D

DevOps Group

User C

AWS Account

TestDev Group

User BUser A

AWS IAM Authorization

Authorization

• Policies:

– Are JSON documents to describe

permissions.

– Are assigned to users, groups or

roles.

IAM User IAM Group

IAM Roles

AWS IAM Policy Elements

{

"Version": "2012-10-17",

"Statement": [

{

"Sid": "Stmt1453690971587",

"Action": [

"ec2:Describe*",

"ec2:StartInstances",

"ec2:StopInstances”

],

"Effect": "Allow",

"Resource": "*",

"Condition": {

"IpAddress": {

"aws:SourceIp": "54.64.34.65/32”

}

}

},

{

"Sid": "Stmt1453690998327",

"Action": [

"s3:GetObject*”

],

"Effect": "Allow",

"Resource": "arn:aws:s3:::example_bucket/*”

}

]

}

IAM Policy

AWS IAM Policy Assignment

IAM UserIAM Group

Assigned Assigned

IAM Policy


IAM UserIAM Group

IAM Roles

Assigned Assigned

Assig

ned

IAM Policy

AWS IAM Roles

• An IAM role uses a policy.

• An IAM role has no associated credentials.

• IAM users, applications, and services may

assume IAM roles.

IAM Roles


IAM UserIAM Group

IAM Roles

Assigned Assigned

Assig

ned

IAM Policy

IAM User

Assumed Assumed

AWS Resources

Example: Application Access to AWS Resources

• Python application hosted on an Amazon EC2

Instance needs to interact with Amazon S3.

• AWS credentials are required:– Option 1: Store AWS Credentials on the Amazon EC2 instance.

– Option 2: Securely distribute AWS credentials to AWS Services

and Applications.

IAM Roles

AWS IAM Roles - Instance Profiles

Amazon EC2

App &EC2 MetaData Service

http://169.254.169.254/latest/meta-data/iam/security-credentials/rolename

Amazon S3

1

2

3

4

Create Instance

Sele

ct

IAM

Ro

le

Ap

plic

atio

n in

tera

cts

with

S3

AWS IAM Roles – Assume Role

IAM Restricted Policy

IAM User A-1

AWS Account A

IAM Admin RoleIAM Admin Policy

Assigned

Assume

Assigned

1

2

IAM User B-1

AWS Account B

Amazon S3

Assume

4

Access

53

Access

1

Temporary Security Credentials (AWS STS)

Use Cases

• Cross account access

• Federation

• Mobile Users

• Key rotation for Amazon EC2-based apps

Session

Access Key ID

Secret Access Key

Session Token

Expiration

Temporary Security Credentials

15 minutes to 36 hours

Application Authentication

AWS IAM Application

No Support No Support

OS

AWS IAM Authentication and Authorization

• Authentication

– AWS Management Console

• User Name and Password

– AWS CLI or SDK API

• Access Key and Secret Key

• Authorization

– Policies

IAM User IAM Group

IAM Roles

AWS IAM Best Practices

• Delete AWS account (root) access keys.

• Create individual IAM users.

• Use groups to assign permissions to IAM users.

• Grant least privilege.

• Configure a strong password policy.

• Enable MFA for privileged users.

AWS IAM Best Practices (cont.)

• Use roles for applications that run on Amazon EC2 instances.

• Delegate by using roles instead of by sharing credentials.

• Rotate credentials regularly.

• Remove unnecessary users and credentials.

• Use policy conditions for extra security.

• Monitor activity in your AWS account.

AWS CloudTrail

• Records AWS API calls for accounts.

• Delivers log files with information to an Amazon

S3 bucket.

• Makes calls using the AWS Management

Console, AWS SDKs, AWS CLI and higher-level

AWS services.

AWS CloudTrail Amazon S3 Bucket

Logs

Continuous Monitoring

• 24/7 monitoring to detect incidents.

• Industry-standard diagnostic procedures to drive

resolution during business-impacting events.

• Preventative maintenance for continued

operability of equipment.

Guess What This Is!

Guess What This Is!

This

To This

Instructor Demo

IAM

AWS Database

SQL and NoSQL Databases

SQL NoSQL

Data Storage Rows and Columns Key-Value

Schemas Fixed Dynamic

Querying Using SQL Focused on collection of

documents

Scalability Vertical Horizontal

ISBN Title Author Format

9182932465265 Cloud Computing

Concepts

Wilson,

Joe

Paperback

3142536475869 The Database

Guru

Gomez,

Maria

eBook

SQL NoSQL

{ISBN: 9182932465265,Title: “Cloud Computing Concepts”,Author: “Wilson, Joe”,Format: “Paperback”

}

Data Storage Considerations

• No one size fits all.

• Analyze your data requirements by considering:– Data formats

– Data size

– Query frequency

– Data access speed

– Data retention period

AWS Managed Database Services

Compute Storage

AWS Global Infrastructure

Database

App Services

Deployment and Administration

Networking

Amazon DynamoDB

Amazon ElastiCache

Amazon RDS

Amazon Redshift

AWS Database Migration Service

Amazon Relational Database Service (RDS)

• Cost-efficient and resizable capacity

• Manages time-consuming database

administration tasks

• Access to the full capabilities of Amazon

Aurora, MySQL, MariaDB, Microsoft SQL

Server, Oracle, and PostgreSQL databases

Amazon

RDS

Amazon RDS

• Simple and fast to deploy

• Manages common database administrative tasks

• Compatible with your applications

• Fast, predictable performance

• Simple and fast to scale

• Secure

• Cost-effective

DB Instances

• DB Instances are the basic building blocks

of Amazon RDS.

• They are an isolated database

environment in the cloud.

• They can contain multiple user-created

databases.

How Amazon RDS Backups Work

• Automatic Backups:– Restore your database to a

point in time.

– Are enabled by default.

– Let you choose a retention

period up to 35 days.

• Manual Snapshots:– Let you build a new

database instance from a

snapshot.

– Are initiated by the user.

– Persist until the user deletes

them.

– Are stored in Amazon S3.

Cross-Region Snapshots

• Are a copy of a database snapshot stored in a different AWS Region.

• Provide a backup for disaster recovery.

• Can be used as a basefor migration to a different region.

Amazon RDS Security

• Run your DB instance in an Amazon VPC.

• Use IAM policies to grant access to Amazon RDS resources.

• Use security groups.

• Use Secure Socket Layer (SSL) connections with DB instances (Amazon Aurora, Oracle, MySQL, MariaDB, PostgreSQL, Microsoft SQL Server).

• Use Amazon RDS encryption to secure your RDS instances and snapshots at rest.

• Use network encryption and transparent data encryption (TDE) with Oracle DB and Microsoft SQL Server instances.

• Use the security features of your DB engine to control access to your DB instance.

A Simple Application Architecture

Amazon RDS database

instance

Amazon EC2

Application Servers

Elastic Load Balancing

load balancer instance

DB snapshots in

Amazon S3

Multi-AZ RDS Deployment

• With Multi-AZ operation, your database is

synchronously replicated to another

Availability Zone in the same AWS Region.

• Failover to the standby automatically occurs in

case of master database failure.

• Planned maintenance is applied first to standby

databases.

A Resilient, Durable Application Architecture

Amazon RDS database instances:

Master and Multi-AZ standby

Application, in Amazon

EC2 instances


load balancer instance

DB snapshots in

Amazon S3

Amazon RDS Best Practices

• Monitor your memory, CPU, and storage usage.

• Use Multi-AZ deployments to automatically provision and maintain a synchronous standby in a different Availability Zone.

• Enable automatic backups.

• Set the backup window to occur during the daily low in WriteIOPS.

• To increase the I/O capacity of a DB instance:– Migrate to a DB instance class with high I/O capacity.

– Convert from standard storage to provisioned IOPS storage and use a DB instance class optimized for provisioned IOPS.

– Provision additional throughput capacity (if using provisioned IOPS storage).

• If your client application is caching the DNS data of your DB instances, set a TTL of less than 30 seconds.

• Test failover for your DB instance.

Amazon DynamoDB

• Allows you to store any amount of data with no limits.

• Provides fast, predictable performance using SSDs.

• Allows you to easily provision and change the request capacity needed for each table.

• Is a fully managed, NoSQL database service.

Amazon

DynamoDB

Provisioned Throughput

• You specify how much provisioned throughput

capacity you need for reads and writes.

• Amazon DynamoDB allocates the necessary

machine resources to meet your needs.

Supported Operations

• Query:– Query a table using the partition key and an optional sort key filter.

– If the table has a secondary index, query using its key.

– It is the most efficient way to retrieve items from a table or secondary index.

• Scan:– You can scan a table or secondary index.

– Scan reads every item – slower than querying.

• You can use conditional expressions in both Query and Scan operations.

Simple Application Architecture

Elastic Load

Balancing Amazon EC2

app instances

Clients

Amazon

DynamoDB

Business logic

Amazon RDS and Amazon DynamoDBFactors Relational (Amazon RDS) NoSQL (Amazon DynamoDB)

Application

Type

• Existing database apps

• Business process–centric apps

• New web-scale applications

• Large number of small writes and

reads

Application

Characteristics

• Relational data models,

transactions

• Complex queries, joins, and

updates

• Simple data models, transactions

• Range queries, simple updates

ScalingApplication or DBA–architected

(clustering, partitions, sharding)

Seamless, on-demand scaling based

on application requirements

QoS

• Performance–depends on data

model, indexing, query, and

storage optimization

• Reliability and availability

• Durability

• Performance–Automatically

optimized by the system

• Reliability and availability

• Durability

Database ConsiderationsIf You Need Consider Using

A relational database

service with minimal

administration

Amazon RDS

• Choice of Amazon Aurora, MySQL, MariaDB, Microsoft

SQL Server, Oracle, or PostgreSQL database engines

• Scale compute and storage

• Multi-AZ availability

A fast, highly scalable

NoSQL database

service

Amazon DynamoDB

• Extremely fast performance

• Seamless scalability and reliability

• Low cost

A database you can

manage on your own

Your choice of AMIs on Amazon EC2

and Amazon EBS that provide scale compute and

storage, complete control over instances, and more.

AWS Elasticity and Management

Tools Belegu Rinon

Imsand Matthias

November Traffic to Amazon.com

Provisioned capacity

November

76%

24%

The challenge is to efficiently

‘guess’ the unknown quantity of

how much compute capacity you

need.

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Benefits of elastic services

Better Cost

Management

Better

AvailabilityBetter Fault

Tolerance

Enable Scalability (1 of 2)

• Ensure that your architecture can handle changes in demand.

A key advantage of a cloud-based

infrastructure is how quickly you can

respond to changes in resource needs.

Anti-pattern

App servers

at full

capacity

Admin

launches

new server

New server

takes time

to launch

Users

prevented

from access


Enable Scalability (2 of 2)

Best practice

App servers

at alarm

threshold

Auto

Scaling is

alerted and

scales out

New server is

ready before

capacity is

reached

Users never

experience

interruption in

accessibility

Ensure that your architecture can handle changes in demand.

A key advantage of a cloud-based

infrastructure is how quickly you can

respond to changes in resource needs.


Vertical vs. Horizontal Scaling

• Vertical scaling

Scale up and down– Change in the specifications of

instances (more CPU, memory,

etc.)

Horizontal scaling

Scale in and out

• Change in the number of

instances (Add and remove

instances as needed)

small xlarge


Triad of Services

Latency

Utilization

CloudWatchAuto Scaling

Elastic Load

Balancing

Auto Scaling group

Execute AS

Policy


• Distributes traffic across multiple EC2 instances,

in multiple Availability Zones

• Supports health checks to detect unhealthy

Amazon EC2 instances

• Supports the routing and load balancing of

HTTP, HTTPS, SSL, and TCP traffic to Amazon

EC2 instances

Elastic Load

Balancing

Classic Load Balancer - How It Works

Register

instances

with your

load

balancer.

Availability Zone A Availability Zone B

load balancer

X

Target Group /mobile

Application Load Balancer – How It Works

Register instances as

targets in a target

group, and route

traffic to a target

group.

load balancer

Listener ListenerRule Rule Rule

Target Group Target Group /api

Target Target Target Target Target Target Target

Health

Check

Health

Check

Health

Check

ELB - Features

• Sticky Sessions

• Connection Draining

• Cross Zone Loadbalancing

• SSL Termination

• IPv6 Support

• Request Tracing (Header Injection)

• WAF Integration

• AWS Shield Integration (DDoS Protection)

Load Balancer Comparison

Classic Load Balancer

benefits include support

for:• EC2-Classic.

• VPC.

• TCP and SSL listeners.

• Sticky sessions.

• OSI Layer 4(network protocol level)

ALB benefits include

support for:• Path-based routing.

• Routing requests to multiple

services on a single EC2 instance.

• Containerized applications.

• Monitoring the health of each

service independently.

• OSI Layer 7(application level)

Amazon CloudWatch

• A monitoring service for AWS cloud resources and the applications you run on AWS

• Visibility into resource utilization, operational performance, and overall demand patterns

• Custom application-specific metrics of your own

• Accessible via AWS Management Console, APIs, SDK, or CLI

Amazon

CloudWatch

Amazon CloudWatch Facts

• Monitor other AWS resources

– View graphics and statistics

• Set Alarms

Amazon CloudWatch Architecture

AWS resources

that support

CloudWatch

Amazon

CloudWatch

Amazon

CloudWatch

Alarm

SNS Email

Notification

Auto Scaling

Available

Statistics

Statistics

Consumer

AWS Management

Console

CloudWatch Metrics

CPUUtilization

StatusCheckFailed

Custom

Application-

Specific Metrics

PageViewCount

CloudWatch Metrics Examples

CloudWatch Alarm Examples

Amazon

EC2

Amazon

RDS

If CPU utilization is > 60% for 5 minutes…

If number of simultaneous connections is > 10 for one minute…

If number of healthy hosts is < 5 for 10 minutes… Amazon

ELB

CloudWatch Alarms and Actions

CloudWatch alarms:

Measure a single

metric and perform

one or more actions

Stop, terminate, reboot, or

recover an Amazon EC2

instance

Scale an Auto Scaling

group in or out

Send message to Amazon

Simple Notification Service

(SNS)

Auto Scaling

• Scale your Amazon EC2 capacity

automatically

• Well-suited for applications that experience

variability in usage

• Available at no additional charge Auto

Scaling

Do Not Guess About Resource Needs

Build a flexible system that will react to changes in customer

demand and manage costs dynamically.

Availability Zone

Auto Scaling group

Availability Zone

Alarm

CloudWatch

Launch Configurations

• A launch configuration is a template that an Auto Scaling group uses to launch EC2 instances.

• When you create a launch configuration, you can specify:– AMI ID

– Instance type

– Key pair

– Security groups

– Block device mapping

– User data

Auto Scaling Groups

• Contain a collection of EC2 instances that share similar

characteristics.

• Instances in an Auto Scaling group are treated as a logical

grouping for the purpose of instance scaling and management.

Auto Scaling group

Minimum size

Desired capacity

Maximum size

Scale out as needed

Scaling Actions

• Dynamic Scaling– You can create a scaling policy that uses CloudWatch alarms to determine:

• When your Auto Scaling group should scale out.

• When your Auto Scaling group should scale in.

– You can use alarms to monitor:

• Any of the metrics that AWS services send to Amazon CloudWatch.

• Your own custom metrics.

• Manual Scaling– Scheduled Actions

– API Calls

Auto Scaling Basic Lifecycle

instances

Auto Scaling group

Scale Out

Amazon CloudWatchScheduled Event

Scale In

Amazon CloudWatchScheduled Event

Launch

Instance

Attach to Group

Detach from

GroupTerminate

Instance X

Triad of Services

Latency

Utilization

CloudWatchAuto Scaling

Elastic Load

Balancing

Auto Scaling group

Execute AS

Policy

AWS Trusted Advisor

• Best practice and recommendation engine.

• Provides AWS customers with performance and

security recommendations in four categories:

– Cost optimization

– Security

– Fault tolerance

– Performance improvement.

AWS Trusted

Advisor

Cost Optimization

• Amazon EC2 Reserved Instance Optimization

• Low-utilization Amazon EC2 Instances

• Idle load balancers

• Underutilized Amazon EBS volumes

• Unassociated Elastic IP addresses

• Amazon RDS idle DB instances

Security

• Security groups

• AWS IAM use

• Amazon S3 bucket permissions

• MFA on Root Account

• AWS IAM password policy

• Amazon RDS security group access risk

Fault Tolerance

• Amazon EBS Snapshots

• Load balancer optimization

• Auto Scaling Group Resources

• Amazon RDS Multi-AZ

• Amazon Route 53 name server delegations

• ELB connection draining

Performance Improvement

• High-utilization Amazon EC2 instances

• Service limits

• Large number of rules in EC2 security group

• Over-utilized Amazon EBS magnetic volumes

• Amazon EC2 to EBS throughput optimization

• Amazon CloudFront alternate domain names

AWS AutoScaling

Demonstration

The Challenges of Cloud Deployments

– Updating live servers.

– Rollouts across multiple

geographical locations.

– Ability to manage a rollback.

– Debugging deployments.

– Managing dependencies on

systems and subsystems.

Technologies for Automated, Repeatable Deployments

Custom Scripts and Applications

• Use AWS CLI or API to automate deployments in a variety of languages.

• Userdata

AWS CloudFormation

• Use a simple, declarative domain-specific language (DSL) to build a template file that creates and deletes a collection of resources together as a single unit (a stack).

AWS OpsWorks

• Use a simple, declarative domain-specific language (DSL) to create AWS resources.

What is AWS CloudFormation?

– Declarative programming language for deploying

AWS resources.

– Supports many AWS services.

– Create, update, and delete a set of resources as a

single unit (stack).

– Infrastructure as Code.

– Free of Charge.

What Does Infrastructure as Code Mean?

• Techniques, practices, and tools from software

development applied to creating reusable,

maintainable, extensible and testable infrastructure.

CloudFormation: Infrastructure as Code

Allows you to launch, configure, and connect AWS resources with JavaScript Object Notation (JSON) and YAML-formatted templates

Template StackAWS CloudFormation

Engine

• JSON-formatted file

describing the resources

to be created

• Treat it as source code:

put it in your repository

• YAML-formatted template

support

• AWS service component

• Interprets AWS

CloudFormation template

into stacks of AWS

resources

• A collection of resources

created by AWS

CloudFormation

• Tracked and reviewable in

the AWS Management

Console

• Cross stack references

Benefits of Treating Infrastructure as Code

template

Development

Repeatability

Production

Reusability

Auto Scaling

group

Load

balancer

Auto Scaling

group

Load

balancer

Auto Scaling

group

Load

balancer

Auto Scaling

group

Load

balancer

Benefits of Treating Infrastructure as Code

template

Development Production

Maintainability, Consistency, and Parallelization

Template

updated

Auto Scaling

group

Load

balancer

security group

Auto Scaling

group

Load

balancer

security group

Auto Scaling

group

Load

balancer

security group

Auto Scaling

group

Load

balancer

security group

AWS Elasticity and Management Tools

Questions

Knowledge Check 1

• How does Auto Scaling scale instances?– Scale up and down, or

– Scale in and out?

Answer:

Scale in and out. In other words, change the quantity of

instances in the Auto Scaling group.

Knowledge Check 2

• True or False: “Memory Utilization” is a basic

monitoring metric of CloudWatch.

Answer:

• False. It is a custom metric and has to be

implemented by using CloudWatch Logs

Knowledge Check 3

• You have configured a CloudWatch alarm to trigger when

CPU rises above 60%. CPU is currently at 80%. What is

the status of the alarm?

– OK

– ALARM

– INSUFFICIENT DATA

Answer:

• Alarm, but only if the period condition has also

been met (i.e. above 60% for one minute).

Knowledge Check 4

• Can you deploy Configuration Files with

CloudFormation?

Answer:

Yes, by using CloudFormation:Init.

Further Information

• Official AWS Events– Transformation Day

– AWSomeDay

• Meetups

• Digicomp Trainings

• Amanox Events– Bootcamp

• DevOps

• Microservices and Docker

https://aws.amazon.com/de/events/transformation-day-cologne/

https://aws.amazon.com/de/events/transformation-day-cologne/

Zürich - Amazon Web Services€¦ · Security Groups • Restrict access to instances by: –Port...

Documents

Transcript of Zürich - Amazon Web Services€¦ · Security Groups • Restrict access to instances by: –Port...