Pragmatic Enterprise Application Migration to AWS

25
© 2015 Cloud Technology Partners, Inc. / Confidential 1 v Pragmatic Enterprise Application Migration to AWS Boston AWS Meetup 7/14/2015

Transcript of Pragmatic Enterprise Application Migration to AWS

Page 1: Pragmatic Enterprise Application Migration to AWS

© 2015 Cloud Technology Partners, Inc. / Confidential 1v

Pragmatic Enterprise Application Migration to AWS

Boston AWS Meetup 7/14/2015

Page 2: Pragmatic Enterprise Application Migration to AWS

© 2015 Cloud Technology Partners, Inc. / Confidential 2

Lift & Shift:

Case Study• Vice President, Principal Architect at Cloud Technology Partners

• AWS Certified Solutions Architect

• Cloud architecture, application migration, cloud data

management, DevOps / ProdOps

[email protected]

@kacyclarke

updraft-downdraft.blogspot.com

Presenter

Kacy Clarke

Page 3: Pragmatic Enterprise Application Migration to AWS

© 2015 Cloud Technology Partners, Inc. / Confidential 3

Enterprise Applications Don’t Always Fit the Cloud

Page 4: Pragmatic Enterprise Application Migration to AWS

© 2015 Cloud Technology Partners, Inc. / Confidential 4

Lift & Shift:

Case Study

Cloud Adoption Starts with these Six Key Tenets

1 2 3 4 5 6

Strategy & Economics

Executive Cloud Benefits, Objectives

and Goals

Cloud Readiness Assessment -

3 Year Actionable Roadmap

ROI / TCO Economic Models

Security & Governance

CSA/ISO 2700X Reference

Architecture

Gap Analysis of InfoSec Policy,

Procedures, and Key Controls

Cloud Security Tooling Gap

Analysis

Application Portfolio

Assessment

Portfolio Assessment

Readiness for Cloud

Cloud Reference Architecture and

Capabilities Matrix

Application Cloud Roadmap & Plans

Application Migration &

Development (MVC)

Design and Delivery of a Minimum Viable

Cloud (MVC)

Application Migration Factory

Infrastructure Automation

DevOps

DevOps Maturity Assessment

DevOps Certification and

Training

DevOps as a Service (DOaaS)

CloudOps

Service Management, Cloud Operational Model

(MSP)

Governance, Billing, Chargeback, Audit, Logging, Escalation

Organizational & Skill Set Training

1 2 3 4 5 6

Page 5: Pragmatic Enterprise Application Migration to AWS

© 2015 Cloud Technology Partners, Inc. / Confidential 5

Objective: Rehost / Lift & Shift

• Migrated to AWS quickly to address rapid

growth, minimize risk with a typical outsourcer

• Minimized changes to the applications and

infrastructure with minimal operational tools

• Used traditional data center application

technology stack: Weblogic cluster, Oracle

RAC, Coherence with stateful Weblogic

session management

• New security architecture did not mesh with

security scanning protocols

Client began moving customer facing products to AWS to leverage

scalability and regional deployments for millions of international users.

Lift and Shift Case Study: Global Media Client

Lessons Learned

• Applications typically relied on infrastructure

supported availability, rather than being designed to

be self-healing for component failure

• Infrastructure oriented monitoring tools make it very

difficult to troubleshoot application problems.

Insufficient knowledge of application execution

• Scale out matters at so many levels – e.g. resource

bottlenecks, noisy neighbors, load distribution and

component performance variability

• Applications were designed for fixed resources, so

had to over allocate reserved instances, driving up

cost

Result: Application Did Not Meet SLA’s

Page 6: Pragmatic Enterprise Application Migration to AWS

© 2015 Cloud Technology Partners, Inc. / Confidential 6

Lift & Shift:

Case Study

• Mission critical Service Level Agreement

• Full production migration

• Multi-tier architecture

• Ecosystem dependencies

• Security and compliance concerns

• Data creation and/or update

• Limited time and resources

Target Application Characteristics

Page 7: Pragmatic Enterprise Application Migration to AWS

© 2015 Cloud Technology Partners, Inc. / Confidential 7

Lift & Shift:

Case Study

• Missed SLA’s

• Performance and latency problems

• Environment inconsistencies

• Gaps in monitoring and operational tools

• Data synchronization issues

• SDLC environment sprawl

• Lack of transparency for what’s happening with the application or the system

• Development and operational skills and knowledge gaps

• Manual or frozen configurations

• Compliance or security findings

• Cost savings objectives not met

Pure Lift and Shift Can Result in Cloud Migration Debt

Page 8: Pragmatic Enterprise Application Migration to AWS

© 2015 Cloud Technology Partners, Inc. / Confidential 8

The Cloud is Very Different

Traditional

Architectures

• Scale Up

• Monolithic

• Stateful

• Infrastructure Dependent

(i.e LAN,SAN, etc.)

• Fixed Capacity

• Latency intolerant

• Consolidated / clustered DB

• Commercial licenses

• Manual build/deploy

• Manual fault recovery

• Active/Passive/DR

• Perimeter Security

• Allocated costs

Cloud Aligned

Architectures

• Scale Out

• Distributed

• Stateless

• Elastic capacity

• WAN, Location transparency

• Latency tolerant

• Loosely coupled

• Sharded / replicated /

distributed DB

• Mobile/thin client

• Cloud PaaS / Open Source

• Automation

• Self healing

• Active/Active

• Metered cost

Refa

cto

r

Con

tin

uo

us

Deliv

ery

Page 9: Pragmatic Enterprise Application Migration to AWS

© 2015 Cloud Technology Partners, Inc. / Confidential 9

Lift & Shift:

Case Study

How Much Do You Change for Your Initial AWS Application Migration?

Which operational tools

should I keep and which

should I replace?

How much application code

do I need to change?

Should I replace my application

technology stack with AWS

services?

Should I automate my

deployments?

Do I need to add security

tools or configuration?

Do I need to implement

autoscaling?

What do I need to do with

my application

integration?

What if other applications

need to access my

database?

How do I address data retention

and compliance?

Page 10: Pragmatic Enterprise Application Migration to AWS

© 2015 Cloud Technology Partners, Inc. / Confidential 10

Lift & Shift:

Case Study

Cloud

CLOUD

WASHED

CLOUD

ADOPTED

CLOUD

OPTIMIZED

CLOUD

NATIVE

- Force fit to run in cloud

environment

- Resources not optimize –

no horizontal scaling

- Minimal app modification

to be cloud compliant

- Infrastructure based

availability

- Reuse of data center

management and

monitoring technologies

- Resources not optimized

- No autoscaling

- Failure intolerant –

connections and

resources may not all

automatically restart

- Some app modification to

be cloud compliant and

use cloud platforms

- Over or under utilization

of resources has

cost/performance impact

- Some cloud aligned

operational tools

- Resources being

optimized – horizontal

scaling possible

- Managed elasticity–

cloud management layer

determines when to

start/stop additional

instances

- Major app modification to

be cloud aligned

- Coarse grained cost and

performance resource

optimization

- Cloud operational tools

- Fully cloud aware – app

communicates with the

cloud management layer

to start-up or shut down

instances as needed

- Designed for failure and

self healing

- Resource efficient

- Cloud native platform

and operational tools

- Fine grained cost and

performance resource

optimization

Cloud Application Maturity

Business Needs Drive Cloud Maturity Target

Page 11: Pragmatic Enterprise Application Migration to AWS

© 2015 Cloud Technology Partners, Inc. / Confidential 11

Lift & Shift:

Case Study

Traditional

(Anti-Pattern)

Cloud

Washed

Cloud

Adopted

Cloud

Optimized

Cloud

Native

AutomationManual or limited

scripts

Manual or limited

scriptsSemi-automated Full automation

Continuous

delivery

Scaling Fixed capacityReserved

capacity

Reserved and

elastic

Resources on

demand

Autoscaling at

every tier

State

ManagementStateful session

mgmt

Stateful sessions,

stickiness

Stateless,

location sensitive

Stateless, multiple

AZ load

distribution

Stateless,

optimized load

management

ResiliencyInfrastructure

oriented

Infrastructure

oriented

Redundant,

retry, restartSelf-healing

Perf/health

sensitive

IntegrationTightly coupled,

native interfacesMixed coupling

Both synch and

asynch, svc bus

Loose coupling,

API

API, eventually

consistent

Availability

ManagementInfrastructure

based, clustering

Infrastructure

based, clustering

Application

aware, manual

initiation

Application

initiated

Application

controlled

DatabaseProprietary, big

iron, physical

Some VM DB,

most on physical

Sharding,

distribution,

eventual

consistency

Cloud aligned

dbms, distributed,

dbms elasticity

Cloud native

dbms

Applications Targeting the Cloud

Page 12: Pragmatic Enterprise Application Migration to AWS

© 2015 Cloud Technology Partners, Inc. / Confidential 12

Lift & Shift:

Case Study

Traditional

(Anti-Pattern)

Cloud

Washed

Cloud

Adopted

Cloud

Optimized

Cloud

Native

StorageSAN, NAS, local

storage

Mounted storage,

shared file

systems

Storage agnostic,

location sensitive

Storage agnostic,

replication for

speed/ resiliency

Storage location

insensitive,

replication for

speed/ resiliency

Network

App tiers and

integration on

LAN, latency

sensitive

App tiers and

integration on

LAN, latency

sensitive, external

WAN

App tiers in

limited AZ’s, LLB,

latency sensitive

App tiers across

AZ’s, GLB,

latency tolerant

GLB, Location

transparency,

latency tolerant

ComputePhysical , some

VM

VM with

exceptions (DB,

FW, …)

VM, speed,

memory sensitive

VM agnostic,

speed/memory

adaptive

VM agnostic,

speed/memory

optimized

OS/Container ProprietaryLinux, Win,

Commercial SW

Linux, Win, mixed

SW stack

Open source/

Cloud PaaSCloud native

Network/Host

SecurityPerimeter, physical

Access controls,

perimeter focus

Service level

security

System Defense

in Depth

Application

Defense in Depth

Data ProtectionEncrypt for

external transfer

Encrypt for

external transfer

Encrypt in flight,

at rest

Encrypt in flight,

at rest

Encrypt in flight,

at rest, in use

Applications Targeting the Cloud

Page 13: Pragmatic Enterprise Application Migration to AWS

© 2015 Cloud Technology Partners, Inc. / Confidential 13

Systems Thinking: A holistic approach to analysis that

focuses on the way that a system's constituent parts

interrelate and how systems work and change over time

and within the context of larger systems. The systems

thinking approach contrasts with traditional analysis,

which studies systems by breaking them down into their

separate elements

Cloud Migration Impacts

– Application Architecture

– Infrastructure Architecture

– Data Lifecycle Management

– Security and Compliance

Application migration requires the holistic analysis of systems thinking

It’s Not an Application, It’s a System

– SDLC

– Operations and Monitoring

– Support

– Cost Management and Planning

Page 14: Pragmatic Enterprise Application Migration to AWS

© 2015 Cloud Technology Partners, Inc. / Confidential 14

Lift & Shift:

Case Study1. Replace key operational tools

2. Enhance security infrastructure

3. Remediate application code to maintain SLA’s

4. Selectively replace platform technology

5. Automate application provisioning and deployment -

automate everything else while you’re at it

6. Create dashboards to monitor the full stack

7. Adopt a DevOps approach

Recommendations

Page 15: Pragmatic Enterprise Application Migration to AWS

© 2015 Cloud Technology Partners, Inc. / Confidential 15

Lift & Shift:

Case Study

Machine Image Creation

– OS hardening and tool installation for base AMI’s

– AMI incremental builds for common application

infrastructure (database, app server)

– Consider an AMI build tool like Packer

Cloud Aware Monitoring

– System monitoring with integration with AWS, CloudWatch

(StackDriver, Data Dog, Science Logic)

– Application Performance Monitoring with distributed environments

(New Relic, App Dynamics)

Log Management

– Infrastructure for both Security and DevOps (ELK, Splunk, SumoLogic)

Replace key operational tools

Page 16: Pragmatic Enterprise Application Migration to AWS

© 2015 Cloud Technology Partners, Inc. / Confidential 16

Lift & Shift:

Case Study

• Add cloud oriented security tools

– Security groups replacing data center firewalls

– Security group configuration scanning (Dome 9, etc.)

– Instance configuration scanning (Alert Logic, CloudPassage, etc.)

– Web application firewalls

– Log management for audit (Alert Logic, ELK, Splunk, etc.)

• Add Key and Certificate Management

• Encrypt data at rest

– File systems

– Object stores (S3, Glacier)

– Database

– Backup

– AMI’s

• Reexamine SSL offloading

Enhance security infrastructure

Page 17: Pragmatic Enterprise Application Migration to AWS

© 2015 Cloud Technology Partners, Inc. / Confidential 17

Minimize app changes to focus on SLA’s,

dynamic environment and security

Remediate application code to maintain SLA’s

Resiliency

– Spread application tiers across Availability

Zones

– Timeout, retry, reconnect for integration points

– Add load balancers between tiers

– Enhance error checking and alerting

– Fail database over between AZ’s

– Selectively add logging

Abstract Configuration

– Replace hard coded IP addresses, host names

– Generate property/config files in automation

Loose Coupling

– Replace RMI, EJB calls with web services

– Selectively replace synch with asynch

integration

– Add database caching if database is

remote

– Remove clustering and session replication

if possible. Move session state to

DynamoDB or other DB

Application Security

– Encrypt data in flight, at rest

– Encrypt or lookup integration credentials

Page 18: Pragmatic Enterprise Application Migration to AWS

© 2015 Cloud Technology Partners, Inc. / Confidential 18

Target areas that would enhance SLA’s, performance, manageability

Selectively replace application platform technology

Component Examples AWS Options Rationale

Messaging

MiddlewareIBM MQ, Tibco SQS, RabbitMQ

Cost, manageability,

effort

Application Server Websphere, WeblogicElasticbeanstalk, Jboss,

TomcatCost, horizontal scaling

CachingCoherence, EHCache,

HazelcastElasticache

Cross-AZ, scalability,

manageability

CDN Akamai, Level 3, Limelight CloudFrontCost, integration,

manageability

Shared File System NetApp, Windows file server AWS EFS Scalability, manageability

DatabaseOracle RAC, SQL Server

ClustersRDS, Enterprise DB Cost, manageability

Load Balancing F5, Netscaler AWS ELBCross-AZ, horizontal

scaling, manageability

Static content File server S3 Availability, scalability

Page 19: Pragmatic Enterprise Application Migration to AWS

© 2015 Cloud Technology Partners, Inc. / Confidential 19

“Everything fails, all the time.” - Werner Vogels

Automate application provisioning and deployment

• Automate full stack, push button deployments,

startup/shutdown

• Forbid manual changes to application

configuration or environments after Dev

• Build full production environments (except

data tier) for each release, and then cutover

when ready

• Bake AMI’s for rapid deployment

• Log automation steps

• Automate acceptance/smoke tests

Page 20: Pragmatic Enterprise Application Migration to AWS

© 2015 Cloud Technology Partners, Inc. / Confidential 20

• Service catalog automation

– AMI builds

– Service lifecycle management

– Testing

• Auto response to common issues

• Failover/failback

• Disaster recovery

• Self-service user support

• Patching

Automate everything else while you’re at it

• Capacity management

• Backup/recovery

• Data archive/restore

• Space reclamation

• Audit

• Reporting

Page 21: Pragmatic Enterprise Application Migration to AWS

© 2015 Cloud Technology Partners, Inc. / Confidential 21

• Iterative implementation for

visualization refinement and

tuning data collection

• Dashboards for:

– Alerts

– Application performance

– Activity/load

– Systems resources

– Change activity

– Historical trends

• Different views for different

stakeholders

• Implement periodic reviews for

continuous improvement

Create dashboards to monitor the full stack and the processes

Page 22: Pragmatic Enterprise Application Migration to AWS

© 2015 Cloud Technology Partners, Inc. / Confidential 22

Adopt a DevOps approach

Page 23: Pragmatic Enterprise Application Migration to AWS

© 2015 Cloud Technology Partners, Inc. / Confidential 23

• Leverage DevOps best practices to change how app teams interact with operations

• ITIL based practices/structure will not vanish overnight, gradually adapt to

DevOps/Kanban

• Treat the platform as a product

Adopt a DevOps approach

Agile/Lean SDLC

DevOps:

Agile Infrastructure

Cloud Service Management

ProdOps: Cloud

Operations

Network ServerSecurity Storage App Infra Database Operations Support

Application Focus Cloud Focus

Page 24: Pragmatic Enterprise Application Migration to AWS

© 2015 Cloud Technology Partners, Inc. / Confidential 24

Example Minimum Viable Cloud for Enterprise Applications

Applications

Application/Data Infrastructure

Full Stack App Automation

Service Catalog

Automation Tools, Log Mgmt, Monitoring,

Image Mgmt, Backup, Support

Network, Security, IAM, Audit

Amazon Web Services

Development Toolkit

Page 25: Pragmatic Enterprise Application Migration to AWS

© 2015 Cloud Technology Partners, Inc. / Confidential 25

Boston Headquarters

263 Summer Street

Fourth Floor

Boston MA, 02210

Contact

617.674.0874

[email protected]

www.cloudtp.com

vv

Questions?

[email protected]