News UK - Our Journey to Cloud

News UKOur Journey to Cloud

Chris Birch IT Director, News UK

•  About us •  The start of the journey

– Relaxing on Sunday Mornings •  Moving from Ad-hoc to planned •  Approach & Methodology •  Current status

Agenda

About us

News UK

The start of our Journey

•  150,000+ Digital subscribers attempting to login between 08:00 – 10:00hrs

•  System limited to 17 TPS •  Access System failed ‘closed’ •  Sunday morning incident bridges required

The Start of the Journey in 2011" - Dealing with Sunday Mornings

ACS was created and has evolved to support changing digital propositions and requirements whilst remaining fast, secure, simple and flexible

Granular entitlements ●  Every request for protected content is individually

authorised

●  Supports group and individual authentication

Multi tenant ●  Single system manages access for many independent

business units

Flexible CRM and CDN support ●  Uses standard, open technologies

●  No proprietary protocols or messaging systems powering

integration points

Big data friendly ●  Real time dashboards

●  Every interaction logged and stored

✔

✔

✔

✔

Highly performant ●  Serviced >4m requests per hour

●  Typically <50ms to authorise requests

●  Scales elastically with demand

✔ Simple and easy to integrate ●  New service integration in four simple steps

●  Addition / config of products completed via a web-based

administration interface

✔

Secure and unobtrusive ●  SSL used for transmission of customer data

●  Entitlements not cached with end user

✔ More than content ●  Protects DRM key service

●  Share token service

✔

Moving from Ad-hoc to planned

We Built ‘DC3’! •  Three Secure VPC’s •  A monitoring capability - Zabbix & SCOM •  A backup capability - Automated snapshotting •  Authentication - LDAP & AD •  Predictable Network performance - 2 x 10GB links between AWS & Our DC’s •  Storage Gateway

The Sequence of Our Journey We decided on a Cloud First strategy

We created a business case! •  Understand the TCO of your current systems, including regular hardware

refresh. •  Understand the costs to replace your current systems •  Work with your finance teams to understand the ramifications of moving to an

Opex cost model •  Understand the costs of migrating your systems •  Understand your run costs in the Cloud

The Sequence of Our Journey

We undertook an Global Application readiness assessment •  Create a usable inventory of your current estate •  Work with partners / cloud suppliers to assess what can run in the Cloud •  Create a detailed Migration plan


Data centre optimisation - it’s more than just cloud migrations •  Right size your current estate •  Decommission that which you don’t need •  Do not neglect your NFR’s - some things may be easy to move to the cloud

but difficult to backup , monitor or recover •  Use automation as much as possible - Cloud formation & Chef/Puppet for us. •  Implement good governance – Tagging and Service Transition, including

Change control


If you thought sprawl was a problem in a Virtual estate, then you need to know it will

be just as hard in the Cloud!slides

Sprawl Alert - News UK current estate

Global Application Assessment for News Corp

Application Migrations 7 Step Approach

W4: Server Consolidation

Milestone 1 Service Review

Milestone 2 Tech POC

Milestone 4 Build & Test

Milestone 5 UAT

Milestone 6 Release

Milestone 7 Consolidate

Milestone 3 Base Optimise

7 Step Approach

Milestone Goals and Objectives Checkpoint Requirements Key Stakeholders

Milestone 1: Service Review ■  Engage Service Owner and review assets in AppBook to validate transformation ■  Identify business concerns and risks

■  Service Owner Sign-Off ■  Service Owner ■  Regional Delivery Lead

Milestone 2: Tech Review

■  Evaluate specific needs and identify key dependencies ■  Service Owner and Transformation Team collaborate to build-out functional test bed ■  Leverage Best Practices and Infrastructure Policies (Public Cloud, Private Cloud) ■  Identify and agree assets that will be decommissioned ■  No new code other than what is needed to re-platform application

■  Tech Review (i.e., AWS POC) ■  Service Owner Sign-Off ■  Transformation Team Sign-Off

■  Service Owner ■  Regional Delivery Lead ■  Cloud Architect ■  Infrastructure Architect

Milestone 3: Base Optimise

■  Transformation Team to make base optimisation recommendations for assets in the target state

■  Design with failure in mind ■  Utility Computing - consider scaling options ■  Architecture finalised ■  OPEX costs established and recorded ■  Sprint planning for deployment and testing into UAT and Service Transition engagement for

Production delivery

■  Architecture Review ■  Transformation Team Sign-Off

■  Regional Delivery Lead ■  Cloud Architect ■  Infrastructure Architect

Milestone 4: Build & Test ■  Incorporate infrastructure architecture principles around scaling, failover, NFRs ■  Architecture Review

■  Transformation Team Sign-Off ■  Regional Delivery Lead ■  Cloud Architect ■  Infrastructure Architect

Milestone 5: UAT ■  Service Owner and Transformation Team collaborate to review assets in the target state ■  Governance Review

■  Service Owner Sign-Off ■  Transformation Team Sign-Off

■  Service Owner ■  Regional Delivery Lead

Milestone 6: Release ■  Deploy assets to the approved target state ■  Service Owner Sign-Off ■  Transformation Team Sign-Off

■  Service Owner ■  Regional Delivery Lead

Milestone 7: Consolidate ■  Decommission agreed assets (i.e., servers, storage, backups, etc.) remaining after Release is complete

■  Transformation Team Sign-Off ■  Regional Delivery Lead

WK-01: Remediation WK-02: Public Cloud Migration, WK-03: Private Cloud Migration

W6: Storage Consolidation

W7: DC Consolidation

W5: Server Redeployment

Step 1: Service Review

Processes Objectives Tasks Dependencies

M1.1 ■  Overview of RACI for each Migration ■  Assign team members project roles and responsibilities ■  Develop shared document location for project. ■  Develop RACI and distribute to all members for ratification.

■  Resources have been assigned to the project from the required teams

M1.2 ■  Action Log Setup and shared to all team members ■  Create action log in shared location ■  Brief team on action log processes and usage

■  Shared location for project documentation is set up.

M1.3 ■  Risk Log review & Risk Analysis ■  Guidance offered to team

■  Prioritise risks and mitigation actions aligned to achieving the business objectives

■  Categorise each risk as blocker or secondary risk

■  Initial high level scope and requirements identified for app to be reviewed

M1.4 ■  Initial schedule of team meetings set ■  Set up workshops with stakeholders/app owners/project team members to review high level scope, requirements and

■  All stakeholders/project team resources identified and assigned.

M1.5 ■  User Case Document started ■  App owner to identify use cases needed to confirm app functionality for later testing

■  App owner has appropriate resource to be assigned to the project to assist

M1.6 ■  EC2 Count & AWS Cost Budget against Global Assessment ■  Confirm budget against predicted IRR ■  Final vendor agreements in place and rates confirmed.

M1.7 ■  Overview of AWS Operating Policy ■  Obtain AWS operating policy with any News specific contractual agreements included.

■  Final AWS agreements in place

M1.8 ■  Funding agreed in principle for Infrastructure Component & Application Components (upgrades, third party isv's etc)

■  Budgetary BoM and other costs identified ■  Business/App owner provided with costs of migration

■  Initial high level scope and requirements identified for app to be reviewed





Milestone 5 UAT

Milestone 6 Release



WK-01: Remediation WK-02: Public Cloud Migration, WK-03: Private Cloud Migration




SAMPLE TASKS

Step 2: Tech Review and POC


M2.1 ■  Application configuration identified ■  Review current app config on existing infrastructure with documented config

■  Key application SMEs available to participate in review

M2.2 ■  Execution of the existing version of the application with all dependencies working or emulated if needs

■  Assess application execution in current environment state and verify all functionality and integration pieces as per use cases provided by the app owner

■  Use cases defined and provided by app owner.

M2.3 ■  Raise and document any perceived risks/blockers ■  Prioritise risks and mitigation actions aligned to achieving the business objectives

■  Categorise each risk as blocker or secondary risk

■  Documentation of application config review and app execution results completed

M2.4 ■  Install of application ■  Document application installation process incorporating configuration steps to complete install

■  Documentation of application config review and app execution results completed

M2.5 ■  Network Analysis based on existing VLAN & Subnet ■  Engage network team where required to review network ■  Completed in conjunction with execution of the existing app

M2.6 ■  Specific ports identified ■  Consult application owner/ server analysis of TCP ports for firewall and AWS security group determination

■  Review of network analysis results

M2.7 ■  Right sizing analysis based on existing Ventre Ops (CPU/Network/Disk/Memory)

M2.8 ■  Technical dependencies on Services/Servers/Databases/HNAS Storage/LUN's/Shares identified

M2.9 ■  Initial HL Architecture Agreed / Documented




Milestone 5 UAT

Milestone 6 Release



WK-01: Remediation WK-02: Public Cloud Migration, WK-03: Private Cloud Migration W4: Server Consolidation




SAMPLE TASKS

Step 3: Base Optimise


M3.1 ■  Review of count of current on prem instances ■  Determine current server count for selected application and compare with design ■  App assessment and architecture document completed

M3.2 ■  Storage and Backup requirements ■  Assess current on prem storage and backup usage and extrapolate requirements to cloud

■  Current state infrastructure reports available for servers in scope

M3.3 ■  Basic Cost POC Approach ■  Define PoC scope and required components ■  Determine required number of instances and software required ■  Determine PoC operating times and availability

■  App assessment and architecture document completed

M3.4 ■  Count of Sprints agreed ■  Perform App estimates and point count. ■  Apply methodology to determine number of sprints required for each application

type

■  Defined scope agreed ■  Costs agreed ■  Agile methodology agreed on

M3.5 ■  Key Milestones ■  Document key deliverables ■  Map deliverables to requirements

■  Sprint count and budget determined

M3.6 ■  Risk/Review update ■  Revise risk log and mitigation actions to confirm continued alignment to achieving the business objectives

■  Defined scope agreed ■  Key milestones agreed ■  PoC costs known

M3.7 ■  AWS Costs & Run

M3.8 ■  KPI's

M3.9 ■  Scrum or checkpoints actions ■  Confirm all deliverables of scrums have been met ■  Review what has been delivered and repeat process for future scrums as

required

■  Scrum artifacts are completed for review

M3.10 ■  Identification of VLAN's

M3.11




Milestone 5 UAT

Milestone 6 Release







SAMPLE TASKS

Step 4: Build & Test


M4.1 ■  AWS Console Account Policies ■  Determine if current account policies support the app to be migrated ■  Architecture Design Document and Use Cases document completed

M4.2 ■  AWS VPC ■  Determine based on app ownership/app role which AWS VPC to build environments in

■  Architecture Design Document and Use Cases document completed

M4.3 ■  AWS Console IAM Users ■  Apply account policies to App Support requirements to determine who is granted access to the console

■  AWS Console Account Policies have been determined

M4.4 ■  Dependencies / AMI / Cloudformation / SCCM / Chef steps and actions to programmatically create build

■  Identify all sources, tools, environment and other components that comprise the app to be built

■  Determine dependencies on other tools or processes needed to successfully build the app

■  Complete operational readiness checklist

■  Operational readiness checklists available and being followed

M4.5 ■  IOteams (IOtools / IOSecurity / IOAWS / IOWindows / IOUnix) involved and sign off at UAT Stage

■  Build environment ■  Organise review of non-functional testing results ■  Conduct review of application readiness and obtain signoff by support

teams on operational readiness

■  Use cases and operational readiness checklists completed.

■  Architectural design and security configurations completed

M4.6 ■  Governance Sign off on AWS TA All Green ■  Obtain signoff from agreed governance stakeholders that app is ready for final UAT and implementation/release

■  All key artifacts have been delivered and signed off by the stakeholders responsible.

M4.7 ■  Infrastructure Monitoring and Alerting ■  Test monitoring software ■  Simulate failures and document monitoring results ■  Determine alerting based on monitoring criteria

■  Monitoring integration and alerting configuration completed

M4.8 ■  Application Heart Beat Monitoring ■  Ensure app monitoring is in place and working using agreed tool for the app being migrated

M4.9 ■  Backup ■  Determine required backup solution based on app and data requirements

■  Implement and test backups are conducted as required by the chosen solution

■  Required backup infrastructure to support backup type is in place, has capacity and is ready for use.




Milestone 5 UAT

Milestone 6 Release







SAMPLE TASKS

Step 5: UAT


M5.1 ■  NFR Checkpoint ■  Non functional test results are documented and have been signed off. ■  Non functional testing has been completed as specified and agreed to by stakeholders

M5.2 ■  AWS Trusted Advisor ■  Review of security configuration for the app is performed and any remaining issues identified

■  Remediate issues identified ■  Sign off on environment security

M5.3 ■  Service Owner Checks ■  Service owner perform verification of all use cases in the environment where possible to verify the solution is ready for migration and implementation

M5.4 ■  AWS Service Owner ■  AWS owner to confirm AWS environment is ready to accept new service.

M5.5 ■  Full Architectural review & signoff ■  Workshop with stakeholders, IO and Architecture teams organised to perform review

■  Document minutes/artifact of acceptance including signoff by all required staff

M5.6 ■  IOTeams sign off

M5.7 ■  Cloudformation




Milestone 5 UAT

Milestone 6 Release







SAMPLE TASKS

Step 6: Release


M6.1 ■  Implementation strategy planned and documented ■  Required teams participate in a workshop to determine and implementation plan to migrate/cutover from the old system to the new cloud based service

■  Testing, acceptance and rollback procedures to be documented ■  Implementation Plan is document and then reviewed by all teams and the

service owner and AWS owner to determine

M6.2 ■  Service Owner raises all the CR required to bring new infrastructure into service

■  Change raised to bring new system into service via chosen method of release/migration

■  Change represented at Change weekly meeting for signoff and for final scheduling.

■  Implementation plan including Testing, and rollback plans to be competed for inclusion into CR.

■  Service Management Plan completed for support

M6.3 ■  User notifications /change management completed ■  Develop user communications in conjunction with the Service Owner to inform users of any changes as a result of the App now being hosted in AWS.

■  Schedule user communications as required by the implementation plan ■  Sign off on SMP

■  Implementation plan completed and CR raised ■  Service Management Plan completed for support




Milestone 5 UAT

Milestone 6 Release







SAMPLE TASKS

Step 7: Decommission – Get the benefits……


M7.1 ■  SO to raise CR to decom storage, vm's and backup schedules ■  Verify list of servers/storage/backup config is accurate ■  Verify with application owner all server/storage/backup infrastructure is no

longer in use ■  Raise decom CR in regional service management tool

■  Service Owner acceptance and completion of CR to bring service into production.

M7.2 ■  Update SSOT data with changes to server, app, other data elements ■  Document final state of application and server changes. ■  Verify all compute services on premise are removed based on closed CR.

■  Completion of CR by all teams

M7.3 ■  Server assets identified as suitable to be available for redeployment ■  Determine if decom hardware asset is suitable for redeployment. ■  If suitable, then add to server redeployment workstream for asset reuse

■  Minimum SOE hardware requirements defined and document

M7.4 ■  Ensure financial systems are updated and maintenance agreements on decom server assets are cancelled

■  Follow asset disposal/maintenance service cancellation procedures for decom assets that are not suitable for redeployment

■  SSOT data has been updated.





Milestone 5 UAT

Milestone 6 Release



WK-01: Remediation WK-02: Public Cloud Migration, Workstream 3: Private Cloud Migration


W7: DC Consolidation/close


SAMPLE TASKS

Current Status & Beyond

Infrastructure Transformation: KPI / Metric Update

KPI / Metric

Australia EMEA North America TOTAL

FY15 Goal

YTD Δ This Period

FY15 Goal

YTD Δ This Period

FY15 Goal

YTD Δ This Period

FY15 Goal

YTD Δ This Period

Cloud Compute %

36% 29% 1% 54% 51% 4% 37% 25% 2% 40% 31% 1%

Data Centre Closure

3 1 1 1 0 0 1 1 1 5 2 2

App Migration 30 2 2 29 7 7 55 2 2 114 11 11

Server Migration 252 15 15 117 22 22 951 33 33 1,320 70 70

Server Decommission

694 274 274 967 437 437 1,980 149 149 3,641 860 860

Current Status

PAGE 3

3 year plan

KPI / Metric Targets

AU EMEA NA GLOBAL

Now FY15 FY16 FY17 Now FY15 FY16 FY17 Now FY15 FY16 FY17 Now FY15 FY16 FY17

Cloud Compute %

29% 36% 53% 71% 51% 54% 74% 89% 25% 37% 61% 85% 31% 40% 62% 83%

Data Centre Count

9 7 3 2 6 5 3 2 9 9 2 2 24 21 8 6

Public Cloud Apps

73 98 168 187 138 168 274 293 315 370 549 741 526 636 991 1,221

On Premise Apps

270 198 104 84 311 204 80 36 624 514 322 121 1,205 916 506 241

Public Cloud Servers

1,284 1,502 2,087 2,734 1,962 2,057 2,558 2,961 2,836 3,858 6,206 8,699 6,082 7,417 10,851

14,394

On Premise Servers

3,105 2,698 1,863 1,142 2,152 1,717 895 366 8,445 6,583 4,029 1,527 13,702

11,054

6,787 3,035

PAGE 5 Shared Technology Services (STS) Update / 03 OCTOBER, 2014

AWS Enterprise Summit

@AWS_UKI

#AWSSummit

News UK - Our Journey to Cloud

Technology

Transcript of News UK - Our Journey to Cloud