Continuous Delivery of a Cloud Deployment at a Large Telecommunications Provider

29
© 2013 IBM Corporation Continuous Delivery of a Cloud Deployment at a Large Telecommunications Provider M Kevin McHugh @mkmchu Eduardo Patrocinio @patrocinio 1717 – February 25, 2014

Transcript of Continuous Delivery of a Cloud Deployment at a Large Telecommunications Provider

Page 1: Continuous Delivery of a Cloud Deployment at a Large Telecommunications Provider

© 2013 IBM Corporation

Continuous Delivery of a Cloud Deployment at a Large Telecommunications ProviderM Kevin McHugh @mkmchu

Eduardo Patrocinio @patrocinio

1717 – February 25, 2014

Page 2: Continuous Delivery of a Cloud Deployment at a Large Telecommunications Provider

1

Agenda

Introduction

Continuous Delivery – what is it, how did we do it

Wrap up

Questions

Page 3: Continuous Delivery of a Cloud Deployment at a Large Telecommunications Provider

2

Agenda

Introduction

Continuous Delivery – what is it, how did we do it

Wrap up

Questions

Page 4: Continuous Delivery of a Cloud Deployment at a Large Telecommunications Provider

3

Solution Architecture

Page 5: Continuous Delivery of a Cloud Deployment at a Large Telecommunications Provider

4

Agenda

Introduction

Continuous Delivery – what is it, how did we do it

Wrap up

Questions

Page 6: Continuous Delivery of a Cloud Deployment at a Large Telecommunications Provider

5

Definition

Continuous Delivery (CD) is a design practice used in software development to

automate and improve the process of software delivery. Techniques such as automated testing, continuous integration and continuous deployment allow software to be

developed to a high standard and easily packaged and deployed to test environments, resulting in the ability to rapidly, reliably and repeatedly push out enhancements and bug fixes to customers at low risk and with minimal manual overhead. The technique was one of the assumptions of extreme programming but at an enterprise level has developed into a discipline of its own, with job descriptions for roles such as "buildmaster" calling for CD skills as mandatory.

(Source: Wikipedia 1/2014 - http://en.wikipedia.org/wiki/Continuous_delivery)

The ability to build a releasable asset with nearly trivial effort

What about Continuous Deployment?

Page 7: Continuous Delivery of a Cloud Deployment at a Large Telecommunications Provider

6

Components

Method

Continuous Integration

Process

Assets

Automated Test

Automated Build of deployable asset

Tools

Page 8: Continuous Delivery of a Cloud Deployment at a Large Telecommunications Provider

7

Method: Agile

7

Team Member

Team Member

Customer

Team Member

Team Lead

Agile

�Value-driven lifecycle with regular production of working software

�Small, co-located team developing straightforward software

�Goal is to develop a high-quality system in an evolutionary, collaborative, and self-organizing manner

�Pre-requisite: can build a product backlog

Agile

Scrum Roles (pigs)

�Product Owner

– Owns the product backlog

– Chooses what work to do next

�Team Member

– Creates the solution

– Owns estimation of difficulty of items in the backlog

– Constrains how much work can be done

�Team Lead (Scrum Master)

– Removes obstacles

– Status reporting

Page 9: Continuous Delivery of a Cloud Deployment at a Large Telecommunications Provider

8

Agile terms and how the team operates

Backlog Grooming

Sprint Planning

Scrums

Velocity

Sprint Demo

Sprint Retrospective

Sample User Stories and point assignments:

8

Story

Points Summary

13 pts As a Bank Customer, I want to apply for a loan, so that I can…

8 pts As a Bank Customer, I want to deposit cash

5 pts As a Bank Customer, I want to change my PIN.

3 pts As a Bank Customer, I want to withdraw cash

1 pt As a Bank Customer, I check my account balance

5 pts As a Bank Customer, I want to buy postage stamps

AgileMethod: Tenants of Agile and its iterative cycle

Sprint 1 Sprint 2

Story Points DeliveredStory Points Delivered Story Points DeliveredStory Points Delivered

Velocity (rolling average of points delivered)Velocity (rolling average of points delivered)

Page 10: Continuous Delivery of a Cloud Deployment at a Large Telecommunications Provider

9

Agile adoption – adoption by execution

Two-week sprints

Single Product Owner

Scrum Master

Rank user stories

Set up the meetings

Method: What we did Agile

Sprint 1 Sprint 2

Story Points DeliveredStory Points Delivered Story Points DeliveredStory Points Delivered

Velocity (rolling average of points delivered)Velocity (rolling average of points delivered)

Page 11: Continuous Delivery of a Cloud Deployment at a Large Telecommunications Provider

1010

Sprint by Sprint view of demonstrated progress of feature delivery

Agile

Release 1.0 - Functional Story Point Burndown

0

50

100

150

200

250

300

350

400

450

Sprin

t 1Spr

int 2

Sprin

t 3Spr

int 4

Sprin

t 5Spr

int 6

Sprin

t 7Spr

int 8

Time

Sto

ry P

oin

ts

Backlog Size

Remaining Backlog

Sprint Velocity

Projected Pts

Planned Pts

Method: Sample Burndown – Measured delivery

Page 12: Continuous Delivery of a Cloud Deployment at a Large Telecommunications Provider

11

Continuous Integration

Continuous integration (CI) is the practice, in software engineering, of merging all developer working copies with a shared mainline several times a day.

(Source: Wikipedia 1/29/2014 - http://en.wikipedia.org/wiki/Continuous_integration)

Develop features or fix defects on isolated branches

Merge branches onto the main

Identify points in time on the main that represent a viable code drop

This all drives the ability to have a stable, automatically tested, “main” which can be sent forward for Test, QA, and Production deployment

Page 13: Continuous Delivery of a Cloud Deployment at a Large Telecommunications Provider

12

Continuous Integration – what we did

Initially

Work space isolation obtained via Business Process Manager (BPM) locking of a business process

The team is working on the main

This limited / prevented concurrent engineering

Led to Sprint-fall

Evolved into

Paired programmers working on separate tool kits, and backend processes

Page 14: Continuous Delivery of a Cloud Deployment at a Large Telecommunications Provider

13

Process: 30,000 ft view

Development Test QA Production

Main

Feature 455

Unit Test

Asset Mgmt deploy deploy deploy

Build

Customer E2E Automated Test

- REST API

Customer Manual Test

- UAT

Automated Test

IBM Automated Test

- REST API

- UI

IBM Automated Test

- REST API

- UI

Customer Manual Test

- UAT

Page 15: Continuous Delivery of a Cloud Deployment at a Large Telecommunications Provider

14

Assets

Assets include any element that the team can change on any of the systems supporting the project. For example:

A WAR file

An executable

An environment variable

A database definition

Page 16: Continuous Delivery of a Cloud Deployment at a Large Telecommunications Provider

15

Assets – what we did

BPM process application and tool kitTWX files

PatternThe topology of an application

PatchThose changes to SCO software and/or associated configuration changes

ConfigurationThose changes to the systems supporting the SCO product

Configuration of the VMs internally and from a VM mgmt standpoint

ImagesThe building block for a pattern

Page 17: Continuous Delivery of a Cloud Deployment at a Large Telecommunications Provider

16

Rational Team Concert

Assets – what we did

Development

Exported as .twxfile, then checked

into RTC along with a change

description

Import .twx from RTC

QATest Production

Page 18: Continuous Delivery of a Cloud Deployment at a Large Telecommunications Provider

17

Automated Test

Automated test provides the means to assess the quality of the main branch or the deployed release

Through the use of Automated Test, the Continuous Delivery framework can then decide of a particular release candidate is of sufficient quality to be promoted to the next level

Page 19: Continuous Delivery of a Cloud Deployment at a Large Telecommunications Provider

18

Automated Test – what we did

Test Architecture

Automated REST API Test

Deliver a solution that can be accessed via REST API

Verify that, through the REST API, all features work

Automated User Interface (UI) Test

Confirm that the solution works with the developed user interface

Page 20: Continuous Delivery of a Cloud Deployment at a Large Telecommunications Provider

19

Automated Build of Deployable Asset

Automated build provides the means to collect the set of assets from the development system such that they can later be deployed to a target system

Automated deploy provides the means to place the assets onto a target system for test and use

Page 21: Continuous Delivery of a Cloud Deployment at a Large Telecommunications Provider

20

Automated Build of Deployable Asset – what we did

Use SmartCloud Orchestrator (SCO), REST based export/import feature

Implemented a persistence layer to encapsulate differences between deployment environments

Leverage build and version control tools

Control asset versions

Initiate automated testing

Track test results

Page 22: Continuous Delivery of a Cloud Deployment at a Large Telecommunications Provider

21

Tools to integrate and support

Integrate SCM and Build management tools toSpeed the development

Deliver to the customer control systems

Automated TestTest the User Interface – record and replay UI tool

Test the REST Interface – test harness to invoke API, record results

Automated BuildBPM – REST API based extract from SCO

Patches, etc handled more conventionally

Automated DeployBPM – REST API based import to SCO

Patches, etc handled more conventionally

Page 23: Continuous Delivery of a Cloud Deployment at a Large Telecommunications Provider

22

Tools to integrate and support – what we did

Page 24: Continuous Delivery of a Cloud Deployment at a Large Telecommunications Provider

23

Tools to integrate and support – what we did

Page 25: Continuous Delivery of a Cloud Deployment at a Large Telecommunications Provider

24

Agenda

Introduction

Continuous Delivery – what is it, how did we do it

Wrap up

Questions

Page 26: Continuous Delivery of a Cloud Deployment at a Large Telecommunications Provider

25

Components

Method

Continuous Integration

Process

Assets

Automated Test

Automated Build of deployable asset

Tools

Page 27: Continuous Delivery of a Cloud Deployment at a Large Telecommunications Provider

26

…In Retrospect

Defect zone vs. Story zone (defect backlog vs. story backlog)

Team commitment to Agile and CD – No boycotts

Sprint granularity of CD vs. Story level of CD

Deliver both Product and Services

Difficulty grasping task ownership and delivery

Page 28: Continuous Delivery of a Cloud Deployment at a Large Telecommunications Provider

27

Questions

Page 29: Continuous Delivery of a Cloud Deployment at a Large Telecommunications Provider

28

Thank you!!