Agile & DevOps : Gaming - HATech DevOps Services, a ... · “The DevOps movement addresses the...

ENABLING ORGANIZATIONALAGILITY

ORGAGILITY

Agile & DevOps : Gaming

Copyright © 2012-2017, OrgAgility. All Rights Reserved.

QUESTIONQuick poll

How many people here are familiar with and have used agile techniques at work?How many of you have worked in companies or on projects that tried agile and failed?

1Copyright © 2012-2017, OrgAgility. All Rights Reserved.

Alex SinghEmail: [email protected]: http://www.orgagility.comBlog: http://orgagility.wordpress.comTel: +1 949-667-1008Tel: +1 646-833-8001

2

Jon HathawayEmail: [email protected]: http://hatech.ioBlog: http://hatech.ioTel: +1 702 875 2330Tel: +1 702 389 8160

3JH


ORGAGILITY

What We DoTransform your business—operations, product and development, working together as one integrated agile team—and drive towards purpose via continuous innovation, discovery, delivery, learning and improvement.


ORGAGILITY

What We BelieveWe are committed to creating environments where people can experience joy, self-expression, partnership, and satisfaction arising from the completion of challenging and meaningful work.

Agenda

• Lean-Agile guiding principles• Agile journey at a gaming company• Q&A

Copyright © 2012-2017, OrgAgility. All Rights Reserved. 6

Guiding Principles

7

SOFTWARE DEVELOPMENT

Positive results, but …Industry Week/MPI Census of Manufacturers, 2008

• Fully achieved their objectives: 2%

• Significant result: 24%

• Not making good progress: 74%

Ken Schwaber, 02/2008

• “I estimate that 75% of organizations using Scrum will not succeed in getting the benefits that they hope for from it.”


Top 5 reasons for agile adoption struggles

• Lack of leadership support• Organization culture at

odds with lean-agile values

• Lack of systems thinking• Ignoring people

development• Cargo cult agile


Simplicity is the ultimate sophistication.


Leonardo Da Vinci

AgileCollection of practicesfocused on iteratively producing functionality


A framework within which people can address complex adaptive problems, while productively and creatively delivering products of the highest possible value

Agility… predictably and

frequently delivering value increments to

customers

… that enables …

Image Source: Westminster 2017 12


Agile needn’t be complicated

It’s not about the practices, the charts and the graphs, but about the smiles!

Outcomes

Clarity of Work

Incremental & Iterative Delivery

of Value & Results

Engaged Customer

Accountable Teams

Ship and Reflect —Responsive & Able

to Reorient Quickly


Agile needn’t be complicated

Intake

1st Guiding Principle: Value


Increase Value

Deliver value early and often

Make it easier, better, faster, cheaper for customers

Engage employees to: • Increase Safety• Increase Morale• Reduce Cost• Improve Productivity

2nd Guiding Principle: Flow


Improve Flow

Optimize end-to-end flow for faster delivery

Minimize delays, queues, non-value add overhead

Tactics: • Visualize & limit WIP• Pull work• Reduce batch size and

manage queue lengths

3rd Guiding Principle: Quality


Enhance QualityDiscover

quality through fast feedback

Create useful, usable, desirable products

Tactics: • Focus (reduce WIP)• Accelerate feedback• Collaborate• Prevent defects (rework)

Lean-Agile Transformation Objectives

Visibility

Predictability

Time to Market

Value / Outcome Driven

Enterprise Agility

Become proficient at one step before ascending to the next


Different goals at different times in your transformation

Your goal is not to do a set of practices, but to improve the way you do what you do.

Alex Singh

PROCESS IMPROVEMENTS DRIVE OUT WASTE AND INCREASE EFFECTIVENESSImprove your process


Practices are designed to solve a problem in a context. When challenges occur, look at the intention of the practice. Then, using principles, look for an alternative to the practice.Finally, decide what to do next, but define the expected outcomes before starting the experiment (or implementing the countermeasure)

Alex Singh & Al Shalloway

BUT FIRST GRASP THE PURPOSE OF THE PRACTICEPractices Shouldn’t be Inviolable!


Success is the ability to …

Continue improvements independently:• Holistically see problems and inefficiencies • Determine root causes and fix them

permanently• Share knowledge, ideas, practices laterally


Gaming Company

22

AGILE JOURNEY

(Platform) At the start …

Goal: Use the cloud to allow remote monitoring of, and deployment to, slot machines on a casino floor.

Copyright © 2012-2017, OrgAgility. All Rights Reserved. 23JH

Where’s the beef?

Investment in architecture must provide some benefit in the short to medium term

for buy-in and continued funding.Image source: https://xkcd.com/974/

JH

Top Challenges

1. Non-collaborative culture

2. Poor technical leadership

3. Lack of “production” mindset

25Copyright © 2012-2017, OrgAgility. All Rights Reserved. JH

(Platform) At the start …

• 18-months and no value• No end in sight


I often say that when you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meager and unsatisfactory kind.

William Thomson (Lord Kelvin)

IF YOU CAN NOT MEASURE IT, YOU CAN NOT IMPROVE ITTo measure is to know


(Platform) Summary …

• 3 teams in 3 cities — low trust• Wrong people in wrong roles• No Operations involvement• No DevOps team or mentality• No representation from Legal

and Regulators


Step 1: Visibility (Platform)

Deliver one sliver of functionality cleanly end-to-end to learn about:• Process — activities, flow, and

blockages• Teamwork — structure and

swarming• Intra-team collaboration


Step 2: Requirement Clarity (Platform)

• Agile users stories — defined outcomes and tasks for all work

• Relative story point estimation• Single master backlog

– Component teams implement related features in concert


What is a Story?Describes functionality valuable to a stakeholder of a system or product.

A short description of what the product should do, emphasizing user goals(not system attributes).

As an <user type>I want to <do something>So that I can <accomplish something of value>

I’ll know this is done when• A• B• C


Step 3: Build Trust (Platform)

• Work together in time boxes– Short-term goals—build vertical

slices of same functionality– Cross-city pairing– Attend each others’ planning

and review sessions

• Zero defects acceptedCopyright © 2012-2017, OrgAgility. All Rights Reserved. 32

The Other Problem

Marketing Product Management

Development Support

The Other Problem

even though our value flows this way

We manage this way

The Other Problem

IT, integration engineering, systems engineering, systems support, operations:

• Overlapping interests• No clearly defined ownership


Typical Engineering Skillset distribution

36

0

1

2

3

4

5

6

7

8

9Open Source Tools

Open Source ProgrammingCoding

Scripting

Test Driven Development

Unit Testing

Automation Tools

Data Management Skills

Business Focused

Measurement and Metrics

Continual Integration

Continual Deployment

Continual Delivery

Backup/RestoreDisaster Recovery

Failover ClusteringSystems HA

Systems Scaling

Performance Tuning

Hardware Management

Hypervisor Management

Systems Monitoring

Application Monitoring

Systems Recovery

Traffic Management

DDOS Mitigation

Orchestration

Bare Metal Deployment

Change ControlRelease Management

Engineering

PrimaryFocus:DrivechangeDriveinnovation

Copyright © 2012-2017, OrgAgility. All Rights Reserved. JHJH

Typical Operations Skillset distribution

37

0

1

2

3

4

5

6

7

8

9Open Source Tools


Scripting


Unit Testing

Automation Tools


Business Focused




Continual Delivery



Systems Scaling

Performance Tuning

Hardware Management


Systems Monitoring


Systems Recovery

Traffic Management

DDOS Mitigation

Orchestration



Operations

PrimaryFocus:ProvidestabilityProvidereliability

Copyright © 2012-2017, OrgAgility. All Rights Reserved. JH

Typical Support Skillset distribution

38

0

1

2

3

4

5

6

7

8

9Open Source Tools


Scripting


Unit Testing

Automation Tools


Business Focused




Continual Delivery



Systems Scaling

Performance Tuning

Hardware Management


Systems Monitoring


Systems Recovery

Traffic Management

DDOS Mitigation

Orchestration



Systems Support

PrimaryFocus:ProvideconsistencyProvideconfidence


DevOps — Multidiscipline Skillset

39

0

1

2

3

4

5

6

7

8

9Open Source Tools


Scripting


Unit Testing

Automation Tools


Business Focused




Continual Delivery



Systems Scaling

Performance Tuning

Hardware Management


Systems Monitoring


Systems Recovery

Traffic Management

DDOS Mitigation

Orchestration



DevOps

PrimaryFocus:CreaterevenueMeetexpectations


The Other Problem


DEV OPS

Production Releases

Change Requests

The Other Problem

• Production releases considered risky• Strained collaboration between

development and operations• Development not appreciating the

realities of operations• Operations not understanding the new

development strategies


What we wanted …

1. Streamline the end-to-end flow2. Reduce the feedback cycle3. Flexible people4. Multidisciplinary “generalizing specialists”5. Standardized infrastructure6. Automation and tools7. Standardized development guidelines8. You build it, you run it


Step 4: Build a DevOps Culture


DEV OPS

Extend delivery to production

Timely operations feedback to teams

Embed project knowledge into Operations

Embed Operations knowledge into projects

“The DevOps movement addresses the dysfunction that results from organizations composed of functional silos. Thus, creating another functional silo that sits between Development and Operations is clearly a poor (and ironic) way to try and solve these problems. DevOpsproposes instead strategies to create better collaboration between functional silos, or doing away with the functional silos altogether and creating cross-functional teams (or some combination of these approaches).”

44Source: http://continuousdelivery.com/2012/10/theres-no-such-thing-as-a-devops-team/

Jez Humble


Goals:

• Automate deployment – existing Windows applications

• Repeatability, supportability, reliability



Business Quality

IT Operations Development

CALMS



• Culture

• Automation

• Lean

• Measurements

• SharingJohn Willis and Damon Edwards 47


Goals:

• Improve other teams • not become another silo!

• Foster deep communication



• Sit together + AV bridge

• Attitude not just skill

• AmbassadorsCopyright © 2012-2017, OrgAgility. All Rights Reserved. 49

Step 5: DevOps Requirements

• Agile users stories• Story point estimates• Add functionality iteratively–Start with a basic blueprint–Augment & build on functionality


Step 5: DevOps Requirements

• Monthly goals• Next most important 2-3 blueprints• Complete blueprints in a week

(longer ones were rare but OK)



Blueprint Flow

Blue

prin

t

53

Blue

prin

t

54

Step 6: DevOps Teamwork

56Image Source: http://andon2013.blogspot.com/2013/11/conclusiones.html

Swarm on blueprints (i.e., work)Face-to-face discussions



• Partnership on the goals

• Group accountabilityIT, integration engineering, systems engineering, systems support, operations


• Quick daily stand-up• Knowledge sharing– Blogs– Technical demos

• Improvement backlog



• Never go home angry• Have fun– Poker daily– Shooting range frequently

• Snacks, donuts, weekly team lunch



60Image Source: http://andon2013.blogspot.com/2013/11/conclusiones.html

Problem Swarming:• Abnormality• Visualize problem• Positive & timely

response• Joint PDCA• Back to standard

Step 7: DevOps Holistic View

• Min and Max limits on work process steps– Lowered gradually to increase flow and teamwork– No feast, no famine

• Step out of your comfort zone• Stories can exit process step only when exit

criteria are met (done)• Follow the work sequencing guidance


DevOps Work Priority Guidance

1. Fix an unhealthy increment (broken build, recently introduced defects, etc.)

2. Help remove a flag (Block)3. Help an existing swarm move their work forward4. Refill a queue (if low before the bottleneck) but

don’t exceed the WIP Limit5. Start a new item. However, if all at limits, join an

existing swarm


Step 8: Integrate Platform + DevOps

Goal: end-to-end development and delivery• Developed, locally tested, integrated,

validated, and deployed• Platform stories and DevOps blueprints in

concert• Platform team representation on DevOps


Step 8: Integrate Platform + DevOps

• Feedback from business on completion of each platform story and DevOps blueprint

• Friday demos — show everything worked on that week– Team only first; stakeholders after 3 weeks,

real customers in attendance after 2-months– Tie everything back to a business goal — tell a

story


Results


0

1

2

3

4

5Identical Environments

Repeatability


Fabric Monitoring

Application Logging

Fabric Logging

Code Documentation

API Documentation

Automated Functional Test

ProvisioningDeployment

Min Downtime

HA

Release Management

Dependency Management

Auto Build

Versioning

Source Control

Branching Strategy

Load Test

Continuous Delivery Roadmap - Platform

JH

Results

• Reduced time to market• Reduced cycle time• Higher customer satisfaction (via NPS)• Reduced mean time to restore service

(MTTRS)• Increased “sharing”• Increased morale


SowhatabouttheTech???

JH

Agileisn’t…….

• Aboutthe tools youuse

• Aboutthecloud youdeployinto

• Aboutpleasingmanagement

• Aboutdoingsomethingcool

JH

Agileis……Simply– Agile isn’ttheGoal:BetteristheGoal!

GettingsomethingintoProduction:• Thatisvaluable• Thatishighquality• Thatisreliable• Thatismaintainable• Hasfewer bugs

JH

ToolsvsMethodsvsProcesses

Adoptingtoolswill:- Showyouwhereyouarebroken,faster- Automatebad processes- Forces youtoworkinawaythetool ‘wants’to

Toolsshouldbeadoptedwhen:- Youhavea‘good’ processthatneedstobespedup- Youwantarepeatable ‘good’process- Youwanttofind outwherethingsarebroken,faster

JH

SoWheredoesAWSfit?

BuiltforScale– nowaitingfor‘environments’

Automation – consistencyoverspeed

Pay-as-you-go – ReducedCapex,speedofsetup

JH

SpeedinguptheFeedbackLoop

AWSCodePipeline – CIWorkflow

AWSCodeBuild – compile,build,text

AWSCodeDeploy – automateddeployment

AWSCodeCommit – ManagedGITrepositories

JH

InfrastructureasCode

AWSCloudformation – create,manage,describe

AWSOpsWorks – ManagedChefService

JH

Compliance?

AWSEC2SystemsManager– track,audit,patch

AWSConfig – RulesbasedVerificationandNotify

AWS CloudTrail – APIActivityUsageandTracking

JH

Micro-services?

AWSECS – Docker,portability

AWSLambda – NoOps,FaaS,TaaS

JH

Is this item:• The most valuable

thing to be working on?

• Going to solve a real customer need?

• Necessary? Necessary now?

• Desirable, valuable, feasible, viable?• How do we know

that?• Going to help us learn

and improve?

Will this action:• Help us finish and deliver

stories faster? How so?• Enable smoother

uninterrupted flow of value?• Improve how

customer/supplier pairs function?

• Simplify our process?• Help us inspect and adapt

sooner?• Improve flow but sacrifice

focus on value delivery?• Eliminate waste but impede

flow?

Are we:• Negotiating features

instead of quality?• Sacrificing quality for

expediency?• Exposing wastes,

inefficiencies, weaknesses, quickly?

• Solving the real problem and tackling the root cause(s)?

• Accelerating feedback?• Preventing the

introduction of defects

Guiding Questions

Agile & DevOps : Gaming - HATech DevOps Services, a ... · “The DevOps movement addresses the...

Documents

Transcript of Agile & DevOps : Gaming - HATech DevOps Services, a ... · “The DevOps movement addresses the...