© 2011 CTG, Inc.

Playing around with Risks

Jurgen Cleuren

Nov. 24th 2011

Introduction

• Projects are done in a probabilistic environment

– Incomplete information

– Parameters change over time

– What is true in the beginning of the project, can be false some time

later

• Games in a probabilistic environment

– Incomplete information (cardgames,…)

– Random element (dice, cards,…)

Which games ?

• Poker

• Monopoly

• Backgammon

Succes:

First learn the rules, then play better than everyone else

- Albert Einstein -

• Rules -> requirements

• Every game starts with learning and agreeing on the rules -> every project

starts with defining and agreeing on the requirements

TEXAS HOLD’EM POKER

Rules of Texas Hold’em

• 2 Blind cards

• Betting round

• 3 Open Community cards (flop)

• Betting round

• 1 Open Community card (Turn)

• Betting Round

• 1 Open Community Card (River)

• Betting Round

• Best hand of 5 cards wins

Hand Ranking

• Highest Card

• Pair

• 2 Pair

• 3 of a kind

• Straight (5 consecutive cards)

• Flush (5 cards of the same suit)

• Full House (3 and 2 )

• 4 of kind

• Straight Flush (5 consecutive cards of the same suit)

General test flow vs Poker round

• Create FTT/Risk Matrix

• Software Delivery

• Full functional test

• Bugfixing

• New Software Delivery

• Retesting and regression

• Bugfixing

• Final Software Version

• Regression Test

• Get Blind cards – evaluate risk - Bet

• Flop (3 cards)

• Re-evalute hand and Risk

• Betting round

• Turn (1 card)

• Re-evaluate hand and Risk

• Betting round

• River (1 card)

• Final bets and showdown

Similarities / Differences

3 Recurring phases

1 big phase (Complete Functional testing vs Flop)

2 small phases (retest + regression vs Turn + River)

Determine Risk before 1st test run (Risk Matrix or FTT vs 1st bet)

Adapt Risk Matrix or FTT after each test run

Should the FTT or Risk Matrix be adapted ?

• Every Test run gives new information

• Likelihood of risks change

– Failed test cases get a higher likelihood

– Passed test cases in unchanged code have a lower likelihood

• Closer to deadline => Risks can change

• Reporting is more clear

– Each Risk Matrix/FTT tree is a snapshot of the project at a certain time

Who ?

• Test manager should take the lead

• Preferably Project Board

• Test manager can do it by himself, but the board should at least be aware

that this activity is done

Justification

• A Poker hand needs justification at any point in the game

– Having the best hand in the beginning doesn’t imply that you are going

to win

– You must be prepared to fold when you are not winning anymore

– The money you’ve already bet doesn’t count

• It is in the pot so it is not your money anymore

• Don’t put more money in a losing hand

– No justification anymore: get out of the hand

– Cut your losses

Project justification

• A project needs justification at any point in time

• During testing: justification after each test run

– New information is given

– Risks are updated

• No Justification means project should be closed

• Previous investments do not count

– Don’t put more money in a failing project

• Cut your losses

Test process justification

• Get Blind cards – evaluate risk - Bet

• Flop (3 cards)

• Re-evalute hand and Risk

• Betting round

• Turn (1 card)

• Re-evaluate hand and Risk

• Betting round

• River (1 card)

• Final bets and showdown

• Create FTT/Risk Matrix

• Software Delivery

• Full functional test

• Justification

• Bugfixing

• New Software Delivery

• Retesting and regression

• Justification

• Bugfixing

• Final Software Version

• Regression Test

• Justification

Result Oriented Thinking

• A decision can be right even if the outcome is not favourable

• Focus more on the decision and the ‘why’ instead of the result

– Tester A completes a test set in 4 days by skipping tests

– Tester B completes the same test set in 6 days through thoroughly

testing

– No defects were discovered in production

– Who would you reward the most ?

Thought process

• In poker, to anticipate and understand your opponents actions, you need

to think as your opponent and not as you in his situation

• What motivates you, does not necessarily motivate another person

• Successful people ask better questions

– WHY? is more important than HOW? or WHO?

“Someone who knows HOW will always have a job

Someone who knows WHY will always be his boss”

Luck

“ Luck is when preparation meets opportunity”

- Seneca –

• Be prepared to get lucky

– In poker, sometimes you need to get lucky. When you get lucky, be sure

to take a big pot out of it.

– Sometimes a best case scenario happens, but we need to take

advantage of it.

– Being ahead of planning is more valuable if you can actually do

something with this situation

MONOPOLY

Aspects of the game

• Investing in houses

• The higher the investment, the bigger the payoff

• Some spaces have a higher probability

• Random element: dice

Analogy to Risk Based Testing

• Different probability of landing on spaces = Likelihood

• Higher Payoff = Impact

• What can monopoly teach us ?

– What is more important: Impact or Likelihood ?

Impact and likelihood on the board

Winning Monopoly strategies

• Complete Orange Colour group and invest as soon as possible

– Why Orange ? It has the biggest probability of other players landing on

it

– Be prepared to even trade down to get this colour group

• Complete Red Colour group as a second priority

– Why Red ? It has the second biggest probability of other players

landing on it

What lessons are to be learned

• Likelihood >>>>>> impact

– The weight of Likelihood should be > 50%

– The weight of Impact should be < 50%

BACKGAMMON

Backgammon

Important rules of Backgammon

• Goal: get all your tiles from one end to the other

• Only tiles that are standing alone on a pillar can be captured

• A captured tile has to be brought back in play at the beginning of the

board

• Random element: Dice

Translation to risks

• Your own checkers: Risks

• Opponents checkers: Causes

• Whenever one of your checkers is captured it’s in fact a cause hitting a

risk

• A risk is mitigated when the checker cannot be captured (2 or more on

one pillar)

Translation of risk priority

• Low risk: checker in the first quadrant

• Medium risk: checker in the second or third quadrant

• High risk: checker in the fourth quadrant

What can Backgammon teach us?

• Which risks should be mitigated and which are low priority ?

• There might be a possibility to remove a cause, but in the same way

creating a new risk. Should we do it ?

– Eg: Back-up testmanager vs full time testing

Backgammon Strategies

• Checkers in the 1st quadrant don’t have to be protected. Moving forward

is the better play.

Þ Risks with low priority don’t have to be mitigated. The correcting cost is

usually way less than the mitigation costs

• Checkers in the 4th quadrant need to be protected at all costs.

Þ Risk with high priority need to be mitigated at all costs

• For checkers in the 2nd and 3rd following rule applies: Always take a

chance to capture

Þ If you can remove a cause and therefore create a medium of low risk,

do so

CONCLUSIONS

Conclusions

• What Poker taught us:

– Adapt FTT/Risk tree after each test run

– Priorities of test items can change

– Justification has to be done after each test run

– Don’t focus on results, focus on decisions

– Be prepared to get lucky!

Conclusions

• Create FTT/Risk Matrix

• Software Delivery

• Full functional test

• Bugfixing

• New Software Delivery

• Retesting and regression

• Bugfixing

• Final Software Version

• Regression Test

• Create FTT/Risk Matrix

• Software Delivery

• Full functional test

• Adapt FTT/Risk Matrix

• Justification

• Bugfixing

• New Software Delivery

• Retesting and regression

• Adapt FTT/Risk Matrix

• Justification

• Bugfixing

• Final Software Version

• Regression Test

• Adapt FTT/Risk Matrix

• Justification

Conclusions

• Monopoly

– Likelihood >>>>> Impact

• Backgammon

– Don’t mitigate low priority risk

– Always mitigate high priority risks

– Removing a cause and creating a medium or low risk is the way to play

QUESTIONS AND ANSWERSJurgen Cleuren (jurgen.cleuren@ctg.com)